Compare commits
28
Commits
de19244ab8
..
master
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
0aae2da92f | ||
|
|
8e23231dc3 | ||
|
|
80b82121a1 | ||
|
|
77f817d402 | ||
|
|
6a97e403ba | ||
|
|
c76af43b7c | ||
|
|
ced322123a | ||
|
|
de1e81d9b0 | ||
|
|
c27bdbf026 | ||
|
|
9798f0541a | ||
|
|
cbfd634f0d | ||
|
|
942aecf27c | ||
|
|
922e7a2a61 | ||
|
|
347f2bec14 | ||
|
|
ad1d96290c | ||
|
|
ce21d8724e | ||
|
|
8a56e9726f | ||
|
|
3ab82509a0 | ||
|
|
6447a74e0c | ||
|
|
b350b5bda9 | ||
|
|
83476dded2 | ||
|
|
b1a631f8b4 | ||
|
|
684cb608e2 | ||
|
|
0b19a75d7b | ||
|
|
2caa383f06 | ||
|
|
912ae30ff0 | ||
|
|
c84f11cd8f | ||
|
|
df6c39ba56 |
@@ -36,3 +36,7 @@ build/
|
||||
|
||||
### Mac OS ###
|
||||
.DS_Store
|
||||
/.idea/encodings.xml
|
||||
|
||||
# 本地运行与测试日志
|
||||
run.out
|
||||
|
||||
+182
@@ -0,0 +1,182 @@
|
||||
# storageNode 项目结构
|
||||
|
||||
## 概述
|
||||
分布式存储/下载节点服务端,用于画廊网站。职责包括:从远程源下载图片集、压缩为 ZIP 归档、通过 HTTP 提供下载(支持断点续传)、管理代理订阅配置(V2Ray/Clash),并通过 Netty 自定义 TCP 协议与中心服务器通信。
|
||||
|
||||
- **Group ID:** `org.lion`
|
||||
- **Version:** `1.0`
|
||||
- **Java 版本:** 21 字节码(生产使用 GraalVM JDK 25)
|
||||
- **构建工具:** Maven(单模块)
|
||||
- **编译目标:** 当前以 JVM/JAR + `lib/` 运行;保留 GraalVM 原生配置但尚未完成 JDK 25 原生验证
|
||||
|
||||
---
|
||||
|
||||
## 目录结构
|
||||
|
||||
```
|
||||
storageNode/
|
||||
├── pom.xml # Maven 构建配置
|
||||
└── src/
|
||||
├── main/
|
||||
│ ├── java/
|
||||
│ │ └── lion/
|
||||
│ │ ├── Main.java # 程序入口
|
||||
│ │ ├── storageNode.java # Netty TCP 核心节点
|
||||
│ │ ├── CustomUtil.java # 工具方法(HTTP 通知、端口查找、ObjectMapper)
|
||||
│ │ ├── MultiThreadedHTTPServer.java # HTTP 文件下载服务(端口 8888)
|
||||
│ │ ├── Config/
|
||||
│ │ │ └── Config.java # 加载 config.properties
|
||||
│ │ ├── Domain/
|
||||
│ │ │ └── GalleryTask.java # 下载任务领域模型
|
||||
│ │ ├── ErrorCode/
|
||||
│ │ │ └── ErrorCode.java # 错误码常量
|
||||
│ │ ├── Externel/
|
||||
│ │ │ └── BackupSubServer.java # 代理订阅文件服务(端口 8889)
|
||||
│ │ ├── Message/
|
||||
│ │ │ ├── AbstractMessage.java # 消息基类 + 消息类型常量
|
||||
│ │ │ ├── MessageCodec.java # Netty 编解码器(ByteBuf ↔ AbstractMessage,JSON 格式)
|
||||
│ │ │ └── Main/
|
||||
│ │ │ ├── AvailableCheckMessage.java # 可用性检查(type=8)
|
||||
│ │ │ ├── DeleteGalleryMessage.java # 删除画廊请求(type=3)
|
||||
│ │ │ ├── DownloadPostMessage.java # 提交下载任务(type=1)
|
||||
│ │ │ ├── DownloadStatusMessage.java # 上报任务状态(type=2)
|
||||
│ │ │ ├── IdentityMessage.java # 身份认证握手(type=6)
|
||||
│ │ │ ├── MaintainMessage.java # 心跳维持(type=7)
|
||||
│ │ │ └── ResponseMessage.java # 通用响应(type=0)
|
||||
│ │ └── Service/
|
||||
│ │ ├── DeleteService.java # 删除画廊目录
|
||||
│ │ ├── DownloadCheckService.java # 下载监控与压缩服务
|
||||
│ │ ├── PrimaryChannelTracker.java # 主站通道引用登记与自愈
|
||||
│ │ └── RekickPolicy.java # 主动重新唤起主站的限流判定
|
||||
│ └── resources/
|
||||
│ ├── config.properties # DouNai 订阅地址配置
|
||||
│ ├── simplelogger.properties # SLF4J 日志配置(输出到 run.out)
|
||||
│ └── reflect-config.json # GraalVM 反射配置(Jackson 序列化)
|
||||
└── test/
|
||||
└── java/ # 协议编解码、节点消息处理与上报循环、
|
||||
# 订阅快照与分发、下载/压缩恢复、
|
||||
# 主站通道自愈、删除与工具方法测试
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 关键依赖
|
||||
|
||||
| 依赖 | 版本 | 用途 |
|
||||
|---|---|---|
|
||||
| `io.netty:netty-all` | 4.1.138.Final | TCP 服务端/客户端 |
|
||||
| `com.fasterxml.jackson.core:jackson-databind` | 2.22.2 | JSON 序列化 |
|
||||
| `org.projectlombok:lombok` | 1.18.48 | 简化样板代码(`@Data`, `@Slf4j`) |
|
||||
| `ch.qos.logback:logback-classic` | 1.5.38 | 日志实现 |
|
||||
| `cn.hutool:hutool-all` | 5.8.47 | 文件操作、ZIP 压缩、HTTP 请求 |
|
||||
| `org.apache.commons:commons-compress` | 1.28.0 | 压缩归档 |
|
||||
| `org.graalvm.buildtools:native-maven-plugin` | 1.1.8 | 原生镜像配置(待验证) |
|
||||
|
||||
---
|
||||
|
||||
## 启动流程
|
||||
|
||||
1. `Main.main()` → 调用 `boot()`(遗留的 Netty Bootstrap),然后执行 `Config.loadConfig()`
|
||||
2. 在新线程中启动 `BackupSubServer`(端口 8889)— 提供代理订阅文件下载
|
||||
3. 在新线程中启动 `MultiThreadedHTTPServer`(端口 8888)— 提供画廊 ZIP 文件下载
|
||||
4. 主线程创建 `storageNode()` 实例(阻塞构造函数,永不返回):
|
||||
- 从端口 26321 开始查找空闲端口,绑定 Netty TCP 服务端
|
||||
- 作为 TCP 客户端连接 `lionwebsite.xyz:26322~26342`,唤醒中心服务器
|
||||
- 启动 `DownloadCheckService` 和 5 秒定时任务 `mainThread()`
|
||||
|
||||
---
|
||||
|
||||
## 通信协议(Netty TCP)
|
||||
|
||||
自定义协议格式:`[1字节类型] + [4字节长度] + [JSON 负载]`
|
||||
|
||||
| 类型字节 | 消息类 | 方向 |
|
||||
|---|---|---|
|
||||
| 0 | `ResponseMessage` | 响应 |
|
||||
| 1 | `DownloadPostMessage` | 服务端 → 节点 |
|
||||
| 2 | `DownloadStatusMessage` | 节点 → 服务端 |
|
||||
| 3 | `DeleteGalleryMessage` | 服务端 → 节点 |
|
||||
| 6 | `IdentityMessage` | 握手 |
|
||||
| 7 | `MaintainMessage` | 心跳 |
|
||||
| 8 | `AvailableCheckMessage` | 可用性检查 |
|
||||
| 9 | `SubscriptionSnapshotMessage` | 主站 → 节点,完整订阅备机快照 |
|
||||
|
||||
---
|
||||
|
||||
## HTTP 服务
|
||||
|
||||
### MultiThreadedHTTPServer(端口 8888)
|
||||
- 提供压缩后的画廊 ZIP 文件下载
|
||||
- 仅接受来自 `lionwebsite.xyz` IP 的连接
|
||||
- 支持 HTTP 206 Partial Content(断点续传)
|
||||
- 参数:`AuthCode`(管理员访问)、`gid`(画廊 ID)
|
||||
|
||||
### BackupSubServer(端口 8889)
|
||||
- 提供 V2Ray 和 Clash 代理订阅文件
|
||||
- 不直接访问上游;主站完成下载和倍率过滤后,通过 Netty 类型 9 推送完整快照
|
||||
- 按公开 Key 的 SHA-256 查找用户绑定的子账号,未知 Key 返回 404
|
||||
- 快照经 SHA-256 和 HMAC 校验后原子落盘,主站离线时继续分发最后成功版本
|
||||
- 文件存储于 `sub/snapshots/{revision}/accounts/{accountId}/`
|
||||
- 没有有效快照或快照超过最大有效期时返回 503,不回退旧共享订阅
|
||||
- `GET /health/subscription` 提供不含 Key 和订阅正文的快照状态
|
||||
|
||||
---
|
||||
|
||||
## 核心服务
|
||||
|
||||
### DownloadCheckService
|
||||
- 扫描下载目录(`/root/gallery/hentai/download/`)监控进度
|
||||
- 通过检测 `galleryinfo.txt` 文件判断下载完成
|
||||
- 将完成的下载任务移入压缩队列
|
||||
- 后台线程每 5 秒执行 ZIP 压缩,完成后删除源目录
|
||||
|
||||
### DeleteService
|
||||
- 按名称删除画廊目录
|
||||
- 失败时返回 `ErrorCode.IO_ERROR` 或 `ErrorCode.FILE_NOT_FOUND`
|
||||
|
||||
### PrimaryChannelTracker
|
||||
- 登记「哪条通道是主站」,供任务状态上报与心跳使用
|
||||
- 只有主站会发的消息类型(身份/任务下发/探活/订阅快照)才可用于认领引用;
|
||||
备机的身份消息与节点自己的出站类型都不算证据
|
||||
- 首选通道断开时立即回退到其它仍可用的已认证通道;引用被清空时,
|
||||
已认证通道上的下一条消息即可恢复它,避免任务状态静默停止上报
|
||||
|
||||
### RekickPolicy
|
||||
- 判定「有待上报任务、却无可用通道」时是否应主动重新唤起主站
|
||||
- 以 30 秒为最小间隔限流,既能快速自愈,又不会在主站确实离线时形成重连风暴
|
||||
|
||||
---
|
||||
|
||||
## 配置说明
|
||||
|
||||
### config.properties(从 `/root/gallery/storageNode/config.properties` 加载)
|
||||
```properties
|
||||
SubscriptionSyncEnabled=false
|
||||
SubscriptionSyncSecret=
|
||||
SubscriptionDataDir=/root/gallery/storageNode/sub
|
||||
SubscriptionMaxStaleSeconds=604800
|
||||
SubscriptionMaxPayloadBytes=52428800
|
||||
SubscriptionHttpPort=8889
|
||||
SubscriptionHttpWorkers=4
|
||||
SubscriptionSocketTimeoutMs=10000
|
||||
```
|
||||
|
||||
生产同步密钥优先通过 `SUBSCRIPTION_SYNC_SECRET` 环境变量提供,不得提交到仓库或写入日志。
|
||||
|
||||
### simplelogger.properties
|
||||
- 日志级别:`info`
|
||||
- 时间戳格式:`yyyy-MM-dd HH:mm:ss`
|
||||
- 输出文件:`run.out`
|
||||
|
||||
---
|
||||
|
||||
## 架构说明
|
||||
|
||||
- 单模块 Maven 项目,现有 62 个测试用例:协议编解码(MessageCodec)、节点消息处理与
|
||||
状态上报循环、备机订阅分发 HTTP(8889)、订阅快照存取、下载/压缩恢复与 gid 匹配、
|
||||
主站通道自愈、删除与工具方法
|
||||
- 硬编码文件系统路径(`/root/gallery/...`)→ 仅限 Linux 部署
|
||||
- 外部连接:`lionwebsite.xyz`、`personal.lionwebsite.xyz`、`aaaa.gay`
|
||||
- GraalVM 原生镜像编译,包含 Jackson 反射配置
|
||||
- 大量使用 Lombok(`@Data`、`@Slf4j`)
|
||||
- 使用 Hutool 工具库处理文件/ZIP/HTTP 操作
|
||||
@@ -9,65 +9,102 @@
|
||||
<version>1.0</version>
|
||||
|
||||
<properties>
|
||||
<maven.compiler.source>21</maven.compiler.source>
|
||||
<maven.compiler.target>21</maven.compiler.target>
|
||||
<maven.compiler.release>21</maven.compiler.release>
|
||||
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
|
||||
</properties>
|
||||
|
||||
<dependencies>
|
||||
<dependency>
|
||||
<groupId>io.netty</groupId>
|
||||
<artifactId>netty-all</artifactId>
|
||||
<version>4.1.101.Final</version>
|
||||
<!--
|
||||
只保留实际用到的模块。原先的 netty-all 是聚合 pom,会拖进 35 个模块
|
||||
(含 osx/aarch64/riscv64 的 epoll/kqueue native、codec-http3/mqtt/redis/
|
||||
smtp/stomp/xml/protobuf、transport-rxtx/sctp/udt 等),
|
||||
而本节点只用 ServerBootstrap/Bootstrap/NioEventLoopGroup/
|
||||
NioServerSocketChannel/NioSocketChannel/ByteBuf/ByteToMessageCodec/
|
||||
LengthFieldBasedFrameDecoder。
|
||||
transport 提供 Bootstrap/EventLoop/Channel,codec 提供
|
||||
ByteToMessageCodec 与 LengthFieldBasedFrameDecoder。
|
||||
buffer/common/resolver 由它们传递引入。这里不用 handler:
|
||||
节点侧未使用 LoggingHandler 等 handler 模块的类。
|
||||
-->
|
||||
<artifactId>netty-transport</artifactId>
|
||||
<version>4.1.138.Final</version>
|
||||
</dependency>
|
||||
|
||||
<dependency>
|
||||
<groupId>io.netty</groupId>
|
||||
<artifactId>netty-codec</artifactId>
|
||||
<version>4.1.138.Final</version>
|
||||
</dependency>
|
||||
|
||||
<dependency>
|
||||
<groupId>com.fasterxml.jackson.core</groupId>
|
||||
<artifactId>jackson-databind</artifactId>
|
||||
<version>2.15.2</version>
|
||||
<version>2.22.2</version>
|
||||
</dependency>
|
||||
|
||||
<dependency>
|
||||
<groupId>org.projectlombok</groupId>
|
||||
<artifactId>lombok</artifactId>
|
||||
<version>1.18.30</version>
|
||||
<version>1.18.48</version>
|
||||
</dependency>
|
||||
|
||||
<dependency>
|
||||
<groupId>org.slf4j</groupId>
|
||||
<artifactId>slf4j-api</artifactId>
|
||||
<version>2.0.9</version>
|
||||
<version>2.0.19</version>
|
||||
</dependency>
|
||||
|
||||
|
||||
<dependency>
|
||||
<groupId>org.slf4j</groupId>
|
||||
<artifactId>slf4j-simple</artifactId>
|
||||
<version>2.0.7</version>
|
||||
<groupId>ch.qos.logback</groupId>
|
||||
<artifactId>logback-classic</artifactId>
|
||||
<version>1.5.38</version>
|
||||
</dependency>
|
||||
|
||||
|
||||
<dependency>
|
||||
<groupId>cn.hutool</groupId>
|
||||
<artifactId>hutool-all</artifactId>
|
||||
<version>5.8.26</version>
|
||||
<version>5.8.47</version>
|
||||
</dependency>
|
||||
|
||||
<dependency>
|
||||
<groupId>org.apache.commons</groupId>
|
||||
<artifactId>commons-compress</artifactId>
|
||||
<version>1.25.0</version>
|
||||
<version>1.28.0</version>
|
||||
</dependency>
|
||||
|
||||
<dependency>
|
||||
<groupId>org.apache.httpcomponents</groupId>
|
||||
<artifactId>httpclient</artifactId>
|
||||
<version>4.5.14</version>
|
||||
<groupId>org.junit.jupiter</groupId>
|
||||
<artifactId>junit-jupiter</artifactId>
|
||||
<version>5.14.4</version>
|
||||
<scope>test</scope>
|
||||
</dependency>
|
||||
</dependencies>
|
||||
|
||||
<build>
|
||||
<plugins>
|
||||
<plugin>
|
||||
<groupId>org.apache.maven.plugins</groupId>
|
||||
<artifactId>maven-surefire-plugin</artifactId>
|
||||
<version>3.2.5</version>
|
||||
</plugin>
|
||||
<plugin>
|
||||
<groupId>org.apache.maven.plugins</groupId>
|
||||
<artifactId>maven-compiler-plugin</artifactId>
|
||||
<version>3.13.0</version>
|
||||
<configuration>
|
||||
<annotationProcessorPaths>
|
||||
<path>
|
||||
<groupId>org.projectlombok</groupId>
|
||||
<artifactId>lombok</artifactId>
|
||||
<version>1.18.48</version>
|
||||
</path>
|
||||
</annotationProcessorPaths>
|
||||
</configuration>
|
||||
</plugin>
|
||||
<!-- <plugin>-->
|
||||
<!-- <groupId>org.apache.maven.plugins</groupId>-->
|
||||
<!-- <artifactId>maven-assembly-plugin</artifactId>-->
|
||||
@@ -94,21 +131,21 @@
|
||||
<plugin>
|
||||
<groupId>org.graalvm.buildtools</groupId>
|
||||
<artifactId>native-maven-plugin</artifactId>
|
||||
<version>0.9.28</version>
|
||||
<version>1.1.8</version>
|
||||
<configuration>
|
||||
<mainClass>lion.Main</mainClass>
|
||||
<imageName>storageNode</imageName>
|
||||
<buildArgs>
|
||||
<arg>-H:+ReportExceptionStackTraces</arg>
|
||||
<arg>--gc=G1</arg>
|
||||
<arg>--enable-url-protocols=https</arg>
|
||||
<arg>-H:IncludeResources="simplelogger.properties"</arg>
|
||||
<arg>--initialize-at-build-time=org.slf4j.simple.SimpleLogger,org.slf4j.simple.SimpleLoggerFactory</arg>
|
||||
<arg>-H:IncludeResources="logback.xml"</arg>
|
||||
<arg>--initialize-at-build-time=ch.qos.logback.classic,ch.qos.logback.core,ch.qos.logback.classic.pattern,ch.qos.logback.core.pattern</arg>
|
||||
<arg>-H:ReflectionConfigurationFiles=src/main/resources/reflect-config.json</arg>
|
||||
</buildArgs>
|
||||
<metadataRepository>
|
||||
<enabled>true</enabled>
|
||||
</metadataRepository>
|
||||
|
||||
</configuration>
|
||||
</plugin>
|
||||
</plugins>
|
||||
|
||||
@@ -3,21 +3,80 @@ package lion.Config;
|
||||
import java.io.FileInputStream;
|
||||
import java.io.IOException;
|
||||
import java.io.InputStream;
|
||||
import java.nio.file.Files;
|
||||
import java.nio.file.Path;
|
||||
import java.util.Properties;
|
||||
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
|
||||
@Slf4j
|
||||
public class Config {
|
||||
public static String DouNaiV2ray;
|
||||
public static String DouNaiClash;
|
||||
public static final String CONFIG_PATH = "/root/gallery/storageNode/config.properties";
|
||||
|
||||
public static boolean subscriptionSyncEnabled;
|
||||
public static String subscriptionSyncSecret;
|
||||
public static String subscriptionDataDir;
|
||||
public static long subscriptionMaxStaleSeconds;
|
||||
public static int subscriptionMaxPayloadBytes;
|
||||
public static int subscriptionHttpPort;
|
||||
public static int subscriptionHttpWorkers;
|
||||
public static int subscriptionSocketTimeoutMs;
|
||||
|
||||
/** 下载服务的管理员口令;为空表示 8888 的管理员直取文件能力关闭。 */
|
||||
public static String adminDownloadCode = "";
|
||||
|
||||
/** 管理员直取文件时允许访问的根目录,越界一律拒绝。 */
|
||||
public static String adminDownloadRoot = "/root/gallery/gallery";
|
||||
|
||||
public static void loadConfig(){
|
||||
loadConfig(CONFIG_PATH);
|
||||
}
|
||||
|
||||
/**
|
||||
* 从指定文件加载配置。
|
||||
*
|
||||
* <p>拆出带路径的版本是为了能直接对「默认值填充」和「启用同步却没配密钥就拒绝启动」
|
||||
* 写断言,不必依赖生产路径存在。文件缺失不是致命错误(保留内置默认值),
|
||||
* 但会记录明确日志,避免以错误配置静默运行。
|
||||
*/
|
||||
public static void loadConfig(String path){
|
||||
Properties prop = new Properties();
|
||||
|
||||
try (InputStream input = new FileInputStream("/root/gallery/storageNode/config.properties")) {
|
||||
prop.load(input);
|
||||
DouNaiV2ray = prop.getProperty("DouNaiV2ray");
|
||||
DouNaiClash = prop.getProperty("DouNaiClash");
|
||||
} catch (IOException ex) {
|
||||
ex.printStackTrace();
|
||||
if (!Files.isRegularFile(Path.of(path))) {
|
||||
log.warn("配置文件不存在:{},使用内置默认值", path);
|
||||
apply(prop);
|
||||
return;
|
||||
}
|
||||
|
||||
try (InputStream input = new FileInputStream(path)) {
|
||||
prop.load(input);
|
||||
} catch (IOException ex) {
|
||||
log.error("加载配置失败:{}", ex.getMessage());
|
||||
apply(prop);
|
||||
return;
|
||||
}
|
||||
apply(prop);
|
||||
}
|
||||
|
||||
private static void apply(Properties prop){
|
||||
subscriptionSyncEnabled = Boolean.parseBoolean(value(prop, "SubscriptionSyncEnabled", "false"));
|
||||
subscriptionSyncSecret = System.getenv().getOrDefault("SUBSCRIPTION_SYNC_SECRET",
|
||||
value(prop, "SubscriptionSyncSecret", ""));
|
||||
subscriptionDataDir = value(prop, "SubscriptionDataDir", "/root/gallery/storageNode/sub");
|
||||
subscriptionMaxStaleSeconds = Long.parseLong(value(prop, "SubscriptionMaxStaleSeconds", "604800"));
|
||||
subscriptionMaxPayloadBytes = Integer.parseInt(value(prop, "SubscriptionMaxPayloadBytes", "52428800"));
|
||||
subscriptionHttpPort = Integer.parseInt(value(prop, "SubscriptionHttpPort", "8889"));
|
||||
subscriptionHttpWorkers = Integer.parseInt(value(prop, "SubscriptionHttpWorkers", "4"));
|
||||
subscriptionSocketTimeoutMs = Integer.parseInt(value(prop, "SubscriptionSocketTimeoutMs", "10000"));
|
||||
// 口令不再写死在源码里:优先取环境变量,其次取配置文件,都为空即关闭该能力。
|
||||
adminDownloadCode = System.getenv().getOrDefault("STORAGE_DOWNLOAD_ADMIN_CODE",
|
||||
value(prop, "AdminDownloadCode", ""));
|
||||
adminDownloadRoot = value(prop, "AdminDownloadRoot", "/root/gallery/gallery");
|
||||
if (subscriptionSyncEnabled && subscriptionSyncSecret.isBlank())
|
||||
throw new IllegalStateException("启用订阅同步时必须配置 SUBSCRIPTION_SYNC_SECRET");
|
||||
}
|
||||
|
||||
private static String value(Properties prop, String key, String fallback) {
|
||||
return prop.getProperty(key, fallback).trim();
|
||||
}
|
||||
}
|
||||
|
||||
@@ -3,26 +3,111 @@ package lion;
|
||||
import cn.hutool.http.HttpRequest;
|
||||
import cn.hutool.http.HttpResponse;
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
import lombok.Data;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
|
||||
import java.util.concurrent.atomic.AtomicInteger;
|
||||
import java.io.*;
|
||||
import java.net.ServerSocket;
|
||||
import java.net.Socket;
|
||||
import java.net.SocketException;
|
||||
import java.net.URLEncoder;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
|
||||
|
||||
@Data
|
||||
@Slf4j
|
||||
public class CustomUtil {
|
||||
|
||||
public static AtomicInteger counter = new AtomicInteger();
|
||||
|
||||
public static ObjectMapper objectMapper = new ObjectMapper();
|
||||
|
||||
public static void notifyMe(String message) {
|
||||
String url = "https://personal.lionwebsite.xyz/message2me?AuthCode=alone&message=" + message;
|
||||
HttpRequest request = HttpRequest.post(url);
|
||||
// 画廊名可能含空格、'&'、'#' 或非 ASCII,必须编码后再拼进查询串,否则会截断或串参数。
|
||||
String url = "https://personal.lionwebsite.xyz/message2me?AuthCode=alone&message="
|
||||
+ URLEncoder.encode(message == null ? "" : message, StandardCharsets.UTF_8);
|
||||
// 外呼必须有超时;调用方虽有专用守护线程,但不设超时仍可能长期占住它。
|
||||
HttpRequest request = HttpRequest.post(url).timeout(5_000);
|
||||
request.header("User-Agent", "Mozilla/5.0");
|
||||
try(HttpResponse response = request.execute()) {
|
||||
if(response.getStatus() != 200) {
|
||||
System.out.println("通知失败, status code:" + response.getStatus() + ", message:" + message);
|
||||
log.warn("通知失败, status code:{}, message:{}", response.getStatus(), message);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
public static int _findIdlePort(int port) {
|
||||
for(int i=port; i<65535; i++){
|
||||
try(ServerSocket ignored = new ServerSocket(i)){
|
||||
return i;
|
||||
}catch (IOException ignored) {
|
||||
}
|
||||
}
|
||||
return -1;
|
||||
}
|
||||
|
||||
public static String getRequestHeader(BufferedReader requestReader) throws IOException {
|
||||
String line;
|
||||
while ((line = requestReader.readLine()) != null) {
|
||||
if (line.trim().isEmpty()) {
|
||||
break;
|
||||
}
|
||||
if (line.startsWith("Range:")) {
|
||||
return line.substring("Range".length() + 1).trim();
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
public static void sendErrorResponse(Socket clientSocket, String statusCode) throws IOException {
|
||||
OutputStream responseStream = clientSocket.getOutputStream();
|
||||
PrintWriter responseWriter = new PrintWriter(responseStream, true);
|
||||
responseWriter.println("HTTP/1.1 " + statusCode);
|
||||
responseWriter.println("Content-Type: text/html");
|
||||
responseWriter.println();
|
||||
responseWriter.println("<h1>" + statusCode + "</h1>");
|
||||
responseStream.close();
|
||||
}
|
||||
|
||||
public static void sendFileRange(Socket clientSocket, File file, long startByte, long endByte) throws IOException {
|
||||
sendFileRange(clientSocket, file, startByte, endByte, null);
|
||||
}
|
||||
|
||||
public static void sendFileRange(Socket clientSocket, File file, long startByte, long endByte, String contentDispositionFileName) throws IOException {
|
||||
long fileLength = file.length();
|
||||
|
||||
// 区间必须夹在文件范围内:客户端可能请求远超文件末端(例如 bytes=0-999999),
|
||||
// 若照原样写进 Content-Length / Content-Range,对端会一直等一个永不到来的长度,
|
||||
// 连接要挂到超时才断。起点晚于终点(含空文件)则无可发送内容,直接回 416。
|
||||
if (startByte < 0)
|
||||
startByte = 0;
|
||||
if (endByte > fileLength - 1)
|
||||
endByte = fileLength - 1;
|
||||
if (startByte > endByte) {
|
||||
sendErrorResponse(clientSocket, "416 Range Not Satisfiable");
|
||||
return;
|
||||
}
|
||||
|
||||
OutputStream responseStream = clientSocket.getOutputStream();
|
||||
PrintWriter responseWriter = new PrintWriter(responseStream, true);
|
||||
responseWriter.println("HTTP/1.1 206 Partial Content");
|
||||
responseWriter.println("Content-Type: application/octet-stream");
|
||||
responseWriter.println("Accept-Ranges: bytes");
|
||||
responseWriter.println("Content-Length: " + (endByte - startByte + 1));
|
||||
responseWriter.println("Content-Range: bytes " + startByte + "-" + endByte + "/" + fileLength);
|
||||
if (contentDispositionFileName != null) {
|
||||
responseWriter.println("Content-Disposition: attachment; filename=\"" + contentDispositionFileName + "\"");
|
||||
}
|
||||
responseWriter.println();
|
||||
|
||||
try (RandomAccessFile randomAccessFile = new RandomAccessFile(file, "r")) {
|
||||
randomAccessFile.seek(startByte);
|
||||
// 1 KiB 每次系统调用太小,大包下载时开销明显;64 KiB 是常见的折中值。
|
||||
byte[] buffer = new byte[64 * 1024];
|
||||
int bytesRead;
|
||||
long bytesRemaining = endByte - startByte + 1;
|
||||
while (bytesRemaining > 0 && (bytesRead = randomAccessFile.read(buffer, 0, (int) Math.min(buffer.length, bytesRemaining))) != -1) {
|
||||
responseStream.write(buffer, 0, bytesRead);
|
||||
bytesRemaining -= bytesRead;
|
||||
}
|
||||
} catch (SocketException ignore) {
|
||||
} finally {
|
||||
responseStream.close();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -6,22 +6,22 @@ import lombok.Data;
|
||||
|
||||
@Data
|
||||
public class GalleryTask {
|
||||
public static byte DOWNLOADING = 1;
|
||||
public static byte DOWNLOAD_COMPLETE = 2;
|
||||
public static byte COMPRESSING = 3;
|
||||
public static byte COMPRESS_COMPLETE = 4;
|
||||
public static final byte DOWNLOADING = 1;
|
||||
public static final byte DOWNLOAD_COMPLETE = 2;
|
||||
public static final byte COMPRESSING = 3;
|
||||
public static final byte COMPRESS_COMPLETE = 4;
|
||||
|
||||
@JsonInclude(JsonInclude.Include.NON_NULL)
|
||||
private String name;
|
||||
private volatile String name;
|
||||
|
||||
private int gid;
|
||||
|
||||
private byte status;
|
||||
private volatile byte status;
|
||||
|
||||
private int proceeding;
|
||||
private volatile int proceeding;
|
||||
|
||||
@JsonIgnore
|
||||
private String path;
|
||||
private volatile String path;
|
||||
|
||||
@JsonIgnore
|
||||
public boolean is_download_complete(){
|
||||
|
||||
@@ -1,295 +1,159 @@
|
||||
package lion.Externel;
|
||||
|
||||
import lion.Config.Config;
|
||||
import lion.CustomUtil;
|
||||
import lion.Service.SubscriptionSnapshotStore;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import org.apache.http.HttpEntity;
|
||||
import org.apache.http.client.methods.CloseableHttpResponse;
|
||||
import org.apache.http.client.methods.HttpGet;
|
||||
import org.apache.http.impl.client.CloseableHttpClient;
|
||||
import org.apache.http.impl.client.HttpClients;
|
||||
|
||||
import java.io.*;
|
||||
import java.net.*;
|
||||
import java.net.ServerSocket;
|
||||
import java.net.Socket;
|
||||
import java.net.URLDecoder;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.nio.file.Files;
|
||||
import java.nio.file.Paths;
|
||||
import java.time.LocalDateTime;
|
||||
import java.util.*;
|
||||
import java.util.concurrent.ExecutorService;
|
||||
import java.util.concurrent.Executors;
|
||||
import java.util.regex.Matcher;
|
||||
import java.util.regex.Pattern;
|
||||
|
||||
import static lion.Config.Config.DouNaiClash;
|
||||
import static lion.Config.Config.DouNaiV2ray;
|
||||
import java.util.concurrent.*;
|
||||
|
||||
/** HTTP distributor for the last-known-good subscription snapshot. */
|
||||
@Slf4j
|
||||
public class BackupSubServer {
|
||||
public final class BackupSubServer implements Runnable {
|
||||
private final SubscriptionSnapshotStore snapshotStore;
|
||||
private final int port;
|
||||
private final ExecutorService workers;
|
||||
|
||||
public static void main(String[] args) {
|
||||
updateSub();
|
||||
ExecutorService threadPool = Executors.newFixedThreadPool(3600);
|
||||
threadPool.submit(() -> {
|
||||
if(LocalDateTime.now().getHour() == 0){
|
||||
updateSub();
|
||||
}
|
||||
});
|
||||
public BackupSubServer(SubscriptionSnapshotStore snapshotStore, int port, int workerCount) {
|
||||
this.snapshotStore = Objects.requireNonNull(snapshotStore);
|
||||
this.port = port;
|
||||
int workersCount = Math.max(1, workerCount);
|
||||
this.workers = new ThreadPoolExecutor(workersCount, workersCount, 0, TimeUnit.MILLISECONDS,
|
||||
new ArrayBlockingQueue<>(workersCount * 32), new ThreadPoolExecutor.AbortPolicy());
|
||||
}
|
||||
|
||||
String ip = "";
|
||||
try(ServerSocket serverSocket = new ServerSocket(8889)) {
|
||||
log.info("Sub Server listening on port {}", 8889);
|
||||
while (true) {
|
||||
Socket clientSocket = serverSocket.accept();
|
||||
ip = clientSocket.getInetAddress().getHostAddress();
|
||||
log.info("Client connected:{}", ip);
|
||||
// 线程池处理下载请求
|
||||
handleClientRequest(clientSocket);
|
||||
@Override
|
||||
public void run() {
|
||||
try (ServerSocket serverSocket = new ServerSocket(port)) {
|
||||
log.info("备机订阅服务监听端口 {}", port);
|
||||
while (!Thread.currentThread().isInterrupted()) {
|
||||
Socket socket = serverSocket.accept();
|
||||
try {
|
||||
workers.execute(() -> handle(socket));
|
||||
} catch (RejectedExecutionException e) {
|
||||
try (socket) {
|
||||
send(socket, 503, "Service Unavailable", "text/plain", new byte[0], false);
|
||||
} catch (IOException ignored) { }
|
||||
}
|
||||
}
|
||||
} catch (IOException e) {
|
||||
log.error("处理http请求时出错,IP:{},ERROR:{}", ip, e.getMessage());
|
||||
log.error("备机订阅服务停止: {}", e.getMessage());
|
||||
} finally {
|
||||
workers.shutdownNow();
|
||||
}
|
||||
}
|
||||
|
||||
public static void updateSub(){
|
||||
File DouNaiClashFile = new File("sub/DouNaiClash.txt");
|
||||
File DouNaiV2rayFile = new File("sub/DouNaiV2ray.txt");
|
||||
File directory = new File("sub");
|
||||
|
||||
if(!directory.isDirectory())
|
||||
try {
|
||||
Files.createDirectory(Paths.get("sub"));
|
||||
} catch (IOException e) {
|
||||
log.error("create directory error:{}", e.getMessage());
|
||||
}
|
||||
|
||||
List<String> DouNaiClash_profile;
|
||||
|
||||
//下载豆奶v2ray订阅
|
||||
try(FileWriter writer = new FileWriter(DouNaiV2rayFile)) {
|
||||
String DouNaiV2rayRaw = Get(DouNaiV2ray).getFirst();
|
||||
String[] v2rayPlain = new String(Base64.getDecoder().decode(DouNaiV2rayRaw)).split("\n");
|
||||
StringBuilder stringBuilder = new StringBuilder();
|
||||
Pattern pattern = Pattern.compile("-?\\d+(\\.\\d+)?");
|
||||
|
||||
//过滤高倍率节点
|
||||
for(String node: v2rayPlain){
|
||||
String name = URLDecoder.decode(node.split("#")[1], StandardCharsets.UTF_8);
|
||||
if(name.contains("流量")){
|
||||
Matcher matcher = pattern.matcher(name.substring(name.indexOf("(") + 1, name.indexOf(")")));
|
||||
|
||||
if (matcher.find()) {
|
||||
// 将匹配到的数字添加到列表中
|
||||
float ratio = Float.parseFloat(matcher.group());
|
||||
if(ratio <= 2) {
|
||||
stringBuilder.append(node).append("\n");
|
||||
continue;
|
||||
}
|
||||
}
|
||||
stringBuilder.append(node).append("\n");
|
||||
}
|
||||
else{
|
||||
stringBuilder.append(node).append("\n");
|
||||
}
|
||||
}
|
||||
writer.write(new String(Base64.getEncoder().encode(stringBuilder.toString().getBytes(StandardCharsets.UTF_8))));
|
||||
|
||||
log.info("load DouNai v2ray complete");
|
||||
}catch (IOException e){
|
||||
log.error("load DouNai v2ray failure: {}", e.getMessage());
|
||||
}
|
||||
|
||||
//下载豆奶clash订阅
|
||||
try(FileWriter writer = new FileWriter(DouNaiClashFile)) {
|
||||
DouNaiClash_profile = Get(DouNaiClash);
|
||||
//过滤高倍率节点
|
||||
ArrayList<String> clashProcessed = new ArrayList<>();
|
||||
boolean isProxies = false;
|
||||
boolean skip = false;
|
||||
for(String line: DouNaiClash_profile){
|
||||
if(line.equals("proxies:"))
|
||||
isProxies = true;
|
||||
else if(line.equals("proxy-groups:") && isProxies)
|
||||
isProxies = false;
|
||||
|
||||
if(isProxies) {
|
||||
if (line.contains("name"))
|
||||
skip = line.contains("流量");
|
||||
if (!skip)
|
||||
clashProcessed.add(line);
|
||||
}
|
||||
else
|
||||
if (!line.contains("流量"))
|
||||
clashProcessed.add(line);
|
||||
}
|
||||
|
||||
for(String line: clashProcessed)
|
||||
writer.write(line + "\n");
|
||||
|
||||
log.info("load DouNai clash complete");
|
||||
}catch (IOException e){
|
||||
log.error("load DouNai clash failure: {}", e.getMessage());
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
public static ArrayList<String> Get(String url) throws IOException {
|
||||
CloseableHttpClient httpClient = HttpClients.createDefault();
|
||||
CloseableHttpResponse httpResponse;
|
||||
HttpGet httpGet = new HttpGet(url);
|
||||
|
||||
httpResponse = httpClient.execute(httpGet);
|
||||
|
||||
HttpEntity responseEntity = httpResponse.getEntity();
|
||||
int statusCode = httpResponse.getStatusLine().getStatusCode();
|
||||
ArrayList<String> temp = new ArrayList<>();
|
||||
|
||||
if (statusCode == 200) {
|
||||
BufferedReader reader = new BufferedReader(new InputStreamReader(responseEntity.getContent()));
|
||||
String str;
|
||||
while ((str = reader.readLine()) != null)
|
||||
temp.add(str);
|
||||
}
|
||||
|
||||
httpClient.close();
|
||||
httpResponse.close();
|
||||
return temp;
|
||||
}
|
||||
|
||||
private static void handleClientRequest(Socket clientSocket) {
|
||||
String fileName = "";
|
||||
try {
|
||||
BufferedReader requestReader = new BufferedReader(new InputStreamReader(clientSocket.getInputStream()));
|
||||
String requestLine = requestReader.readLine();
|
||||
|
||||
// Parse the request line to get the method and path
|
||||
String[] requestParts = requestLine.split(" ");
|
||||
String method = requestParts[0];
|
||||
Map<String, String> paramMap = parseRequestLine(requestParts[1]);//path
|
||||
if(paramMap == null){
|
||||
sendErrorResponse(clientSocket, "404");
|
||||
private void handle(Socket socket) {
|
||||
try (socket) {
|
||||
socket.setSoTimeout(Config.subscriptionSocketTimeoutMs);
|
||||
BufferedReader reader = new BufferedReader(new InputStreamReader(socket.getInputStream(), StandardCharsets.US_ASCII));
|
||||
String requestLine = reader.readLine();
|
||||
if (requestLine == null || requestLine.length() > 2048) {
|
||||
send(socket, 400, "Bad Request", "text/plain", new byte[0], false);
|
||||
return;
|
||||
}
|
||||
log.info(Arrays.toString(requestParts));
|
||||
|
||||
// Only handle GET requests
|
||||
if (method.equals("GET")) {
|
||||
// Set the file path for download
|
||||
File file = new File(fileName);
|
||||
switch (paramMap.get("Client")) {
|
||||
case "v2" -> file = new File("sub/DouNaiV2ray.txt");
|
||||
case "cat" -> file = new File("sub/DouNaiClash.txt");
|
||||
}
|
||||
fileName = file.getName();
|
||||
log.info(file.getAbsolutePath());
|
||||
// Check if the file exists and is readable
|
||||
if (file.exists() && file.isFile() && file.canRead()) {
|
||||
// Get the file length
|
||||
long fileLength = file.length();
|
||||
|
||||
// Get the range information for resuming download
|
||||
long startByte = 0;
|
||||
long endByte = fileLength - 1;
|
||||
String rangeHeader = getRequestHeader(requestReader);
|
||||
if (rangeHeader != null && rangeHeader.startsWith("bytes=")) {
|
||||
String[] rangeValues = rangeHeader.substring(6).split("-");
|
||||
startByte = Long.parseLong(rangeValues[0]);
|
||||
if (rangeValues.length > 1 && !rangeValues[1].isEmpty()) {
|
||||
endByte = Long.parseLong(rangeValues[1]);
|
||||
}
|
||||
}
|
||||
|
||||
// Send the HTTP response headers
|
||||
OutputStream responseStream = clientSocket.getOutputStream();
|
||||
PrintWriter responseWriter = new PrintWriter(responseStream, true);
|
||||
responseWriter.println("HTTP/1.1 206 Partial Content");
|
||||
responseWriter.println("Content-Type: application/octet-stream");
|
||||
responseWriter.println("Accept-Ranges: bytes");
|
||||
responseWriter.println("Content-Length: " + (endByte - startByte + 1));
|
||||
responseWriter.println("Content-Range: bytes " + startByte + "-" + endByte + "/" + fileLength);
|
||||
responseWriter.println();
|
||||
|
||||
// Send the file content
|
||||
try (RandomAccessFile randomAccessFile = new RandomAccessFile(file, "r")) {
|
||||
randomAccessFile.seek(startByte);
|
||||
byte[] buffer = new byte[1024];
|
||||
int bytesRead;
|
||||
long bytesRemaining = endByte - startByte + 1;
|
||||
while (bytesRemaining > 0 && (bytesRead = randomAccessFile.read(buffer, 0, (int) Math.min(buffer.length, bytesRemaining))) != -1) {
|
||||
responseStream.write(buffer, 0, bytesRead);
|
||||
bytesRemaining -= bytesRead;
|
||||
}
|
||||
}catch (SocketException ignore){
|
||||
|
||||
}
|
||||
|
||||
// Close the response output stream
|
||||
responseStream.close();
|
||||
} else {
|
||||
// File not found or not readable, send 404 response
|
||||
sendErrorResponse(clientSocket, "404 Not Found");
|
||||
}
|
||||
} else {
|
||||
// Non-GET requests, send 501 response
|
||||
sendErrorResponse(clientSocket, "501 Not Implemented");
|
||||
String[] parts = requestLine.split(" ", 3);
|
||||
if (parts.length != 3 || (!"GET".equals(parts[0]) && !"HEAD".equals(parts[0]))) {
|
||||
send(socket, 405, "Method Not Allowed", "text/plain", new byte[0], false);
|
||||
return;
|
||||
}
|
||||
|
||||
// Close the request reader and client socket
|
||||
requestReader.close();
|
||||
clientSocket.close();
|
||||
} catch (IOException e) {
|
||||
log.error("处理文件下载时出错,IP:{}, 文件:{}, ERROR:{}", clientSocket.getInetAddress().getHostAddress(), fileName, e.getMessage());
|
||||
Map<String, String> headers = readHeaders(reader);
|
||||
if (headers == null) {
|
||||
send(socket, 400, "Bad Request", "text/plain", new byte[0], false);
|
||||
return;
|
||||
}
|
||||
String path = parts[1].split("\\?", 2)[0];
|
||||
if ("/health/subscription".equals(path)) {
|
||||
byte[] body = CustomUtil.objectMapper.writeValueAsBytes(snapshotStore.status());
|
||||
send(socket, 200, "OK", "application/json; charset=utf-8", body, "HEAD".equals(parts[0]));
|
||||
return;
|
||||
}
|
||||
String[] segments = path.split("/");
|
||||
if (segments.length != 4 || !"sub".equals(segments[1]) || !("v2".equals(segments[2]) || "cat".equals(segments[2]))) {
|
||||
send(socket, 404, "Not Found", "text/plain", new byte[0], false);
|
||||
return;
|
||||
}
|
||||
String key = URLDecoder.decode(segments[3], StandardCharsets.UTF_8);
|
||||
if (key.length() < 6 || key.length() > 512) {
|
||||
send(socket, 404, "Not Found", "text/plain", new byte[0], false);
|
||||
return;
|
||||
}
|
||||
SubscriptionSnapshotStore.Lookup lookup = snapshotStore.lookup(segments[2], key);
|
||||
if (lookup == null) {
|
||||
SubscriptionSnapshotStore.Status status = snapshotStore.status();
|
||||
int code = "unavailable".equals(status.state()) || "expired".equals(status.state()) ? 503 : 404;
|
||||
send(socket, code, code == 503 ? "Service Unavailable" : "Not Found", "text/plain", new byte[0], false);
|
||||
return;
|
||||
}
|
||||
serveContent(socket, headers.get("range"), lookup.content(), "v2".equals(segments[2]), "HEAD".equals(parts[0]));
|
||||
} catch (Exception e) {
|
||||
log.debug("处理备机订阅请求失败: {}", e.getMessage());
|
||||
}
|
||||
}
|
||||
|
||||
private static String getRequestHeader(BufferedReader requestReader) throws IOException {
|
||||
private static Map<String, String> readHeaders(BufferedReader reader) throws IOException {
|
||||
Map<String, String> headers = new HashMap<>();
|
||||
int total = 0;
|
||||
String line;
|
||||
while ((line = requestReader.readLine()) != null) {
|
||||
if (line.trim().isEmpty()) {
|
||||
break;
|
||||
}
|
||||
|
||||
if (line.startsWith("Range" + ":")) {
|
||||
return line.substring("Range".length() + 1).trim();
|
||||
}
|
||||
while ((line = reader.readLine()) != null) {
|
||||
total += line.length();
|
||||
if (total > 8192) return null;
|
||||
if (line.isEmpty()) return headers;
|
||||
int colon = line.indexOf(':');
|
||||
if (colon <= 0) return null;
|
||||
headers.put(line.substring(0, colon).trim().toLowerCase(Locale.ROOT), line.substring(colon + 1).trim());
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
public static Map<String, String> parseRequestLine(String requestLine) {
|
||||
Map<String, String> pathParams = new HashMap<>();
|
||||
|
||||
if(requestLine == null)
|
||||
return null;
|
||||
|
||||
String path;
|
||||
if(requestLine.contains("?"))
|
||||
path = requestLine.split("\\?")[0];
|
||||
else
|
||||
path = requestLine;
|
||||
|
||||
String[] vars = path.split("/");
|
||||
|
||||
if(vars.length < 4)
|
||||
return null;
|
||||
|
||||
pathParams.put("Key", vars[3]);
|
||||
pathParams.put("Client", vars[2]);
|
||||
|
||||
if(!pathParams.get("Client").equals("cat") && !pathParams.get("Client").equals("v2"))
|
||||
return null;
|
||||
|
||||
if(pathParams.get("Key").length()<6)
|
||||
return null;
|
||||
|
||||
return pathParams;
|
||||
private static void serveContent(Socket socket, String range, byte[] content, boolean v2, boolean head) throws IOException {
|
||||
long start = 0;
|
||||
long end = content.length - 1L;
|
||||
int status = 200;
|
||||
String reason = "OK";
|
||||
if (range != null && range.startsWith("bytes=")) {
|
||||
String value = range.substring(6).split(",", 2)[0];
|
||||
String[] values = value.split("-", 2);
|
||||
try {
|
||||
if (values.length != 2 || values[0].isEmpty()) throw new NumberFormatException();
|
||||
start = Long.parseLong(values[0]);
|
||||
if (!values[1].isEmpty()) end = Long.parseLong(values[1]);
|
||||
if (start < 0 || start > end || start >= content.length) throw new NumberFormatException();
|
||||
end = Math.min(end, content.length - 1L);
|
||||
status = 206;
|
||||
reason = "Partial Content";
|
||||
} catch (NumberFormatException e) {
|
||||
send(socket, 416, "Range Not Satisfiable", v2 ? "text/plain" : "text/yaml", new byte[0], head);
|
||||
return;
|
||||
}
|
||||
}
|
||||
byte[] body = Arrays.copyOfRange(content, (int) start, (int) end + 1);
|
||||
send(socket, status, reason, v2 ? "text/plain; charset=utf-8" : "text/yaml; charset=utf-8", body, head,
|
||||
status == 206 ? "bytes " + start + "-" + end + "/" + content.length : null);
|
||||
}
|
||||
|
||||
private static void sendErrorResponse(Socket clientSocket, String statusCode) throws IOException {
|
||||
OutputStream responseStream = clientSocket.getOutputStream();
|
||||
PrintWriter responseWriter = new PrintWriter(responseStream, true);
|
||||
responseWriter.println("HTTP/1.1 " + statusCode);
|
||||
responseWriter.println("Content-Type: text/html");
|
||||
responseWriter.println();
|
||||
responseWriter.println("<h1>" + statusCode + "</h1>");
|
||||
responseStream.close();
|
||||
private static void send(Socket socket, int status, String reason, String type, byte[] body, boolean head) throws IOException {
|
||||
send(socket, status, reason, type, body, head, null);
|
||||
}
|
||||
|
||||
private static void send(Socket socket, int status, String reason, String type, byte[] body, boolean head, String range) throws IOException {
|
||||
OutputStream output = socket.getOutputStream();
|
||||
StringBuilder header = new StringBuilder()
|
||||
.append("HTTP/1.1 ").append(status).append(' ').append(reason).append("\r\n")
|
||||
.append("Content-Type: ").append(type).append("\r\n")
|
||||
.append("Content-Length: ").append(body.length).append("\r\n")
|
||||
.append("Accept-Ranges: bytes\r\n")
|
||||
.append("Connection: close\r\n");
|
||||
if (range != null) header.append("Content-Range: ").append(range).append("\r\n");
|
||||
header.append("\r\n");
|
||||
output.write(header.toString().getBytes(StandardCharsets.US_ASCII));
|
||||
if (!head) output.write(body);
|
||||
output.flush();
|
||||
}
|
||||
}
|
||||
|
||||
@@ -3,6 +3,7 @@ package lion;
|
||||
import io.netty.bootstrap.Bootstrap;
|
||||
import lion.Config.Config;
|
||||
import lion.Externel.BackupSubServer;
|
||||
import lion.Service.SubscriptionSnapshotStore;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
|
||||
@Slf4j
|
||||
@@ -11,9 +12,14 @@ public class Main {
|
||||
public static void main(String[] args) {
|
||||
boot();
|
||||
Config.loadConfig();
|
||||
new Thread(() -> BackupSubServer.main(null)).start();
|
||||
SubscriptionSnapshotStore snapshotStore = new SubscriptionSnapshotStore(
|
||||
java.nio.file.Paths.get(Config.subscriptionDataDir), Config.subscriptionSyncSecret,
|
||||
Config.subscriptionMaxStaleSeconds, Config.subscriptionMaxPayloadBytes);
|
||||
snapshotStore.load();
|
||||
new Thread(new BackupSubServer(snapshotStore, Config.subscriptionHttpPort, Config.subscriptionHttpWorkers),
|
||||
"subscription-backup-http").start();
|
||||
new Thread(() -> MultiThreadedHTTPServer.main(null)).start();
|
||||
new storageNode();
|
||||
new storageNode(snapshotStore);
|
||||
}
|
||||
|
||||
public static void boot(){
|
||||
|
||||
@@ -14,6 +14,10 @@ public class AbstractMessage {
|
||||
|
||||
public static final byte MAINTAIN_MESSAGE = 7;
|
||||
|
||||
public static final byte AVAILABLE_CHECK_MESSAGE = 8;
|
||||
|
||||
public static final byte SUBSCRIPTION_SNAPSHOT_MESSAGE = 9;
|
||||
|
||||
public byte messageType;
|
||||
|
||||
public int messageId;
|
||||
|
||||
@@ -0,0 +1,11 @@
|
||||
package lion.Message.Main;
|
||||
|
||||
import lion.Message.AbstractMessage;
|
||||
import lombok.Data;
|
||||
|
||||
@Data
|
||||
public class AvailableCheckMessage extends AbstractMessage {
|
||||
{
|
||||
messageType = AVAILABLE_CHECK_MESSAGE;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,16 @@
|
||||
package lion.Message.Main;
|
||||
|
||||
import lombok.Data;
|
||||
import lombok.NoArgsConstructor;
|
||||
|
||||
@Data
|
||||
@NoArgsConstructor
|
||||
public class SubscriptionAccountSnapshot {
|
||||
private Integer accountId;
|
||||
private boolean enabled;
|
||||
private boolean filterHighMultiplier;
|
||||
private String v2ContentBase64;
|
||||
private String v2Sha256;
|
||||
private String clashContentBase64;
|
||||
private String clashSha256;
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
package lion.Message.Main;
|
||||
|
||||
import lombok.Data;
|
||||
import lombok.NoArgsConstructor;
|
||||
|
||||
@Data
|
||||
@NoArgsConstructor
|
||||
public class SubscriptionBindingSnapshot {
|
||||
private String publicKeySha256;
|
||||
private Integer accountId;
|
||||
}
|
||||
@@ -0,0 +1,22 @@
|
||||
package lion.Message.Main;
|
||||
|
||||
import lombok.Data;
|
||||
import lombok.NoArgsConstructor;
|
||||
import lombok.ToString;
|
||||
import lion.Message.AbstractMessage;
|
||||
|
||||
@Data
|
||||
@NoArgsConstructor
|
||||
@ToString(exclude = {"payloadBase64", "signature"})
|
||||
public class SubscriptionSnapshotMessage extends AbstractMessage {
|
||||
{
|
||||
messageType = SUBSCRIPTION_SNAPSHOT_MESSAGE;
|
||||
}
|
||||
|
||||
private int schemaVersion;
|
||||
private String revision;
|
||||
private long generatedAt;
|
||||
private String payloadBase64;
|
||||
private String payloadSha256;
|
||||
private String signature;
|
||||
}
|
||||
@@ -0,0 +1,15 @@
|
||||
package lion.Message.Main;
|
||||
|
||||
import lombok.Data;
|
||||
import lombok.NoArgsConstructor;
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
|
||||
@Data
|
||||
@NoArgsConstructor
|
||||
public class SubscriptionSnapshotPayload {
|
||||
private int schemaVersion;
|
||||
private List<SubscriptionAccountSnapshot> accounts = new ArrayList<>();
|
||||
private List<SubscriptionBindingSnapshot> bindings = new ArrayList<>();
|
||||
}
|
||||
@@ -5,6 +5,8 @@ import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
import io.netty.buffer.ByteBuf;
|
||||
import io.netty.channel.ChannelHandlerContext;
|
||||
import io.netty.handler.codec.ByteToMessageCodec;
|
||||
import io.netty.handler.codec.CorruptedFrameException;
|
||||
import io.netty.handler.codec.EncoderException;
|
||||
import lion.CustomUtil;
|
||||
import lion.Message.Main.*;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
@@ -15,6 +17,9 @@ import java.util.List;
|
||||
@Slf4j
|
||||
public class MessageCodec extends ByteToMessageCodec<AbstractMessage> {
|
||||
|
||||
/** 单帧负载上限,与上游 LengthFieldBasedFrameDecoder 的 100MB 保持一致。 */
|
||||
private static final int MAX_FRAME_BYTES = 100_000_000;
|
||||
|
||||
ObjectMapper objectMapper;
|
||||
|
||||
public MessageCodec(){
|
||||
@@ -23,9 +28,15 @@ public class MessageCodec extends ByteToMessageCodec<AbstractMessage> {
|
||||
|
||||
@Override
|
||||
protected void encode(ChannelHandlerContext channelHandlerContext, AbstractMessage abstractMessage, ByteBuf byteBuf) {
|
||||
// 先序列化、再写帧头:否则中途失败会留下只有类型字节、没有长度和 JSON 的半帧,
|
||||
// 对端的分帧器会为了等长度前缀一直挂住这条连接。
|
||||
byte[] bytes;
|
||||
try {
|
||||
bytes = objectMapper.writeValueAsBytes(abstractMessage);
|
||||
} catch (Exception e) {
|
||||
throw new EncoderException("序列化消息失败:" + e.getMessage(), e);
|
||||
}
|
||||
byteBuf.writeByte(abstractMessage.messageType);
|
||||
|
||||
byte[] bytes = objectMapper.valueToTree(abstractMessage).toString().getBytes(StandardCharsets.UTF_8);
|
||||
byteBuf.writeInt(bytes.length);
|
||||
byteBuf.writeBytes(bytes);
|
||||
}
|
||||
@@ -34,6 +45,8 @@ public class MessageCodec extends ByteToMessageCodec<AbstractMessage> {
|
||||
protected void decode(ChannelHandlerContext channelHandlerContext, ByteBuf byteBuf, List<Object> list) throws Exception {
|
||||
byte messageType = byteBuf.readByte();
|
||||
int length = byteBuf.readInt();
|
||||
if (length < 0 || length > MAX_FRAME_BYTES)
|
||||
throw new CorruptedFrameException("非法的帧长度: " + length);
|
||||
byte[] bytes = new byte[length];
|
||||
byteBuf.readBytes(bytes);
|
||||
final String metadata = new String(bytes, StandardCharsets.UTF_8);
|
||||
@@ -45,6 +58,8 @@ public class MessageCodec extends ByteToMessageCodec<AbstractMessage> {
|
||||
case AbstractMessage.DELETE_GALLERY_MESSAGE -> objectMapper.readValue(metadata, DeleteGalleryMessage.class);
|
||||
case AbstractMessage.IDENTITY_MESSAGE -> objectMapper.readValue(metadata, IdentityMessage.class);
|
||||
case AbstractMessage.MAINTAIN_MESSAGE -> objectMapper.readValue(metadata, MaintainMessage.class);
|
||||
case AbstractMessage.AVAILABLE_CHECK_MESSAGE -> objectMapper.readValue(metadata, AvailableCheckMessage.class);
|
||||
case AbstractMessage.SUBSCRIPTION_SNAPSHOT_MESSAGE -> objectMapper.readValue(metadata, SubscriptionSnapshotMessage.class);
|
||||
default -> null;
|
||||
};
|
||||
|
||||
|
||||
@@ -1,19 +1,28 @@
|
||||
package lion;
|
||||
|
||||
import lion.Config.Config;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
|
||||
import java.io.*;
|
||||
import java.net.*;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.nio.file.Path;
|
||||
import java.security.MessageDigest;
|
||||
import java.util.Arrays;
|
||||
import java.util.HashMap;
|
||||
import java.util.Map;
|
||||
import java.util.concurrent.ExecutorService;
|
||||
import java.util.concurrent.Executors;
|
||||
|
||||
@Slf4j
|
||||
public class MultiThreadedHTTPServer {
|
||||
private static final int PORT = 8888;
|
||||
private static final int BUFFER_SIZE = 1024;
|
||||
|
||||
/** 已安装的客户端可能连上却不发请求;没有读超时就会一直占住工作线程。 */
|
||||
private static final int SOCKET_READ_TIMEOUT_MILLIS = 30_000;
|
||||
|
||||
/** 兜底用的不存在路径:让「按文件名找不到归档」走统一的 404 分支。 */
|
||||
private static final File FILE_NOT_FOUND = new File("/root/abc");
|
||||
|
||||
public static void main(String[] args) {
|
||||
ExecutorService threadPool = Executors.newCachedThreadPool();
|
||||
@@ -23,34 +32,54 @@ public class MultiThreadedHTTPServer {
|
||||
} catch (UnknownHostException ignored){
|
||||
real_ip = "207.60.50.74";
|
||||
}
|
||||
try(ServerSocket serverSocket = new ServerSocket(PORT)) {
|
||||
log.info("Server listening on port {}", PORT);
|
||||
while (true) {
|
||||
Socket clientSocket = serverSocket.accept();
|
||||
String ip = clientSocket.getInetAddress().getHostAddress();
|
||||
if(ip.equals(real_ip)){
|
||||
log.info("Client connected");
|
||||
// 线程池处理下载请求
|
||||
threadPool.submit(() -> handleClientRequest(clientSocket));
|
||||
}else{
|
||||
log.info("unknown ip: " + ip);
|
||||
clientSocket.close();
|
||||
// accept 循环必须扛得住单次失败:否则一次异常就让下载服务整体退出,
|
||||
// 之后所有下载都 404,直到有人手工重启。
|
||||
while (!Thread.currentThread().isInterrupted()) {
|
||||
try (ServerSocket serverSocket = new ServerSocket(PORT)) {
|
||||
log.info("Server listening on port {}", PORT);
|
||||
while (true) {
|
||||
Socket clientSocket = serverSocket.accept();
|
||||
String ip = clientSocket.getInetAddress().getHostAddress();
|
||||
if(ip.equals(real_ip)){
|
||||
log.info("Client connected");
|
||||
threadPool.submit(() -> handleClientRequest(clientSocket));
|
||||
}else{
|
||||
log.info("unknown ip: " + ip);
|
||||
clientSocket.close();
|
||||
}
|
||||
}
|
||||
} catch (IOException e) {
|
||||
log.error("处理http请求时出错,IP:{},ERROR:{}", real_ip, e.getMessage());
|
||||
try {
|
||||
Thread.sleep(5_000);
|
||||
} catch (InterruptedException interrupted) {
|
||||
Thread.currentThread().interrupt();
|
||||
}
|
||||
|
||||
}
|
||||
} catch (IOException e) {
|
||||
log.error("处理http请求时出错,IP:{},ERROR:{}", real_ip, e.getMessage());
|
||||
}
|
||||
}
|
||||
|
||||
private static void handleClientRequest(Socket clientSocket) {
|
||||
// 包级可见:与 parseRequestLine 一样留作测试入口,便于对真实 socket 断言请求处理。
|
||||
static void handleClientRequest(Socket clientSocket) {
|
||||
String fileName = "";
|
||||
try {
|
||||
BufferedReader requestReader = new BufferedReader(new InputStreamReader(clientSocket.getInputStream()));
|
||||
try (Socket socket = clientSocket) {
|
||||
// 没有读超时的连接可以永远占住一个工作线程,必须设一个上限。
|
||||
socket.setSoTimeout(SOCKET_READ_TIMEOUT_MILLIS);
|
||||
BufferedReader requestReader = new BufferedReader(
|
||||
new InputStreamReader(socket.getInputStream(), StandardCharsets.ISO_8859_1));
|
||||
String requestLine = requestReader.readLine();
|
||||
|
||||
// 空行或没有空格的请求行会让 requestParts[1] 越界。这种异常不是 IOException,
|
||||
// 原样抛出会被线程池静默吞掉、socket 也关不掉,因此在这里显式判非法。
|
||||
if (requestLine == null || requestLine.isBlank())
|
||||
return;
|
||||
|
||||
// Parse the request line to get the method and path
|
||||
String[] requestParts = requestLine.split(" ");
|
||||
String[] requestParts = requestLine.trim().split("\\s+");
|
||||
if (requestParts.length < 2) {
|
||||
CustomUtil.sendErrorResponse(socket, "400 Bad Request");
|
||||
return;
|
||||
}
|
||||
String method = requestParts[0];
|
||||
Map<String, String> paramMap = parseRequestLine(requestParts[1]);//path
|
||||
log.info(Arrays.toString(requestParts));
|
||||
@@ -59,46 +88,36 @@ public class MultiThreadedHTTPServer {
|
||||
if (method.equals("GET")) {
|
||||
// Set the file path for download
|
||||
File file;
|
||||
if(paramMap.get("AuthCode") != null)
|
||||
if(paramMap.get("AuthCode").equals("alone")){
|
||||
String path = URLDecoder.decode(requestParts[1].split("\\?")[0], StandardCharsets.UTF_8);
|
||||
file = new File(path);
|
||||
String authCode = paramMap.get("AuthCode");
|
||||
if (authCode == null) {
|
||||
CustomUtil.sendErrorResponse(socket, "403 Forbidden");
|
||||
return;
|
||||
}
|
||||
String requestPath = URLDecoder.decode(requestParts[1].split("\\?")[0], StandardCharsets.UTF_8);
|
||||
if (isAdminRequest(authCode)) {
|
||||
// 管理员直取文件从「任意绝对路径」改为「限定在配置根目录之下」:
|
||||
// ../、绝对路径、符号链接逃逸等越界一律按找不到处理。
|
||||
file = resolveAdminFile(requestPath);
|
||||
if (file == null) {
|
||||
log.warn("管理员下载越界,已拒绝:{}", requestPath);
|
||||
CustomUtil.sendErrorResponse(socket, "404 Not Found");
|
||||
return;
|
||||
}
|
||||
else {
|
||||
String filePath = "/root/gallery/gallery";
|
||||
String path = URLDecoder.decode(requestParts[1].split("\\?")[0], StandardCharsets.UTF_8);
|
||||
if(!path.contains(".")){
|
||||
file = new File("/root/abc");
|
||||
}else {
|
||||
String name = path.substring(0, path.lastIndexOf('.'));
|
||||
} else {
|
||||
String filePath = "/root/gallery/gallery";
|
||||
String gid = paramMap.get("gid");
|
||||
file = gid == null ? null : findGalleryZipByGid(new File(filePath), gid);
|
||||
|
||||
//兼容没有gid参数的旧下载链接,再尝试按链接中的文件名查找
|
||||
if (file == null) {
|
||||
if (requestPath.contains(".")) {
|
||||
String name = requestPath.substring(0, requestPath.lastIndexOf('.'));
|
||||
name = filePath + name + "/" + name + ".zip";
|
||||
file = new File(name);
|
||||
|
||||
//该文件不存在
|
||||
if(!file.isFile()){
|
||||
String gid = paramMap.get("gid");
|
||||
|
||||
//文件不存在的情况下gid也不存在,直接404
|
||||
if(gid == null)
|
||||
file = new File("/root/abc");
|
||||
|
||||
//gid存在的情况下尝试查找对应的文件
|
||||
else {
|
||||
File[] galleryDirectories = (new File(filePath)).listFiles();
|
||||
|
||||
assert galleryDirectories != null;
|
||||
for (File galleryDirectory : galleryDirectories)
|
||||
if (galleryDirectory.getName().contains(gid)) {
|
||||
file = new File(galleryDirectory.getAbsolutePath(), galleryDirectory.getName() + ".zip");
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
} else {
|
||||
file = FILE_NOT_FOUND;
|
||||
}
|
||||
}
|
||||
else{
|
||||
sendErrorResponse(clientSocket, "403 Forbidden");
|
||||
return;
|
||||
}
|
||||
fileName = file.getName();
|
||||
log.info(file.getAbsolutePath());
|
||||
@@ -110,69 +129,99 @@ public class MultiThreadedHTTPServer {
|
||||
// Get the range information for resuming download
|
||||
long startByte = 0;
|
||||
long endByte = fileLength - 1;
|
||||
String rangeHeader = getRequestHeader(requestReader);
|
||||
String rangeHeader = CustomUtil.getRequestHeader(requestReader);
|
||||
if (rangeHeader != null && rangeHeader.startsWith("bytes=")) {
|
||||
String[] rangeValues = rangeHeader.substring(6).split("-");
|
||||
startByte = Long.parseLong(rangeValues[0]);
|
||||
if (rangeValues.length > 1 && !rangeValues[1].isEmpty()) {
|
||||
endByte = Long.parseLong(rangeValues[1]);
|
||||
// 非法 Range 不得中断处理:解析失败退回整文件,
|
||||
// 区间越界则由 sendFileRange 统一夹取或回 416。
|
||||
String value = rangeHeader.substring(6).split(",", 2)[0];
|
||||
String[] rangeValues = value.split("-", 2);
|
||||
try {
|
||||
if (rangeValues[0].isEmpty())
|
||||
throw new NumberFormatException();
|
||||
startByte = Long.parseLong(rangeValues[0]);
|
||||
if (rangeValues.length > 1 && !rangeValues[1].isEmpty())
|
||||
endByte = Long.parseLong(rangeValues[1]);
|
||||
} catch (NumberFormatException invalidRange) {
|
||||
startByte = 0;
|
||||
endByte = fileLength - 1;
|
||||
}
|
||||
}
|
||||
|
||||
// Send the HTTP response headers
|
||||
OutputStream responseStream = clientSocket.getOutputStream();
|
||||
PrintWriter responseWriter = new PrintWriter(responseStream, true);
|
||||
responseWriter.println("HTTP/1.1 206 Partial Content");
|
||||
responseWriter.println("Content-Type: application/octet-stream");
|
||||
responseWriter.println("Accept-Ranges: bytes");
|
||||
responseWriter.println("Content-Length: " + (endByte - startByte + 1));
|
||||
responseWriter.println("Content-Range: bytes " + startByte + "-" + endByte + "/" + fileLength);
|
||||
responseWriter.println("Content-Disposition: attachment; filename=\"" + fileName + "\"");
|
||||
responseWriter.println();
|
||||
|
||||
// Send the file content
|
||||
try (RandomAccessFile randomAccessFile = new RandomAccessFile(file, "r")) {
|
||||
randomAccessFile.seek(startByte);
|
||||
byte[] buffer = new byte[BUFFER_SIZE];
|
||||
int bytesRead;
|
||||
long bytesRemaining = endByte - startByte + 1;
|
||||
while (bytesRemaining > 0 && (bytesRead = randomAccessFile.read(buffer, 0, (int) Math.min(buffer.length, bytesRemaining))) != -1) {
|
||||
responseStream.write(buffer, 0, bytesRead);
|
||||
bytesRemaining -= bytesRead;
|
||||
}
|
||||
}catch (SocketException ignore){
|
||||
|
||||
}
|
||||
|
||||
// Close the response output stream
|
||||
responseStream.close();
|
||||
CustomUtil.sendFileRange(socket, file, startByte, endByte, fileName);
|
||||
} else {
|
||||
// File not found or not readable, send 404 response
|
||||
sendErrorResponse(clientSocket, "404 Not Found");
|
||||
CustomUtil.sendErrorResponse(socket, "404 Not Found");
|
||||
}
|
||||
} else {
|
||||
// Non-GET requests, send 501 response
|
||||
sendErrorResponse(clientSocket, "501 Not Implemented");
|
||||
CustomUtil.sendErrorResponse(socket, "501 Not Implemented");
|
||||
}
|
||||
|
||||
// Close the request reader and client socket
|
||||
// socket 由 try-with-resources 关闭,这里只收尾 reader。
|
||||
requestReader.close();
|
||||
clientSocket.close();
|
||||
} catch (IOException e) {
|
||||
} catch (Exception e) {
|
||||
// 不能只捕 IOException:越界等运行时异常会绕过 finally,导致连接泄漏。
|
||||
log.error("处理文件下载时出错,IP:{}, 文件:{}, ERROR:{}", clientSocket.getInetAddress().getHostAddress(), fileName, e.getMessage());
|
||||
}
|
||||
}
|
||||
|
||||
private static String getRequestHeader(BufferedReader requestReader) throws IOException {
|
||||
String line;
|
||||
while ((line = requestReader.readLine()) != null) {
|
||||
if (line.trim().isEmpty()) {
|
||||
break;
|
||||
}
|
||||
/**
|
||||
* 该请求是否走管理员直取文件通道。
|
||||
*
|
||||
* <p>口令不再写死在源码里,改为配置(环境变量 STORAGE_DOWNLOAD_ADMIN_CODE 优先,
|
||||
* 其次配置文件 AdminDownloadCode)。未配置即关闭该能力;比较用固定时间算法,
|
||||
* 避免按前缀长度泄露口令。
|
||||
*/
|
||||
static boolean isAdminRequest(String authCode){
|
||||
String expected = Config.adminDownloadCode;
|
||||
if (expected == null || expected.isBlank() || authCode == null)
|
||||
return false;
|
||||
return MessageDigest.isEqual(expected.getBytes(StandardCharsets.UTF_8),
|
||||
authCode.getBytes(StandardCharsets.UTF_8));
|
||||
}
|
||||
|
||||
if (line.startsWith("Range" + ":")) {
|
||||
return line.substring("Range".length() + 1).trim();
|
||||
}
|
||||
/**
|
||||
* 把管理员的请求路径解析为受配置根目录约束的文件。
|
||||
*
|
||||
* <p>用真实路径做前缀判断,因此 {@code ..} 与指向根目录之外的符号链接都会被拒。
|
||||
* 文件不存在(含越界)返回 null,由调用方回 404。
|
||||
*/
|
||||
static File resolveAdminFile(String requestPath){
|
||||
if (requestPath == null || requestPath.isBlank())
|
||||
return null;
|
||||
try {
|
||||
Path root = Path.of(Config.adminDownloadRoot).toRealPath();
|
||||
// 兼容两种调用:绝对路径(主站历史上就是这么传的)与相对根目录的路径。
|
||||
Path raw = Path.of(requestPath);
|
||||
Path candidate = (raw.isAbsolute() ? raw : root.resolve(raw)).normalize();
|
||||
if (!candidate.startsWith(root))
|
||||
return null;
|
||||
// 必须落到真实路径再判断一次:这样指向根目录之外的符号链接也会被拒。
|
||||
Path resolved = candidate.toRealPath();
|
||||
return resolved.startsWith(root) && resolved.toFile().isFile() ? resolved.toFile() : null;
|
||||
} catch (Exception e) {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
private static File findGalleryZipByGid(File galleryRoot, String gid){
|
||||
File[] galleryDirectories = galleryRoot.listFiles(File::isDirectory);
|
||||
if(galleryDirectories == null)
|
||||
return null;
|
||||
|
||||
String marker = "[" + gid;
|
||||
for(File galleryDirectory : galleryDirectories){
|
||||
String name = galleryDirectory.getName();
|
||||
int markerIndex = name.lastIndexOf(marker);
|
||||
if(markerIndex < 0)
|
||||
continue;
|
||||
int suffixIndex = markerIndex + marker.length();
|
||||
if(suffixIndex >= name.length() || (name.charAt(suffixIndex) != ']' && name.charAt(suffixIndex) != '-'))
|
||||
continue;
|
||||
|
||||
File zip = new File(galleryDirectory, name + ".zip");
|
||||
if(zip.isFile())
|
||||
return zip;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
@@ -184,7 +233,7 @@ public class MultiThreadedHTTPServer {
|
||||
return null;
|
||||
|
||||
if (requestLine.contains("?")) {
|
||||
String[] requestParts = requestLine.split("\\?");
|
||||
String[] requestParts = requestLine.split("\\?", 2);
|
||||
String path = requestParts[0];
|
||||
queryParams.put("path", path);
|
||||
|
||||
@@ -194,7 +243,8 @@ public class MultiThreadedHTTPServer {
|
||||
|
||||
// Split the query string into individual parameter key-value pairs
|
||||
for (String paramPair : paramPairs) {
|
||||
String[] keyValue = paramPair.split("=");
|
||||
// limit=2:参数值里含有 '='(例如 base64 结尾的 padding)时不能被截断。
|
||||
String[] keyValue = paramPair.split("=", 2);
|
||||
if (keyValue.length == 2) {
|
||||
String key = URLDecoder.decode(keyValue[0], StandardCharsets.UTF_8);
|
||||
String value = URLDecoder.decode(keyValue[1], StandardCharsets.UTF_8);
|
||||
@@ -206,14 +256,4 @@ public class MultiThreadedHTTPServer {
|
||||
}
|
||||
return queryParams;
|
||||
}
|
||||
|
||||
private static void sendErrorResponse(Socket clientSocket, String statusCode) throws IOException {
|
||||
OutputStream responseStream = clientSocket.getOutputStream();
|
||||
PrintWriter responseWriter = new PrintWriter(responseStream, true);
|
||||
responseWriter.println("HTTP/1.1 " + statusCode);
|
||||
responseWriter.println("Content-Type: text/html");
|
||||
responseWriter.println();
|
||||
responseWriter.println("<h1>" + statusCode + "</h1>");
|
||||
responseStream.close();
|
||||
}
|
||||
}
|
||||
@@ -4,6 +4,7 @@ import lion.ErrorCode.ErrorCode;
|
||||
import cn.hutool.core.io.FileUtil;
|
||||
|
||||
import java.io.File;
|
||||
import java.nio.file.Path;
|
||||
|
||||
public class DeleteService {
|
||||
public static byte deleteAll(String path){
|
||||
@@ -16,4 +17,23 @@ public class DeleteService {
|
||||
else
|
||||
return ErrorCode.IO_ERROR;
|
||||
}
|
||||
|
||||
/**
|
||||
* 删除某个根目录下的子目录,且保证不越界。
|
||||
*
|
||||
* <p>画廊名来自主站下发的消息,直接拼接根路径会让 {@code ../} 或绝对路径
|
||||
* 操纵到根目录之外(例如删掉节点上任意目录)。这里按规范化后的路径判断包含关系,
|
||||
* 越界与空名一律按「找不到」处理,绝不落到磁盘删除。
|
||||
*/
|
||||
public static byte deleteWithin(String rootPath, String galleryName){
|
||||
if (galleryName == null || galleryName.isBlank())
|
||||
return ErrorCode.FILE_NOT_FOUND;
|
||||
|
||||
Path root = new File(rootPath).toPath().toAbsolutePath().normalize();
|
||||
Path target = root.resolve(galleryName).normalize();
|
||||
if (target.equals(root) || !target.startsWith(root))
|
||||
return ErrorCode.FILE_NOT_FOUND;
|
||||
|
||||
return deleteAll(target.toString());
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,26 +1,24 @@
|
||||
package lion.Service;
|
||||
|
||||
import io.netty.channel.Channel;
|
||||
import io.netty.channel.DefaultEventLoop;
|
||||
import io.netty.channel.EventLoop;
|
||||
import io.netty.util.concurrent.Promise;
|
||||
import lion.CustomUtil;
|
||||
import lion.Domain.GalleryTask;
|
||||
import cn.hutool.core.io.FileUtil;
|
||||
import cn.hutool.core.util.ZipUtil;
|
||||
import lion.Message.AbstractMessage;
|
||||
import lombok.Data;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
|
||||
import java.io.*;
|
||||
import java.util.*;
|
||||
import java.util.concurrent.ExecutorService;
|
||||
import java.util.concurrent.Executors;
|
||||
import java.util.concurrent.ScheduledThreadPoolExecutor;
|
||||
import java.util.concurrent.TimeUnit;
|
||||
import java.util.concurrent.locks.ReentrantLock;
|
||||
|
||||
import java.nio.file.*;
|
||||
import java.util.concurrent.ConcurrentHashMap;
|
||||
import java.util.zip.CRC32;
|
||||
import java.util.zip.ZipEntry;
|
||||
import java.util.zip.ZipFile;
|
||||
|
||||
@Slf4j
|
||||
@Data
|
||||
public class DownloadCheckService {
|
||||
Map<Integer, GalleryTask> queue;
|
||||
|
||||
@@ -30,21 +28,53 @@ public class DownloadCheckService {
|
||||
|
||||
ScheduledThreadPoolExecutor convert_thread;
|
||||
|
||||
ArrayList<GalleryTask> compress_queue;
|
||||
final ArrayList<GalleryTask> compress_queue;
|
||||
|
||||
Channel node;
|
||||
final Map<Integer, Long> retryAfter = new ConcurrentHashMap<>();
|
||||
|
||||
HashMap<Integer, Promise<AbstractMessage>> promises;
|
||||
/**
|
||||
* 已校验通过的归档缓存:路径 → (归档大小, 校验时刻)。
|
||||
*
|
||||
* <p>归档一旦落盘就不再变化(发布用 ATOMIC_MOVE),因此同一路径只有在大小变化时
|
||||
* 才需要重新校验。没有它时,每次 {@link #addToQueue} 都会把候选 ZIP 整包读取并
|
||||
* 重算 CRC——主站重连触发 resetUndone 时会对所有未完成任务走一遍,纯属浪费磁盘 I/O。
|
||||
*/
|
||||
final Map<String, VerifiedArchive> verifiedArchives = new ConcurrentHashMap<>();
|
||||
|
||||
EventLoop eventLoop;
|
||||
private static final long ARCHIVE_REVERIFY_MILLIS = 3_600_000L;
|
||||
|
||||
public DownloadCheckService(Map<Integer, GalleryTask> queue, HashMap<Integer, Promise<AbstractMessage>> promises){
|
||||
private record VerifiedArchive(long size, long verifiedAt) {}
|
||||
|
||||
/**
|
||||
* 完成通知的专用线程。
|
||||
*
|
||||
* <p>{@link CustomUtil#notifyMe} 是同步 HTTP,原先在 {@link #addToQueue} 里直接调用,
|
||||
* 而 addToQueue 由 Netty IO 线程持节点主锁执行,一次慢请求就会把整个节点的状态上报
|
||||
* 与删除处理一起卡住。改为异步投递后,外呼延迟不再占用锁。
|
||||
*/
|
||||
private final ExecutorService notificationExecutor;
|
||||
|
||||
/** 是否发送完成通知;仅生产构造(启动调度器)时启用,单元测试保持静默不外呼。 */
|
||||
private final boolean notificationsEnabled;
|
||||
|
||||
public DownloadCheckService(Map<Integer, GalleryTask> queue){
|
||||
this(queue, true);
|
||||
}
|
||||
|
||||
// Tests use temporary directories and invoke scans explicitly.
|
||||
DownloadCheckService(Map<Integer, GalleryTask> queue, boolean startScheduler){
|
||||
this.queue = queue;
|
||||
this.promises = promises;
|
||||
eventLoop = new DefaultEventLoop();
|
||||
compress_queue = new ArrayList<>(0);
|
||||
convert_thread = new ScheduledThreadPoolExecutor(1);
|
||||
convert_thread.scheduleAtFixedRate(this::compress, 0, 5, TimeUnit.SECONDS);
|
||||
this.notificationsEnabled = startScheduler;
|
||||
this.notificationExecutor = Executors.newSingleThreadExecutor(r -> {
|
||||
Thread thread = new Thread(r, "storage-node-notify");
|
||||
thread.setDaemon(true);
|
||||
return thread;
|
||||
});
|
||||
if (startScheduler) {
|
||||
convert_thread = new ScheduledThreadPoolExecutor(1);
|
||||
convert_thread.scheduleWithFixedDelay(this::compress, 0, 5, TimeUnit.SECONDS);
|
||||
}
|
||||
}
|
||||
|
||||
public boolean downloadCheck(){
|
||||
@@ -72,7 +102,7 @@ public class DownloadCheckService {
|
||||
|
||||
while(fileIterator.hasNext()){
|
||||
File file = fileIterator.next();
|
||||
if(!file.getName().contains(String.valueOf(galleryTask.getGid())))
|
||||
if(!file.isDirectory() || !matchesGid(file.getName(), galleryTask.getGid()))
|
||||
continue;
|
||||
|
||||
galleryTask.setStatus(GalleryTask.DOWNLOADING);
|
||||
@@ -98,9 +128,12 @@ public class DownloadCheckService {
|
||||
|
||||
//压缩队列
|
||||
for(GalleryTask galleryTask: queue.values())
|
||||
if (galleryTask.is_download_complete()) {
|
||||
if (galleryTask.is_download_complete()
|
||||
&& System.currentTimeMillis() >= retryAfter.getOrDefault(galleryTask.getGid(), 0L)) {
|
||||
galleryTask.setStatus(GalleryTask.COMPRESSING);
|
||||
compress_queue.add(galleryTask);
|
||||
synchronized (compress_queue) {
|
||||
compress_queue.add(galleryTask);
|
||||
}
|
||||
}
|
||||
|
||||
return result;
|
||||
@@ -110,65 +143,190 @@ public class DownloadCheckService {
|
||||
* 压缩线程:将压缩队列的任务复制一份,进行转换
|
||||
*/
|
||||
public void compress() {
|
||||
if(compress_queue.isEmpty())
|
||||
return;
|
||||
ReentrantLock reentrantLock = new ReentrantLock();
|
||||
reentrantLock.lock();
|
||||
ArrayList<GalleryTask> galleryTasks = new ArrayList<>(compress_queue);
|
||||
compress_queue.clear();
|
||||
reentrantLock.unlock();
|
||||
for (GalleryTask galleryTask : galleryTasks) {
|
||||
ArrayList<GalleryTask> galleryTasks;
|
||||
synchronized (compress_queue) {
|
||||
if (compress_queue.isEmpty())
|
||||
return;
|
||||
galleryTasks = new ArrayList<>(compress_queue);
|
||||
compress_queue.clear();
|
||||
}
|
||||
for (GalleryTask galleryTask : galleryTasks) {
|
||||
Path temporary = null;
|
||||
try {
|
||||
log.info("开始压缩:{}", galleryTask.getName());
|
||||
Path directory = Paths.get(storagePath, galleryTask.getName());
|
||||
Files.createDirectories(directory);
|
||||
Path archive = directory.resolve(galleryTask.getName() + ".zip");
|
||||
temporary = Files.createTempFile(directory, ".compress-", ".zip.part");
|
||||
ZipUtil.zip(galleryTask.getPath(), temporary.toString());
|
||||
if (!isValidArchive(temporary.toFile()))
|
||||
throw new IOException("压缩包校验失败");
|
||||
// Publish only a closed, verified archive. A crash leaves a .part file.
|
||||
try {
|
||||
log.info("开始压缩:{}", galleryTask.getName());
|
||||
//创建文件夹
|
||||
File file = new File(storagePath + galleryTask.getName());
|
||||
if (file.isDirectory() || file.mkdirs()) {
|
||||
log.info("{}文件夹创建成功", galleryTask.getName());
|
||||
} else {
|
||||
log.error("{}文件夹创建失败", galleryTask.getName());
|
||||
continue;
|
||||
}
|
||||
|
||||
//生成压缩包
|
||||
ZipUtil.zip(galleryTask.getPath(), storagePath + galleryTask.getName() + "/" + galleryTask.getName() + ".zip");
|
||||
log.info("{}压缩完成", galleryTask.getName());
|
||||
|
||||
FileUtil.del(galleryTask.getPath());
|
||||
galleryTask.setStatus(GalleryTask.COMPRESS_COMPLETE);
|
||||
} catch (Exception e){
|
||||
log.error("{}压缩失败:{}", galleryTask, e.getMessage());
|
||||
Files.move(temporary, archive, StandardCopyOption.ATOMIC_MOVE, StandardCopyOption.REPLACE_EXISTING);
|
||||
} catch (AtomicMoveNotSupportedException e) {
|
||||
Files.move(temporary, archive, StandardCopyOption.REPLACE_EXISTING);
|
||||
}
|
||||
temporary = null;
|
||||
galleryTask.setStatus(GalleryTask.COMPRESS_COMPLETE);
|
||||
retryAfter.remove(galleryTask.getGid());
|
||||
if (!FileUtil.del(galleryTask.getPath()))
|
||||
log.warn("压缩已完成,但源目录清理失败: {}", galleryTask.getPath());
|
||||
log.info("{}压缩完成", galleryTask.getName());
|
||||
} catch (Exception e) {
|
||||
// Keep the source and restore an existing, retryable protocol state.
|
||||
if (!galleryTask.is_compress_complete()) {
|
||||
retryAfter.put(galleryTask.getGid(), System.currentTimeMillis() + 30_000);
|
||||
galleryTask.setStatus(GalleryTask.DOWNLOAD_COMPLETE);
|
||||
}
|
||||
log.error("{}压缩或清理失败,源文件保留,稍后可重试", galleryTask.getName(), e);
|
||||
} finally {
|
||||
if (temporary != null) {
|
||||
try { Files.deleteIfExists(temporary); }
|
||||
catch (IOException e) { log.warn("清理压缩临时文件失败: {}", temporary, e); }
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
static boolean isValidArchive(File file) {
|
||||
if (!file.isFile())
|
||||
return false;
|
||||
try (ZipFile zip = new ZipFile(file)) {
|
||||
if (zip.size() == 0)
|
||||
return false;
|
||||
byte[] buffer = new byte[8192];
|
||||
Enumeration<? extends ZipEntry> entries = zip.entries();
|
||||
while (entries.hasMoreElements()) {
|
||||
ZipEntry entry = entries.nextElement();
|
||||
if (entry.isDirectory())
|
||||
continue;
|
||||
CRC32 crc = new CRC32();
|
||||
long size = 0;
|
||||
try (InputStream input = zip.getInputStream(entry)) {
|
||||
int count;
|
||||
while ((count = input.read(buffer)) != -1) {
|
||||
crc.update(buffer, 0, count);
|
||||
size += count;
|
||||
}
|
||||
}
|
||||
if (size != entry.getSize() || crc.getValue() != entry.getCrc())
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
} catch (IOException e) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
public GalleryTask addToQueue(GalleryTask galleryTask){
|
||||
GalleryTask active = queue.get(galleryTask.getGid());
|
||||
if (active != null && active.is_compressing())
|
||||
return active;
|
||||
retryAfter.remove(galleryTask.getGid());
|
||||
// A reconnect can resend a task whose name is stale (for example, the
|
||||
// downloader appended a resolution suffix). Resolve completed archives
|
||||
// by gid first, because the gid is stable while the directory name is not.
|
||||
File storedDirectory = findStoredDirectoryByGid(galleryTask.getGid());
|
||||
if(storedDirectory != null){
|
||||
queue.remove(galleryTask.getGid());
|
||||
galleryTask.setName(storedDirectory.getName());
|
||||
galleryTask.setStatus(GalleryTask.COMPRESS_COMPLETE);
|
||||
notifyAlreadyStored(galleryTask.getName());
|
||||
return galleryTask;
|
||||
}
|
||||
|
||||
// If the task is still downloading, retain the actual directory name so
|
||||
// subsequent progress and compression use the same identity.
|
||||
File downloadingDirectory = findDirectoryByGid(new File(downloadPath), galleryTask.getGid());
|
||||
if(downloadingDirectory != null){
|
||||
GalleryTask queuedTask = queue.get(galleryTask.getGid());
|
||||
if(queuedTask == null){
|
||||
refreshDownloadingTask(galleryTask, downloadingDirectory);
|
||||
queue.put(galleryTask.getGid(), galleryTask);
|
||||
return galleryTask;
|
||||
}
|
||||
if(!queuedTask.is_compressing() && !queuedTask.is_compress_complete())
|
||||
refreshDownloadingTask(queuedTask, downloadingDirectory);
|
||||
return queuedTask;
|
||||
}
|
||||
|
||||
GalleryTask queuedTask = queue.putIfAbsent(galleryTask.getGid(), galleryTask);
|
||||
return queuedTask == null ? galleryTask : queuedTask;
|
||||
}
|
||||
|
||||
private void refreshDownloadingTask(GalleryTask galleryTask, File downloadingDirectory){
|
||||
galleryTask.setName(downloadingDirectory.getName());
|
||||
File[] pages = downloadingDirectory.listFiles((dir, name) -> !name.equals("galleryinfo.txt"));
|
||||
galleryTask.setProceeding(pages == null ? 0 : pages.length);
|
||||
if(new File(downloadingDirectory, "galleryinfo.txt").isFile()){
|
||||
galleryTask.setStatus(GalleryTask.DOWNLOAD_COMPLETE);
|
||||
galleryTask.setPath(downloadingDirectory.getPath());
|
||||
}else{
|
||||
galleryTask.setStatus(GalleryTask.DOWNLOADING);
|
||||
}
|
||||
}
|
||||
|
||||
private File findStoredDirectoryByGid(int gid){
|
||||
File storageDirectory = new File(storagePath);
|
||||
File[] directories = storageDirectory.listFiles(File::isDirectory);
|
||||
if(directories == null)
|
||||
return null;
|
||||
|
||||
for(File directory : directories){
|
||||
if(matchesGid(directory.getName(), gid)
|
||||
&& isVerifiedArchive(new File(directory, directory.getName() + ".zip")))
|
||||
return directory;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* 检查改任务是否为已完成任务,如已完成则返回true,若未完成则加入队列
|
||||
* @return true if compress complete, false otherwise
|
||||
* 压缩包校验带缓存:归档按大小不变,校验结果可复用。
|
||||
* 大小变化(重新压缩)或距上次校验超过一小时才重算,避免重复整包读取。
|
||||
*/
|
||||
public boolean addToQueue(GalleryTask galleryTask){
|
||||
//是否含有名字,进行中任务一般有名字,没有名字则肯定为初始任务,存在名字至少在下载路径出现过
|
||||
if(galleryTask.getName() == null || !galleryTask.getName().isEmpty()){
|
||||
queue.putIfAbsent(galleryTask.getGid(), galleryTask);
|
||||
private boolean isVerifiedArchive(File archive){
|
||||
if(!archive.isFile())
|
||||
return false;
|
||||
}
|
||||
|
||||
//查询hah下载路径中,是否存在该任务下载路径,存在则为下载中或下载完成任务,加入队列
|
||||
if(new File(downloadPath + galleryTask.getGid()).isDirectory()){
|
||||
queue.putIfAbsent(galleryTask.getGid(), galleryTask);
|
||||
return false;
|
||||
}
|
||||
|
||||
|
||||
//查询存放路径中是否含有该任务的压缩包,存在则为下载完成任务
|
||||
if(new File(storagePath + galleryTask.getName() + "/" + galleryTask.getName() + ".zip").exists()){
|
||||
galleryTask.setStatus(GalleryTask.COMPRESS_COMPLETE);
|
||||
CustomUtil.notifyMe(String.format("任务:%s在添加时已下载完成,更新任务状态", galleryTask.getName()));
|
||||
long size = archive.length();
|
||||
long now = System.currentTimeMillis();
|
||||
VerifiedArchive cached = verifiedArchives.get(archive.getPath());
|
||||
if(cached != null && cached.size() == size && now - cached.verifiedAt() < ARCHIVE_REVERIFY_MILLIS)
|
||||
return true;
|
||||
if(!isValidArchive(archive)){
|
||||
verifiedArchives.remove(archive.getPath());
|
||||
return false;
|
||||
}
|
||||
verifiedArchives.put(archive.getPath(), new VerifiedArchive(size, now));
|
||||
return true;
|
||||
}
|
||||
|
||||
private void notifyAlreadyStored(String taskName){
|
||||
if(!notificationsEnabled)
|
||||
return;
|
||||
notificationExecutor.execute(() ->
|
||||
CustomUtil.notifyMe(String.format("任务:%s在添加时已下载完成,更新任务状态", taskName)));
|
||||
}
|
||||
|
||||
//异常情况,发送通知
|
||||
CustomUtil.notifyMe(String.format("任务:%s存在名字,但是下载路径为空且不存在压缩包", galleryTask.getName() ));
|
||||
return false;
|
||||
private File findDirectoryByGid(File parentDirectory, int gid){
|
||||
File[] directories = parentDirectory.listFiles(File::isDirectory);
|
||||
if(directories == null)
|
||||
return null;
|
||||
|
||||
for(File directory : directories)
|
||||
if(matchesGid(directory.getName(), gid))
|
||||
return directory;
|
||||
return null;
|
||||
}
|
||||
|
||||
private boolean matchesGid(String name, int gid){
|
||||
String gidMarker = "[" + gid;
|
||||
int markerIndex = name.lastIndexOf(gidMarker);
|
||||
if(markerIndex < 0)
|
||||
return false;
|
||||
|
||||
int suffixIndex = markerIndex + gidMarker.length();
|
||||
return suffixIndex < name.length()
|
||||
&& (name.charAt(suffixIndex) == ']' || name.charAt(suffixIndex) == '-');
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,92 @@
|
||||
package lion.Service;
|
||||
|
||||
import io.netty.channel.Channel;
|
||||
import io.netty.util.AttributeKey;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
|
||||
import java.util.Set;
|
||||
import java.util.concurrent.ConcurrentHashMap;
|
||||
|
||||
/**
|
||||
* 「哪条通道是主站」的唯一登记处。
|
||||
*
|
||||
* <p>历史缺陷:主站重连期间可能出现多条已认证通道,节点先引用 X,随后被后建立的 Y
|
||||
* 覆盖;Y 断开时把引用清空,而仍可用的 X 继续发送心跳与任务消息,节点却因为引用为空
|
||||
* 不再上报任何任务状态。主站侧的存活探测又能收到 X 上的响应,双方于是都判定
|
||||
* 「连接正常」,未完成任务的状态就此永久卡住。
|
||||
*
|
||||
* <p>因此把引用管理收敛到这里,并保留全部已认证通道:首选通道一旦失效,立即回退到
|
||||
* 其它仍可用的已认证通道;引用被误清时,任何一条已认证通道上的消息也能把它恢复。
|
||||
* 这样「有已认证通道在,就一定能上报」成为不变式,不必等人工重连或下一个探测周期。
|
||||
*
|
||||
* <p>「已认证」的口径是「该通道发来过只有主站会发的消息类型」。事故中出问题的那条通道
|
||||
* 正是靠可用性探测消息证明自己仍在工作的,若只认 {@code IdentityMessage},这条通道
|
||||
* 就永远无法重新取得引用。节点数据端口本就只应面向主站开放,凭证由网络边界提供。
|
||||
*/
|
||||
@Slf4j
|
||||
public class PrimaryChannelTracker {
|
||||
|
||||
static final AttributeKey<Boolean> AUTHENTICATED = AttributeKey.valueOf("primaryAuthenticated");
|
||||
|
||||
/** 全部已认证通道,用于在首选通道失效时立即回退。 */
|
||||
private final Set<Channel> authenticated = ConcurrentHashMap.newKeySet();
|
||||
|
||||
private volatile Channel primary;
|
||||
|
||||
/** 收到只有主站会发的消息:认证该通道;若当前引用不是它,则改指向它。 */
|
||||
public void onPrimaryMessage(Channel channel) {
|
||||
markAuthenticated(channel);
|
||||
if (primary == null) {
|
||||
primary = channel;
|
||||
log.info("主站通道引用已建立");
|
||||
} else if (channel != primary) {
|
||||
switchTo(channel);
|
||||
}
|
||||
}
|
||||
|
||||
private void markAuthenticated(Channel channel) {
|
||||
if (Boolean.TRUE.equals(channel.attr(AUTHENTICATED).get()))
|
||||
return;
|
||||
channel.attr(AUTHENTICATED).set(Boolean.TRUE);
|
||||
channel.closeFuture().addListener(ignored -> unregister(channel));
|
||||
authenticated.add(channel);
|
||||
}
|
||||
|
||||
/** 通道退出:移出已认证集合,必要时把引用移交给其它仍可用的通道。 */
|
||||
public void unregister(Channel channel) {
|
||||
authenticated.remove(channel);
|
||||
if (channel != primary)
|
||||
return;
|
||||
primary = firstUsable();
|
||||
}
|
||||
|
||||
private void switchTo(Channel channel) {
|
||||
log.info("主站通道引用已切换到仍在工作的通道");
|
||||
primary = channel;
|
||||
}
|
||||
|
||||
public Channel current() {
|
||||
Channel channel = primary;
|
||||
if (channel != null && channel.isActive())
|
||||
return channel;
|
||||
// 首选通道已失效:立即回退,避免出现「有可用通道却无人上报」的静默状态。
|
||||
Channel fallback = firstUsable();
|
||||
if (fallback != null && fallback != channel) {
|
||||
log.info("主站首选通道不可用,回退到其它已认证通道");
|
||||
primary = fallback;
|
||||
}
|
||||
return fallback;
|
||||
}
|
||||
|
||||
/** 引用是否存在且仍可写。 */
|
||||
public boolean usable() {
|
||||
return current() != null;
|
||||
}
|
||||
|
||||
private Channel firstUsable() {
|
||||
for (Channel channel : authenticated)
|
||||
if (channel.isActive())
|
||||
return channel;
|
||||
return null;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,34 @@
|
||||
package lion.Service;
|
||||
|
||||
/**
|
||||
* 「是否该主动重新唤起主站」的判定。
|
||||
*
|
||||
* <p>节点持有待上报的任务状态却没有可用通道时,只等主站下一个探测周期可能长达半小时;
|
||||
* 期间用户在下载器里看到的是任务永远停在「已提交」。这里按固定间隔限流地允许重连,
|
||||
* 既能在几十秒内自愈,又不会在主站确实离线时形成重连风暴。
|
||||
*
|
||||
* <p>把判定单独拆出来是为了能直接对时间轴写断言,不必启动 Netty。
|
||||
*/
|
||||
public class RekickPolicy {
|
||||
|
||||
/** 两次主动唤起之间的最小间隔。 */
|
||||
static final long MIN_INTERVAL_MILLIS = 30_000L;
|
||||
|
||||
private long lastAttemptMillis;
|
||||
|
||||
/**
|
||||
* 本轮是否应当唤起主站。
|
||||
*
|
||||
* @param pendingReports 是否有待上报的任务状态
|
||||
* @param channelUsable 是否已存在可用通道
|
||||
* @param nowMillis 当前时间
|
||||
*/
|
||||
public boolean shouldRekick(boolean pendingReports, boolean channelUsable, long nowMillis) {
|
||||
if (!pendingReports || channelUsable)
|
||||
return false;
|
||||
if (lastAttemptMillis != 0 && nowMillis - lastAttemptMillis < MIN_INTERVAL_MILLIS)
|
||||
return false;
|
||||
lastAttemptMillis = nowMillis;
|
||||
return true;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,369 @@
|
||||
package lion.Service;
|
||||
|
||||
import com.fasterxml.jackson.databind.JsonNode;
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
import lion.CustomUtil;
|
||||
import lion.Message.Main.SubscriptionAccountSnapshot;
|
||||
import lion.Message.Main.SubscriptionBindingSnapshot;
|
||||
import lion.Message.Main.SubscriptionSnapshotMessage;
|
||||
import lion.Message.Main.SubscriptionSnapshotPayload;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
|
||||
import javax.crypto.Mac;
|
||||
import javax.crypto.spec.SecretKeySpec;
|
||||
import java.io.ByteArrayInputStream;
|
||||
import java.io.IOException;
|
||||
import java.io.UncheckedIOException;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.nio.file.*;
|
||||
import java.security.MessageDigest;
|
||||
import java.util.*;
|
||||
import java.util.concurrent.atomic.AtomicReference;
|
||||
import java.util.zip.GZIPInputStream;
|
||||
|
||||
/**
|
||||
* Last-known-good subscription data used by the standby HTTP server.
|
||||
* The store never downloads upstream subscriptions and never stores upstream keys.
|
||||
*/
|
||||
@Slf4j
|
||||
public final class SubscriptionSnapshotStore {
|
||||
public static final byte APPLY_SUCCESS = 0;
|
||||
public static final byte APPLY_INVALID = 1;
|
||||
public static final byte APPLY_IO_ERROR = 2;
|
||||
public static final byte APPLY_OLD = 3;
|
||||
|
||||
private final Path root;
|
||||
private final byte[] syncSecret;
|
||||
private final long maxStaleMillis;
|
||||
private final int maxPayloadBytes;
|
||||
private final ObjectMapper objectMapper = CustomUtil.objectMapper;
|
||||
private final AtomicReference<Snapshot> current = new AtomicReference<>();
|
||||
/**
|
||||
* 最近一次收到主站存活信号的时刻(本地接收时间)。
|
||||
*
|
||||
* <p>「过期」的语义是「主站已失联」,因此新鲜度必须由「主站的任何存活信号」驱动,
|
||||
* 而不能只看「收到快照」。快照是内容寻址的:内容长期不变时主站没有理由反复推送,
|
||||
* 若以收到快照为准,备机会在内容不动的第 7 天误判过期;反过来,若主站为此定时重发
|
||||
* 整份快照,又只为续期而传输约 346 KiB/次。
|
||||
*
|
||||
* <p>主站本就有常规存活探测(每 30 分钟的 AvailableCheckMessage),节点回它以
|
||||
* ResponseMessage。把该信号纳入新鲜度后:过期 ⇔ 主站失联超过有效期,
|
||||
* 语义精确,且不再需要任何为「续期」而生的专用推送。
|
||||
*
|
||||
* <p>时刻取自节点本地时钟、不参与签名,攻击者无法通过重放旧 revision 续期。
|
||||
*/
|
||||
private volatile long lastPrimaryContactAt;
|
||||
|
||||
public SubscriptionSnapshotStore(Path root, String syncSecret, long maxStaleSeconds, int maxPayloadBytes) {
|
||||
this.root = Objects.requireNonNull(root);
|
||||
this.syncSecret = syncSecret == null ? new byte[0] : syncSecret.getBytes(StandardCharsets.UTF_8);
|
||||
this.maxStaleMillis = Math.max(0, maxStaleSeconds) * 1000L;
|
||||
this.maxPayloadBytes = maxPayloadBytes;
|
||||
}
|
||||
|
||||
public void load() {
|
||||
List<String> candidates = new ArrayList<>();
|
||||
try {
|
||||
Path pointer = root.resolve("current-revision");
|
||||
if (Files.isRegularFile(pointer)) {
|
||||
String revision = Files.readString(pointer, StandardCharsets.UTF_8).trim();
|
||||
if (isRevision(revision)) candidates.add(revision);
|
||||
}
|
||||
Path snapshots = root.resolve("snapshots");
|
||||
if (Files.isDirectory(snapshots)) {
|
||||
try (var stream = Files.list(snapshots)) {
|
||||
stream.filter(Files::isDirectory)
|
||||
.map(path -> path.getFileName().toString())
|
||||
.filter(SubscriptionSnapshotStore::isRevision)
|
||||
.filter(revision -> !candidates.contains(revision))
|
||||
.sorted(Comparator.reverseOrder())
|
||||
.forEach(candidates::add);
|
||||
}
|
||||
}
|
||||
} catch (Exception e) {
|
||||
log.warn("扫描订阅快照失败: {}", e.getMessage());
|
||||
}
|
||||
for (String revision : candidates) {
|
||||
try {
|
||||
Snapshot snapshot = loadSnapshot(root.resolve("snapshots").resolve(revision));
|
||||
current.set(snapshot);
|
||||
// 重启后还没有收到过主站快照,先以落盘内容的生成时间作为新鲜度起点,
|
||||
// 否则重启即视为「刚同步过」,会让超期快照被错误续期。
|
||||
lastPrimaryContactAt = snapshot.generatedAt();
|
||||
writePointer(revision);
|
||||
log.info("加载订阅快照成功 revision={} accounts={} bindings={}", shortRevision(revision), snapshot.accountCount(), snapshot.bindingCount());
|
||||
return;
|
||||
} catch (Exception e) {
|
||||
log.warn("订阅快照损坏,尝试上一版本 revision={}: {}", shortRevision(revision), e.getMessage());
|
||||
}
|
||||
}
|
||||
current.set(null);
|
||||
log.warn("没有可用的订阅快照,备机订阅暂不可用");
|
||||
}
|
||||
|
||||
public ApplyResult apply(SubscriptionSnapshotMessage message) {
|
||||
Path staging = null;
|
||||
try {
|
||||
if (message == null || message.getSchemaVersion() != 1 || !isRevision(message.getRevision()))
|
||||
return new ApplyResult(APPLY_INVALID, "消息版本或 revision 非法");
|
||||
if (syncSecret.length == 0)
|
||||
return new ApplyResult(APPLY_INVALID, "同步密钥未配置");
|
||||
|
||||
byte[] compressed = decodeBase64(message.getPayloadBase64(), maxPayloadBytes);
|
||||
if (!constantEquals(message.getPayloadSha256(), sha256(compressed)))
|
||||
return new ApplyResult(APPLY_INVALID, "payload SHA-256 校验失败");
|
||||
String signatureInput = message.getSchemaVersion() + "\n" + message.getRevision() + "\n"
|
||||
+ message.getGeneratedAt() + "\n" + message.getPayloadSha256();
|
||||
if (!constantEquals(message.getSignature(), hmac(signatureInput.getBytes(StandardCharsets.UTF_8))))
|
||||
return new ApplyResult(APPLY_INVALID, "快照签名校验失败");
|
||||
|
||||
byte[] payloadBytes = gunzip(compressed, maxPayloadBytes);
|
||||
if (!constantEquals(message.getRevision(), sha256(payloadBytes)))
|
||||
return new ApplyResult(APPLY_INVALID, "revision 与 payload 不一致");
|
||||
SubscriptionSnapshotPayload payload = objectMapper.readValue(payloadBytes, SubscriptionSnapshotPayload.class);
|
||||
SnapshotData data = validatePayload(payload);
|
||||
Snapshot old = current.get();
|
||||
if (old != null) {
|
||||
if (message.getRevision().equals(old.revision())) {
|
||||
// 内容未变但主站仍在同步:刷新新鲜度,避免备机因「内容长期不变」而误判过期。
|
||||
lastPrimaryContactAt = System.currentTimeMillis();
|
||||
return new ApplyResult(APPLY_OLD, "revision 已存在");
|
||||
}
|
||||
if (message.getGeneratedAt() < old.generatedAt())
|
||||
return new ApplyResult(APPLY_OLD, "快照时间早于当前版本");
|
||||
}
|
||||
|
||||
Path snapshots = root.resolve("snapshots");
|
||||
Files.createDirectories(snapshots);
|
||||
staging = snapshots.resolve(".staging-" + message.getRevision());
|
||||
deleteRecursively(staging);
|
||||
Files.createDirectories(staging.resolve("accounts"));
|
||||
for (AccountFiles account : data.accounts.values()) {
|
||||
Path dir = staging.resolve("accounts").resolve(String.valueOf(account.accountId()));
|
||||
Files.createDirectories(dir);
|
||||
Files.write(dir.resolve("v2ray.txt"), account.v2(), StandardOpenOption.CREATE_NEW);
|
||||
Files.write(dir.resolve("clash.yaml"), account.clash(), StandardOpenOption.CREATE_NEW);
|
||||
}
|
||||
Map<String, Object> manifest = new LinkedHashMap<>();
|
||||
manifest.put("revision", message.getRevision());
|
||||
manifest.put("generatedAt", message.getGeneratedAt());
|
||||
// 保留主站签名对应的原始 JSON 字节,重启时不依赖 Jackson 再序列化顺序。
|
||||
manifest.put("payloadBase64", Base64.getEncoder().encodeToString(payloadBytes));
|
||||
Files.write(staging.resolve("manifest.json"), objectMapper.writeValueAsBytes(manifest), StandardOpenOption.CREATE_NEW);
|
||||
|
||||
Path destination = snapshots.resolve(message.getRevision());
|
||||
if (Files.exists(destination))
|
||||
deleteRecursively(destination);
|
||||
atomicMove(staging, destination);
|
||||
writePointer(message.getRevision());
|
||||
|
||||
Snapshot snapshot = new Snapshot(message.getRevision(), message.getGeneratedAt(), data.byKeyHash,
|
||||
data.accounts.size(), data.bindingCount);
|
||||
current.set(snapshot);
|
||||
lastPrimaryContactAt = System.currentTimeMillis();
|
||||
cleanupOldSnapshots(message.getRevision());
|
||||
return new ApplyResult(APPLY_SUCCESS, "同步成功");
|
||||
} catch (Exception e) {
|
||||
log.error("应用订阅快照失败: {}", e.getMessage());
|
||||
return new ApplyResult(APPLY_IO_ERROR, e.getMessage() == null ? e.getClass().getSimpleName() : e.getMessage());
|
||||
} finally {
|
||||
if (staging != null && Files.exists(staging)) {
|
||||
try { deleteRecursively(staging); }
|
||||
catch (IOException e) { log.warn("清理订阅快照 staging 失败: {}", staging); }
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private void writePointer(String revision) throws IOException {
|
||||
Files.createDirectories(root);
|
||||
Path pointerTmp = root.resolve("current-revision.tmp");
|
||||
Files.writeString(pointerTmp, revision + "\n", StandardCharsets.UTF_8,
|
||||
StandardOpenOption.CREATE, StandardOpenOption.TRUNCATE_EXISTING);
|
||||
atomicMove(pointerTmp, root.resolve("current-revision"));
|
||||
}
|
||||
|
||||
public Lookup lookup(String client, String publicKey) {
|
||||
Snapshot snapshot = current.get();
|
||||
if (snapshot == null || isStale(System.currentTimeMillis()))
|
||||
return null;
|
||||
AccountFiles account = snapshot.byKeyHash().get(sha256(publicKey.getBytes(StandardCharsets.UTF_8)));
|
||||
if (account == null)
|
||||
return null;
|
||||
return new Lookup("v2".equals(client) ? account.v2() : account.clash(), snapshot.revision(), snapshot.generatedAt());
|
||||
}
|
||||
|
||||
public Status status() {
|
||||
Snapshot snapshot = current.get();
|
||||
if (snapshot == null)
|
||||
return new Status("unavailable", null, 0, 0, 0);
|
||||
long now = System.currentTimeMillis();
|
||||
long age = Math.max(0, now - lastPrimaryContactAt);
|
||||
return new Status(isStale(now) ? "expired" : "ready", snapshot.revision(), snapshot.accountCount(), snapshot.bindingCount(), age);
|
||||
}
|
||||
|
||||
/**
|
||||
* 主站是否已失联超过最大有效期。
|
||||
*
|
||||
* <p>由主站的任何存活信号刷新(见 {@link #markPrimaryContact()}),不限于快照。
|
||||
*/
|
||||
private boolean isStale(long now) {
|
||||
return maxStaleMillis > 0 && now - lastPrimaryContactAt > maxStaleMillis;
|
||||
}
|
||||
|
||||
/**
|
||||
* 记录一次「主站仍在」的证据。
|
||||
*
|
||||
* <p>用于快照之外的常规存活信号(例如主站每 30 分钟的可用性检查)。
|
||||
* 只要主站在线,本节点的新鲜度就会持续被刷新,因此备机不会因为
|
||||
* 「订阅内容长期不变、主站没理由重发快照」而被判过期。
|
||||
*/
|
||||
public void markPrimaryContact() {
|
||||
lastPrimaryContactAt = System.currentTimeMillis();
|
||||
}
|
||||
|
||||
private Snapshot loadSnapshot(Path directory) throws IOException {
|
||||
JsonNode manifest = objectMapper.readTree(Files.readAllBytes(directory.resolve("manifest.json")));
|
||||
String revision = manifest.path("revision").asText();
|
||||
long generatedAt = manifest.path("generatedAt").asLong(0);
|
||||
byte[] payloadBytes = decodeBase64(manifest.path("payloadBase64").asText(null), maxPayloadBytes);
|
||||
SubscriptionSnapshotPayload payload = objectMapper.readValue(payloadBytes, SubscriptionSnapshotPayload.class);
|
||||
SnapshotData data = validatePayload(payload);
|
||||
if (!isRevision(revision) || !constantEquals(revision, sha256(payloadBytes)))
|
||||
throw new IOException("快照 manifest revision 校验失败");
|
||||
Map<Integer, AccountFiles> accounts = new HashMap<>();
|
||||
for (SubscriptionAccountSnapshot account : payload.getAccounts()) {
|
||||
Path dir = directory.resolve("accounts").resolve(String.valueOf(account.getAccountId()));
|
||||
byte[] v2 = Files.readAllBytes(dir.resolve("v2ray.txt"));
|
||||
byte[] clash = Files.readAllBytes(dir.resolve("clash.yaml"));
|
||||
if (!constantEquals(account.getV2Sha256(), sha256(v2)) || !constantEquals(account.getClashSha256(), sha256(clash)))
|
||||
throw new IOException("缓存文件校验失败");
|
||||
accounts.put(account.getAccountId(), new AccountFiles(account.getAccountId(), v2, clash));
|
||||
}
|
||||
Map<String, AccountFiles> byKey = new HashMap<>();
|
||||
for (SubscriptionBindingSnapshot binding : payload.getBindings())
|
||||
byKey.put(binding.getPublicKeySha256(), accounts.get(binding.getAccountId()));
|
||||
return new Snapshot(revision, generatedAt, byKey, accounts.size(), payload.getBindings().size());
|
||||
}
|
||||
|
||||
private SnapshotData validatePayload(SubscriptionSnapshotPayload payload) throws IOException {
|
||||
if (payload == null || payload.getSchemaVersion() != 1 || payload.getAccounts() == null || payload.getBindings() == null)
|
||||
throw new IOException("payload 版本或字段非法");
|
||||
if (payload.getAccounts().size() > 100 || payload.getBindings().size() > 10000)
|
||||
throw new IOException("快照条目数量超限");
|
||||
Map<Integer, AccountFiles> accounts = new HashMap<>();
|
||||
for (SubscriptionAccountSnapshot account : payload.getAccounts()) {
|
||||
if (account == null || account.getAccountId() == null || account.getAccountId() <= 0 || !account.isEnabled()
|
||||
|| !isBase64Sha(account.getV2Sha256()) || !isBase64Sha(account.getClashSha256()))
|
||||
throw new IOException("账号字段非法");
|
||||
byte[] v2 = decodeBase64(account.getV2ContentBase64(), maxPayloadBytes);
|
||||
byte[] clash = decodeBase64(account.getClashContentBase64(), maxPayloadBytes);
|
||||
if (!constantEquals(account.getV2Sha256(), sha256(v2)) || !constantEquals(account.getClashSha256(), sha256(clash)))
|
||||
throw new IOException("账号缓存 SHA-256 校验失败");
|
||||
if (accounts.put(account.getAccountId(), new AccountFiles(account.getAccountId(), v2, clash)) != null)
|
||||
throw new IOException("账号 ID 重复");
|
||||
}
|
||||
Map<String, AccountFiles> byKey = new HashMap<>();
|
||||
for (SubscriptionBindingSnapshot binding : payload.getBindings()) {
|
||||
if (binding == null || !isBase64Sha(binding.getPublicKeySha256()) || !accounts.containsKey(binding.getAccountId()))
|
||||
throw new IOException("绑定字段或账号引用非法");
|
||||
if (byKey.put(binding.getPublicKeySha256(), accounts.get(binding.getAccountId())) != null)
|
||||
throw new IOException("公开 Key Hash 重复");
|
||||
}
|
||||
return new SnapshotData(accounts, byKey, payload.getBindings().size());
|
||||
}
|
||||
|
||||
private void cleanupOldSnapshots(String currentRevision) {
|
||||
try {
|
||||
Path snapshots = root.resolve("snapshots");
|
||||
List<Path> dirs;
|
||||
try (var stream = Files.list(snapshots)) {
|
||||
dirs = stream.filter(Files::isDirectory)
|
||||
.filter(p -> !p.getFileName().toString().startsWith(".staging-"))
|
||||
.sorted(Comparator.comparingLong(SubscriptionSnapshotStore::lastModified).reversed())
|
||||
.toList();
|
||||
}
|
||||
dirs.stream().filter(p -> !p.getFileName().toString().equals(currentRevision))
|
||||
.skip(1).forEach(p -> {
|
||||
try { deleteRecursively(p); } catch (IOException e) { log.warn("清理旧订阅快照失败: {}", p); }
|
||||
});
|
||||
} catch (IOException e) {
|
||||
log.warn("扫描旧订阅快照失败: {}", e.getMessage());
|
||||
}
|
||||
}
|
||||
|
||||
private static byte[] gunzip(byte[] compressed, int limit) throws IOException {
|
||||
try (GZIPInputStream input = new GZIPInputStream(new ByteArrayInputStream(compressed))) {
|
||||
byte[] buffer = new byte[8192];
|
||||
var output = new java.io.ByteArrayOutputStream();
|
||||
int total = 0, read;
|
||||
while ((read = input.read(buffer)) != -1) {
|
||||
total += read;
|
||||
if (total > limit) throw new IOException("解压后 payload 超限");
|
||||
output.write(buffer, 0, read);
|
||||
}
|
||||
return output.toByteArray();
|
||||
}
|
||||
}
|
||||
|
||||
private String hmac(byte[] input) throws Exception {
|
||||
Mac mac = Mac.getInstance("HmacSHA256");
|
||||
mac.init(new SecretKeySpec(syncSecret, "HmacSHA256"));
|
||||
return hex(mac.doFinal(input));
|
||||
}
|
||||
|
||||
private static byte[] decodeBase64(String value, int maxBytes) throws IOException {
|
||||
if (value == null || value.length() > maxBytes * 2L)
|
||||
throw new IOException("Base64 数据超限");
|
||||
try {
|
||||
byte[] result = Base64.getDecoder().decode(value);
|
||||
if (result.length > maxBytes) throw new IOException("数据超限");
|
||||
return result;
|
||||
} catch (IllegalArgumentException e) {
|
||||
throw new IOException("Base64 数据非法");
|
||||
}
|
||||
}
|
||||
|
||||
private static String sha256(byte[] bytes) {
|
||||
try { return hex(MessageDigest.getInstance("SHA-256").digest(bytes)); }
|
||||
catch (Exception e) { throw new IllegalStateException(e); }
|
||||
}
|
||||
|
||||
private static String hex(byte[] bytes) {
|
||||
return HexFormat.of().formatHex(bytes);
|
||||
}
|
||||
|
||||
private static boolean constantEquals(String expected, String actual) {
|
||||
return expected != null && MessageDigest.isEqual(expected.toLowerCase(Locale.ROOT).getBytes(StandardCharsets.US_ASCII),
|
||||
actual.toLowerCase(Locale.ROOT).getBytes(StandardCharsets.US_ASCII));
|
||||
}
|
||||
|
||||
private static boolean isRevision(String value) { return isBase64Sha(value); }
|
||||
private static boolean isBase64Sha(String value) { return value != null && value.matches("[0-9a-fA-F]{64}"); }
|
||||
private static String shortRevision(String revision) { return revision == null ? null : revision.substring(0, Math.min(12, revision.length())); }
|
||||
|
||||
private static long lastModified(Path path) {
|
||||
try { return Files.getLastModifiedTime(path).toMillis(); }
|
||||
catch (IOException e) { return 0; }
|
||||
}
|
||||
|
||||
private static void atomicMove(Path source, Path target) throws IOException {
|
||||
try { Files.move(source, target, StandardCopyOption.ATOMIC_MOVE, StandardCopyOption.REPLACE_EXISTING); }
|
||||
catch (AtomicMoveNotSupportedException e) { Files.move(source, target, StandardCopyOption.REPLACE_EXISTING); }
|
||||
}
|
||||
|
||||
private static void deleteRecursively(Path path) throws IOException {
|
||||
if (!Files.exists(path)) return;
|
||||
try (var stream = Files.walk(path)) {
|
||||
stream.sorted(Comparator.reverseOrder()).forEach(p -> {
|
||||
try { Files.deleteIfExists(p); } catch (IOException e) { throw new UncheckedIOException(e); }
|
||||
});
|
||||
} catch (UncheckedIOException e) { throw e.getCause(); }
|
||||
}
|
||||
|
||||
public record ApplyResult(byte code, String message) {}
|
||||
public record Lookup(byte[] content, String revision, long generatedAt) {}
|
||||
public record Status(String state, String revision, int accountCount, int bindingCount, long ageMillis) {}
|
||||
private record AccountFiles(Integer accountId, byte[] v2, byte[] clash) {}
|
||||
private record SnapshotData(Map<Integer, AccountFiles> accounts, Map<String, AccountFiles> byKeyHash, int bindingCount) {}
|
||||
private record Snapshot(String revision, long generatedAt, Map<String, AccountFiles> byKeyHash, int accountCount, int bindingCount) {}
|
||||
}
|
||||
@@ -1,11 +1,14 @@
|
||||
package lion;
|
||||
|
||||
import io.netty.util.concurrent.Promise;
|
||||
import lion.Config.Config;
|
||||
import lion.Domain.GalleryTask;
|
||||
import lion.Message.*;
|
||||
import lion.Message.Main.*;
|
||||
import lion.Service.DeleteService;
|
||||
import lion.Service.SubscriptionSnapshotStore;
|
||||
import lion.Service.DownloadCheckService;
|
||||
import lion.Service.PrimaryChannelTracker;
|
||||
import lion.Service.RekickPolicy;
|
||||
import io.netty.bootstrap.ServerBootstrap;
|
||||
import io.netty.channel.*;
|
||||
import io.netty.channel.nio.NioEventLoopGroup;
|
||||
@@ -14,7 +17,6 @@ import io.netty.channel.socket.nio.NioSocketChannel;
|
||||
import io.netty.handler.codec.LengthFieldBasedFrameDecoder;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
|
||||
|
||||
import java.net.InetSocketAddress;
|
||||
import java.net.Socket;
|
||||
import java.util.HashMap;
|
||||
@@ -25,11 +27,25 @@ import java.util.concurrent.locks.ReentrantLock;
|
||||
@Slf4j
|
||||
public class storageNode {
|
||||
|
||||
ChannelFuture channelFuture;
|
||||
// 主站通道引用由 PrimaryChannelTracker 统一管理,避免重复认证把可用通道误清空。
|
||||
final PrimaryChannelTracker primaryChannel = new PrimaryChannelTracker();
|
||||
|
||||
Channel server;
|
||||
// 有待上报任务却无可用通道时,主动重新唤起主站,避免干等下一个探测周期。
|
||||
final RekickPolicy rekickPolicy = new RekickPolicy();
|
||||
|
||||
Channel node;
|
||||
// 重新唤起主站是阻塞式端口探测(最多 21×3 秒),不得占用 5 秒调度线程,
|
||||
// 否则下载进度扫描与压缩都会被拖住;同一时刻只允许一次探测。
|
||||
final java.util.concurrent.atomic.AtomicBoolean rekicking = new java.util.concurrent.atomic.AtomicBoolean();
|
||||
|
||||
// 专门用于阻塞式重连探测;不能复用单线程的 checkThreadPool,
|
||||
// 在同一个线程池里提交阻塞任务会把调度线程自己堵住。
|
||||
final ExecutorService rekickExecutor = Executors.newSingleThreadExecutor(r -> {
|
||||
Thread thread = new Thread(r, "storage-node-rekick");
|
||||
thread.setDaemon(true);
|
||||
return thread;
|
||||
});
|
||||
|
||||
volatile Channel node;
|
||||
|
||||
DownloadCheckService downloadCheckService;
|
||||
|
||||
@@ -39,22 +55,28 @@ public class storageNode {
|
||||
|
||||
ScheduledExecutorService checkThreadPool;
|
||||
|
||||
HashMap<Integer, Promise<AbstractMessage>> promises;
|
||||
|
||||
int counter;
|
||||
|
||||
ReentrantLock lock;
|
||||
|
||||
final SubscriptionSnapshotStore subscriptionSnapshotStore;
|
||||
|
||||
final ExecutorService subscriptionApplyExecutor;
|
||||
|
||||
public static String storagePath = "/root/gallery/gallery/";
|
||||
|
||||
public storageNode(){
|
||||
queue = new HashMap<>();
|
||||
public storageNode(SubscriptionSnapshotStore subscriptionSnapshotStore){
|
||||
this.subscriptionSnapshotStore = subscriptionSnapshotStore;
|
||||
this.subscriptionApplyExecutor = Executors.newSingleThreadExecutor(r -> {
|
||||
Thread thread = new Thread(r, "subscription-snapshot-apply");
|
||||
thread.setDaemon(true);
|
||||
return thread;
|
||||
});
|
||||
queue = new ConcurrentHashMap<>();
|
||||
tempQueue = new HashMap<>();
|
||||
lock = new ReentrantLock();
|
||||
counter = 0;
|
||||
promises = new HashMap<>();
|
||||
|
||||
channelFuture = new ServerBootstrap()
|
||||
int real_port = CustomUtil._findIdlePort(26321);
|
||||
|
||||
ChannelFuture bindFuture = new ServerBootstrap()
|
||||
.channel(NioServerSocketChannel.class)
|
||||
.group(new NioEventLoopGroup())
|
||||
.childHandler(new ChannelInitializer<NioSocketChannel>() {
|
||||
@@ -64,25 +86,100 @@ public class storageNode {
|
||||
channel.pipeline().addLast(new MessageCodec());
|
||||
channel.pipeline().addLast(new MyChannelInboundHandlerAdapter(tempQueue));
|
||||
}
|
||||
})
|
||||
.bind(26321);
|
||||
}).bind(real_port);
|
||||
// 必须等绑定结果:端口被别人抢到(探测与绑定之间的 TOCTOU)时,若不检查,
|
||||
// 节点会「启动成功」却根本没在监听,主站怎么连都连不上。
|
||||
try {
|
||||
bindFuture.sync();
|
||||
log.info("listening on port {}", real_port);
|
||||
} catch (InterruptedException e) {
|
||||
Thread.currentThread().interrupt();
|
||||
throw new IllegalStateException("绑定端口 " + real_port + " 被中断", e);
|
||||
} catch (Exception e) {
|
||||
throw new IllegalStateException("绑定端口 " + real_port + " 失败", e);
|
||||
}
|
||||
|
||||
try(Socket socket = new Socket()){
|
||||
socket.connect(new InetSocketAddress("lionwebsite.xyz", 26322));
|
||||
} catch (Exception ignored) {}
|
||||
downloadCheckService = new DownloadCheckService(queue, promises);
|
||||
wakeMainServer();
|
||||
downloadCheckService = new DownloadCheckService(queue);
|
||||
checkThreadPool = Executors.newScheduledThreadPool(1);
|
||||
checkThreadPool.scheduleAtFixedRate(this::mainThread, 5, 5, TimeUnit.SECONDS);
|
||||
}
|
||||
|
||||
/**
|
||||
* 在独立线程里重新唤起主站。
|
||||
*
|
||||
* <p>{@link #wakeMainServer()} 是阻塞的端口探测,最坏可占 21×3 秒;直接在
|
||||
* {@link #mainThread()} 里调用会让下载扫描与压缩停摆,因此投递到独立线程执行。
|
||||
* 该执行器与 5 秒调度器分离,避免阻塞任务把调度线程自己堵住;
|
||||
* 同一时刻只允许一次探测,防止主站长时间离线时堆积。
|
||||
*/
|
||||
void rekickMainServerAsync(){
|
||||
if (!rekicking.compareAndSet(false, true))
|
||||
return;
|
||||
try {
|
||||
rekickExecutor.execute(() -> {
|
||||
try {
|
||||
wakeMainServer();
|
||||
} finally {
|
||||
rekicking.set(false);
|
||||
}
|
||||
});
|
||||
} catch (Exception e) {
|
||||
rekicking.set(false);
|
||||
log.warn("安排主站重连失败:{}", e.getMessage());
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 按既定顺序敲主站监听端口,请它回来建立数据通道。
|
||||
*
|
||||
* <p>启动时调用一次,另在「有待上报任务但无可用通道」时按 {@link RekickPolicy}
|
||||
* 限流重试。主站侧的可用性探测只能覆盖它认为连接仍在的情况;一旦双方都判定
|
||||
* 对方不在,只有本方法能重新建立通道,否则任务状态会一直卡在未完成。
|
||||
*/
|
||||
void wakeMainServer(){
|
||||
int i;
|
||||
for(i=0; i<=20; i++) {
|
||||
try (Socket socket = new Socket()) {
|
||||
socket.setSoTimeout(3000);
|
||||
log.info("wake up main server on port {}", 26322 + i);
|
||||
socket.connect(new InetSocketAddress("lionwebsite.xyz", 26322 + i));
|
||||
byte[] bytes = socket.getInputStream().readAllBytes();
|
||||
if(bytes.length > 0 && new String(bytes).equals("lionwebsite")) {
|
||||
break;
|
||||
}
|
||||
} catch (Exception ignored) {
|
||||
}
|
||||
}
|
||||
if (i==20) {
|
||||
log.info("server connect failed");
|
||||
}
|
||||
}
|
||||
|
||||
public void mainThread(){
|
||||
try {
|
||||
lock.lock();
|
||||
if(!tempQueue.isEmpty()){
|
||||
queue.putAll(tempQueue);
|
||||
tempQueue.clear();
|
||||
try {
|
||||
if(!tempQueue.isEmpty()){
|
||||
queue.putAll(tempQueue);
|
||||
tempQueue.clear();
|
||||
}
|
||||
} finally {
|
||||
lock.unlock();
|
||||
}
|
||||
lock.unlock();
|
||||
|
||||
// 有暂存内容却无可用通道:告警,并按限流主动重新唤起主站。
|
||||
// 这里只告警与重连,不提前返回——本地下载与压缩必须继续推进,
|
||||
// 否则主站恢复前任务会在节点侧整体停摆。
|
||||
// 该判断在 downloadCheck 之前:正在下载、进度无变化的任务稍后会走
|
||||
// 「无需上报」的提前返回路径,若只在回报分支里判定,通道失联后
|
||||
// 这类任务既不会被发现也不会触发重连,就复现了历史上的静默卡死。
|
||||
if (!queue.isEmpty() && !primaryChannel.usable()) {
|
||||
log.warn("主站通道不可用,{} 个任务状态暂存待上报", queue.size());
|
||||
if (rekickPolicy.shouldRekick(true, false, System.currentTimeMillis()))
|
||||
rekickMainServerAsync();
|
||||
}
|
||||
|
||||
//检查,当任务状态发生变化即方法返回true时,再更新,否则return
|
||||
if (!downloadCheckService.downloadCheck()) {
|
||||
boolean isSkip = true;
|
||||
@@ -95,30 +192,61 @@ public class storageNode {
|
||||
}
|
||||
if(isSkip) {
|
||||
counter++;
|
||||
if (server != null && server.isActive() && counter > 10) {
|
||||
server.writeAndFlush(new MaintainMessage());
|
||||
if (primaryChannel.usable() && counter > 10) {
|
||||
primaryChannel.current().writeAndFlush(new MaintainMessage());
|
||||
counter = 0;
|
||||
}
|
||||
return;
|
||||
}
|
||||
}
|
||||
//主站未连接时不上报,队列保留、等主站重连时重放(无法写通道时上面的分支已兜住)。
|
||||
Channel target = primaryChannel.current();
|
||||
if (target == null)
|
||||
return;
|
||||
//没有待上报任务时不必发送空数组(此时 downloadCheck 为 false 且无压缩完成任务)。
|
||||
if (queue.isEmpty())
|
||||
return;
|
||||
//发送
|
||||
//上锁后再发送,避免出现发送完之后再下载完成
|
||||
lock.lock();
|
||||
DownloadStatusMessage downloadStatusMessage = new DownloadStatusMessage();
|
||||
downloadStatusMessage.setGalleryTasks(queue.values().toArray(GalleryTask[]::new));
|
||||
server.writeAndFlush(downloadStatusMessage);
|
||||
try {
|
||||
DownloadStatusMessage downloadStatusMessage = new DownloadStatusMessage();
|
||||
downloadStatusMessage.setGalleryTasks(queue.values().toArray(GalleryTask[]::new));
|
||||
target.writeAndFlush(downloadStatusMessage);
|
||||
|
||||
queue.entrySet().removeIf(entry -> entry.getValue().is_compress_complete());
|
||||
log.info("任务状态发送完成");
|
||||
|
||||
|
||||
lock.unlock();
|
||||
queue.entrySet().removeIf(entry -> entry.getValue().is_compress_complete());
|
||||
log.info("任务状态发送完成");
|
||||
} finally {
|
||||
lock.unlock();
|
||||
}
|
||||
}catch (Exception e){
|
||||
log.error("发送任务状态时发生异常:{}", e.getMessage());
|
||||
}
|
||||
}
|
||||
|
||||
int counter;
|
||||
|
||||
/**
|
||||
* 该消息是否只有主站会发。
|
||||
*
|
||||
* <p>白名单而非黑名单:只有这些类型能证明对端是主站并据此认领通道引用。
|
||||
* 名单严格等于主站在 {@code RemoteService} 里实际会发出的类型;
|
||||
* {@code ResponseMessage}、{@code DownloadStatusMessage}、{@code MaintainMessage}
|
||||
* 都是本节点自己的出站类型,备机({@code lionwebsiteside})也会发身份消息,
|
||||
* 都不能用来认领主站引用。
|
||||
*/
|
||||
static boolean isPrimaryEvidence(AbstractMessage message) {
|
||||
if (message.messageType == AbstractMessage.IDENTITY_MESSAGE) {
|
||||
// 备机也用身份消息自我介绍,只有 identity 为 lionwebsite 的才是主站。
|
||||
IdentityMessage identity = (IdentityMessage) message;
|
||||
return "lionwebsite".equals(identity.getIdentity());
|
||||
}
|
||||
return message.messageType == AbstractMessage.DOWNLOAD_POST_MESSAGE
|
||||
|| message.messageType == AbstractMessage.DELETE_GALLERY_MESSAGE
|
||||
|| message.messageType == AbstractMessage.AVAILABLE_CHECK_MESSAGE
|
||||
|| message.messageType == AbstractMessage.SUBSCRIPTION_SNAPSHOT_MESSAGE;
|
||||
}
|
||||
|
||||
class MyChannelInboundHandlerAdapter extends ChannelInboundHandlerAdapter{
|
||||
Map<Integer, GalleryTask> queue;
|
||||
|
||||
@@ -128,40 +256,82 @@ public class storageNode {
|
||||
|
||||
@Override
|
||||
public void channelRead(ChannelHandlerContext ctx, Object msg) {
|
||||
log.info(String.valueOf(msg));
|
||||
if (msg instanceof SubscriptionSnapshotMessage snapshot)
|
||||
log.info("收到订阅快照 revision={}", shortRevision(snapshot.getRevision()));
|
||||
else
|
||||
log.info(String.valueOf(msg));
|
||||
AbstractMessage abstractMessage = (AbstractMessage) msg;
|
||||
|
||||
// 只有主站会发的消息类型(身份/任务下发/删除/探活/订阅快照)同时承担两件事:
|
||||
// 证明「主站仍在」,以及恢复通道引用。并发认证时引用可能先指向 X 再被 Y 覆盖,
|
||||
// Y 断开后引用被清空,而仍可用的 X 继续发送探活等消息;收到这些消息即证明
|
||||
// X 就是主站,必须无条件认领——引用被清空时 current() 为 null,
|
||||
// 任何以其为前提的判断都不成立,自愈也就无从触发。
|
||||
if (isPrimaryEvidence(abstractMessage)) {
|
||||
primaryChannel.onPrimaryMessage(ctx.channel());
|
||||
subscriptionSnapshotStore.markPrimaryContact();
|
||||
}
|
||||
|
||||
switch (abstractMessage.messageType){
|
||||
case AbstractMessage.IDENTITY_MESSAGE -> {
|
||||
IdentityMessage identityMessage = (IdentityMessage) abstractMessage;
|
||||
if(identityMessage.getIdentity().equals("lionwebsite")) {
|
||||
server = ctx.channel();
|
||||
// 引用已在上面按消息类型认领,这里只保留「上线」日志语义。
|
||||
log.info("server 上线");
|
||||
} else if(identityMessage.getIdentity().equals("lionwebsiteside")){
|
||||
node = ctx.channel();
|
||||
log.info("node上线");
|
||||
downloadCheckService.setNode(node);
|
||||
}
|
||||
}
|
||||
case AbstractMessage.DOWNLOAD_POST_MESSAGE -> {
|
||||
DownloadPostMessage dpm = (DownloadPostMessage) abstractMessage;
|
||||
lock.lock();
|
||||
//添加到队列方法返回真说明该任务已下载完成,直接发送下载进度
|
||||
if(downloadCheckService.addToQueue(dpm.getGalleryTask())){
|
||||
try {
|
||||
//每次收到任务都重新检查并立即回传当前状态,供主站的单任务重试接口使用
|
||||
GalleryTask currentTask = downloadCheckService.addToQueue(dpm.getGalleryTask());
|
||||
DownloadStatusMessage downloadStatusMessage = new DownloadStatusMessage();
|
||||
downloadStatusMessage.setGalleryTasks(new GalleryTask[]{dpm.getGalleryTask()});
|
||||
server.writeAndFlush(downloadStatusMessage);
|
||||
downloadStatusMessage.setGalleryTasks(new GalleryTask[]{currentTask});
|
||||
Channel target = primaryChannel.current();
|
||||
if (target != null && target.isActive())
|
||||
target.writeAndFlush(downloadStatusMessage);
|
||||
log.info(String.valueOf(queue));
|
||||
} finally {
|
||||
lock.unlock();
|
||||
}
|
||||
log.info(String.valueOf(queue));
|
||||
lock.unlock();
|
||||
ctx.writeAndFlush(new ResponseMessage(dpm.messageId, (byte) 0));
|
||||
}
|
||||
case AbstractMessage.DELETE_GALLERY_MESSAGE -> {
|
||||
DeleteGalleryMessage deleteGalleryMessage = (DeleteGalleryMessage) abstractMessage;
|
||||
byte result = DeleteService.deleteAll(storagePath + deleteGalleryMessage.getGalleryName());
|
||||
String galleryName = deleteGalleryMessage.getGalleryName();
|
||||
lock.lock();
|
||||
try {
|
||||
// 删除任务时也要从待下载队列移除,避免继续向主站上报已经删除的任务状态。
|
||||
storageNode.this.queue.entrySet().removeIf(entry -> galleryName.equals(entry.getValue().getName()));
|
||||
queue.entrySet().removeIf(entry -> galleryName.equals(entry.getValue().getName()));
|
||||
} finally {
|
||||
lock.unlock();
|
||||
}
|
||||
byte result = DeleteService.deleteWithin(storagePath, galleryName);
|
||||
ResponseMessage responseMessage = new ResponseMessage(deleteGalleryMessage.messageId, result);
|
||||
ctx.writeAndFlush(responseMessage);
|
||||
}
|
||||
case AbstractMessage.AVAILABLE_CHECK_MESSAGE -> {
|
||||
AvailableCheckMessage acm = (AvailableCheckMessage) abstractMessage;
|
||||
ResponseMessage responseMessage = new ResponseMessage(acm.messageId, (byte)0);
|
||||
ctx.writeAndFlush(responseMessage);
|
||||
}
|
||||
case AbstractMessage.SUBSCRIPTION_SNAPSHOT_MESSAGE -> {
|
||||
SubscriptionSnapshotMessage snapshotMessage = (SubscriptionSnapshotMessage) abstractMessage;
|
||||
if (!Config.subscriptionSyncEnabled || !ctx.channel().equals(primaryChannel.current())) {
|
||||
ctx.writeAndFlush(new ResponseMessage(snapshotMessage.messageId, SubscriptionSnapshotStore.APPLY_INVALID));
|
||||
return;
|
||||
}
|
||||
subscriptionApplyExecutor.execute(() -> {
|
||||
SubscriptionSnapshotStore.ApplyResult result = subscriptionSnapshotStore.apply(snapshotMessage);
|
||||
ctx.writeAndFlush(new ResponseMessage(snapshotMessage.messageId, result.code()));
|
||||
log.info("订阅快照处理完成 revision={} result={}", shortRevision(snapshotMessage.getRevision()), result.code());
|
||||
});
|
||||
}
|
||||
}
|
||||
//
|
||||
// //修复预览
|
||||
@@ -171,15 +341,17 @@ public class storageNode {
|
||||
|
||||
@Override
|
||||
public void channelUnregistered(ChannelHandlerContext ctx) {
|
||||
if(ctx.channel().equals(server)) {
|
||||
if(ctx.channel().equals(primaryChannel.current())) {
|
||||
log.info("server 下线");
|
||||
server = null;
|
||||
} else if(ctx.channel().equals(node)){
|
||||
log.info("node 下线");
|
||||
node = null;
|
||||
downloadCheckService.setNode(null);
|
||||
}
|
||||
primaryChannel.unregister(ctx.channel());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private static String shortRevision(String revision) {
|
||||
return revision == null ? null : revision.substring(0, Math.min(12, revision.length()));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,2 +1,9 @@
|
||||
DouNaiV2ray=https://aaaa.gay/link/X7zEqkIx5gtIGugO?client=v2
|
||||
DouNaiClash=https://aaaa.gay/link/X7zEqkIx5gtIGugO?client=clashmeta
|
||||
# 订阅快照由主站通过 Netty 长连接推送;不要在此保存上游订阅地址或 Key。
|
||||
SubscriptionSyncEnabled=false
|
||||
SubscriptionSyncSecret=
|
||||
SubscriptionDataDir=/root/gallery/storageNode/sub
|
||||
SubscriptionMaxStaleSeconds=604800
|
||||
SubscriptionMaxPayloadBytes=52428800
|
||||
SubscriptionHttpPort=8889
|
||||
SubscriptionHttpWorkers=4
|
||||
SubscriptionSocketTimeoutMs=10000
|
||||
|
||||
@@ -0,0 +1,21 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<configuration>
|
||||
<appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
|
||||
<file>run.out</file>
|
||||
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
|
||||
<fileNamePattern>run.%i.out</fileNamePattern>
|
||||
<minIndex>1</minIndex>
|
||||
<maxIndex>10</maxIndex>
|
||||
</rollingPolicy>
|
||||
<triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
|
||||
<maxFileSize>10MB</maxFileSize>
|
||||
</triggeringPolicy>
|
||||
<encoder>
|
||||
<pattern>%d{yyyy-MM-dd HH:mm:ss} [%level] %msg%n</pattern>
|
||||
</encoder>
|
||||
</appender>
|
||||
|
||||
<root level="info">
|
||||
<appender-ref ref="FILE"/>
|
||||
</root>
|
||||
</configuration>
|
||||
@@ -53,6 +53,60 @@
|
||||
"allDeclaredFields" : true,
|
||||
"allPublicFields" : true
|
||||
},
|
||||
{
|
||||
"name": "lion.Message.Main.AvailableCheckMessage",
|
||||
"allDeclaredConstructors" : true,
|
||||
"allPublicConstructors" : true,
|
||||
"allDeclaredMethods" : true,
|
||||
"allPublicMethods" : true,
|
||||
"allDeclaredFields" : true,
|
||||
"allPublicFields" : true
|
||||
},
|
||||
{
|
||||
"name": "lion.Message.Main.SubscriptionSnapshotMessage",
|
||||
"allDeclaredConstructors" : true,
|
||||
"allPublicConstructors" : true,
|
||||
"allDeclaredMethods" : true,
|
||||
"allPublicMethods" : true,
|
||||
"allDeclaredFields" : true,
|
||||
"allPublicFields" : true
|
||||
},
|
||||
{
|
||||
"name": "lion.Message.Main.SubscriptionSnapshotPayload",
|
||||
"allDeclaredConstructors" : true,
|
||||
"allPublicConstructors" : true,
|
||||
"allDeclaredMethods" : true,
|
||||
"allPublicMethods" : true,
|
||||
"allDeclaredFields" : true,
|
||||
"allPublicFields" : true
|
||||
},
|
||||
{
|
||||
"name": "lion.Message.Main.SubscriptionAccountSnapshot",
|
||||
"allDeclaredConstructors" : true,
|
||||
"allPublicConstructors" : true,
|
||||
"allDeclaredMethods" : true,
|
||||
"allPublicMethods" : true,
|
||||
"allDeclaredFields" : true,
|
||||
"allPublicFields" : true
|
||||
},
|
||||
{
|
||||
"name": "lion.Message.Main.SubscriptionBindingSnapshot",
|
||||
"allDeclaredConstructors" : true,
|
||||
"allPublicConstructors" : true,
|
||||
"allDeclaredMethods" : true,
|
||||
"allPublicMethods" : true,
|
||||
"allDeclaredFields" : true,
|
||||
"allPublicFields" : true
|
||||
},
|
||||
{
|
||||
"name": "lion.Service.SubscriptionSnapshotStore$Status",
|
||||
"allDeclaredConstructors" : true,
|
||||
"allPublicConstructors" : true,
|
||||
"allDeclaredMethods" : true,
|
||||
"allPublicMethods" : true,
|
||||
"allDeclaredFields" : true,
|
||||
"allPublicFields" : true
|
||||
},
|
||||
{
|
||||
"name": "lion.Message.AbstractMessage",
|
||||
"allDeclaredConstructors" : true,
|
||||
@@ -70,5 +124,15 @@
|
||||
"allPublicMethods" : true,
|
||||
"allDeclaredFields" : true,
|
||||
"allPublicFields" : true
|
||||
},
|
||||
{
|
||||
"name": "lion.storageNode$MyChannelInboundHandlerAdapter",
|
||||
"allDeclaredMethods" : true,
|
||||
"allPublicMethods" : true
|
||||
},
|
||||
{
|
||||
"name": "lion.Message.MessageCodec",
|
||||
"allDeclaredMethods" : true,
|
||||
"allPublicMethods" : true
|
||||
}
|
||||
]
|
||||
@@ -1,9 +0,0 @@
|
||||
|
||||
org.slf4j.simpleLogger.defaultLogLevel=info
|
||||
|
||||
org.slf4j.simpleLogger.showDateTime=true
|
||||
|
||||
org.slf4j.simpleLogger.dateTimeFormat=yyyy-MM-dd HH:mm:ss
|
||||
|
||||
org.slf4j.simpleLogger.logFile=run.out
|
||||
|
||||
@@ -0,0 +1,109 @@
|
||||
package lion;
|
||||
|
||||
import lion.Config.Config;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import org.junit.jupiter.api.io.TempDir;
|
||||
|
||||
import java.nio.file.Files;
|
||||
import java.nio.file.Path;
|
||||
|
||||
import static org.junit.jupiter.api.Assertions.*;
|
||||
|
||||
/**
|
||||
* 配置加载。
|
||||
*
|
||||
* <p>生产配置来自 {@code /root/gallery/storageNode/config.properties},任何一项读错都会改变
|
||||
* 订阅同步与分发端口的行为。这里对默认值填充、显式覆盖、文件缺失回退与「启用同步却
|
||||
* 没配密钥就拒绝启动」逐条断言。
|
||||
*/
|
||||
class ConfigTest {
|
||||
|
||||
@Test
|
||||
void fillsBuiltInDefaultsWhenKeysAreAbsent(@TempDir Path root) throws Exception {
|
||||
Path file = Files.writeString(root.resolve("config.properties"), "");
|
||||
|
||||
Config.loadConfig(file.toString());
|
||||
|
||||
assertFalse(Config.subscriptionSyncEnabled);
|
||||
assertEquals("/root/gallery/storageNode/sub", Config.subscriptionDataDir);
|
||||
assertEquals(604800L, Config.subscriptionMaxStaleSeconds);
|
||||
assertEquals(52428800, Config.subscriptionMaxPayloadBytes);
|
||||
assertEquals(8889, Config.subscriptionHttpPort);
|
||||
assertEquals(4, Config.subscriptionHttpWorkers);
|
||||
assertEquals(10000, Config.subscriptionSocketTimeoutMs);
|
||||
}
|
||||
|
||||
@Test
|
||||
void readsExplicitValues(@TempDir Path root) throws Exception {
|
||||
Path file = Files.writeString(root.resolve("config.properties"), String.join("\n",
|
||||
"SubscriptionSyncEnabled=true",
|
||||
"SubscriptionSyncSecret=file-secret",
|
||||
"SubscriptionDataDir=/tmp/sub",
|
||||
"SubscriptionMaxStaleSeconds=120",
|
||||
"SubscriptionMaxPayloadBytes=4096",
|
||||
"SubscriptionHttpPort=9999",
|
||||
"SubscriptionHttpWorkers=2",
|
||||
"SubscriptionSocketTimeoutMs=500"));
|
||||
|
||||
Config.loadConfig(file.toString());
|
||||
|
||||
assertTrue(Config.subscriptionSyncEnabled);
|
||||
assertEquals("file-secret", Config.subscriptionSyncSecret);
|
||||
assertEquals("/tmp/sub", Config.subscriptionDataDir);
|
||||
assertEquals(120L, Config.subscriptionMaxStaleSeconds);
|
||||
assertEquals(4096, Config.subscriptionMaxPayloadBytes);
|
||||
assertEquals(9999, Config.subscriptionHttpPort);
|
||||
assertEquals(2, Config.subscriptionHttpWorkers);
|
||||
assertEquals(500, Config.subscriptionSocketTimeoutMs);
|
||||
}
|
||||
|
||||
/** 配置文件缺失不是致命错误,但要落到默认值而不是保留上一次的加载结果。 */
|
||||
@Test
|
||||
void missingFileFallsBackToDefaults(@TempDir Path root) {
|
||||
Config.subscriptionHttpPort = 12345;
|
||||
|
||||
Config.loadConfig(root.resolve("absent.properties").toString());
|
||||
|
||||
assertEquals(8889, Config.subscriptionHttpPort, "缺失文件应回到默认值");
|
||||
}
|
||||
|
||||
/**
|
||||
* 管理员口令默认必须为空(即关闭 8888 的管理员直取能力),
|
||||
* 且不再有写死在源码里的默认口令。
|
||||
*/
|
||||
@Test
|
||||
void adminDownloadCodeDefaultsToDisabled(@TempDir Path root) throws Exception {
|
||||
Path file = Files.writeString(root.resolve("config.properties"), "");
|
||||
|
||||
Config.loadConfig(file.toString());
|
||||
|
||||
assertEquals("", Config.adminDownloadCode, "未配置口令时管理员能力必须关闭");
|
||||
assertEquals("/root/gallery/gallery", Config.adminDownloadRoot, "默认根目录应与下载目录一致");
|
||||
}
|
||||
|
||||
/** 口令与根目录都应能从配置文件读入。 */
|
||||
@Test
|
||||
void readsAdminDownloadSettings(@TempDir Path root) throws Exception {
|
||||
Path file = Files.writeString(root.resolve("config.properties"), String.join("\n",
|
||||
"AdminDownloadCode=alone",
|
||||
"AdminDownloadRoot=/root/gallery"));
|
||||
|
||||
Config.loadConfig(file.toString());
|
||||
|
||||
assertEquals("alone", Config.adminDownloadCode);
|
||||
assertEquals("/root/gallery", Config.adminDownloadRoot);
|
||||
}
|
||||
|
||||
/**
|
||||
* 启用订阅同步却没有密钥时必须拒绝启动。
|
||||
* 否则备机会以空密钥运行,签名校验形同虚设。
|
||||
*/
|
||||
@Test
|
||||
void enabledSyncWithoutSecretIsRejected(@TempDir Path root) throws Exception {
|
||||
Path file = Files.writeString(root.resolve("config.properties"), "SubscriptionSyncEnabled=true");
|
||||
|
||||
IllegalStateException failure = assertThrows(IllegalStateException.class,
|
||||
() -> Config.loadConfig(file.toString()));
|
||||
assertTrue(failure.getMessage().contains("SUBSCRIPTION_SYNC_SECRET"));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,61 @@
|
||||
package lion;
|
||||
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
import java.io.BufferedReader;
|
||||
import java.io.StringReader;
|
||||
import java.net.ServerSocket;
|
||||
|
||||
import static org.junit.jupiter.api.Assertions.*;
|
||||
|
||||
/** 下载链路用到的两个工具方法:断点续传请求头解析与空闲端口探测。 */
|
||||
class CustomUtilTest {
|
||||
|
||||
/** Range 头必须原样取出「bytes=...」,供 sendFileRange 计算区间。 */
|
||||
@Test
|
||||
void extractsRangeHeader() throws Exception {
|
||||
BufferedReader reader = new BufferedReader(new StringReader(
|
||||
"Host: lionwebsite.xyz\r\nUser-Agent: Mozilla/5.0\r\nRange: bytes=1024-2047\r\n\r\n"));
|
||||
|
||||
assertEquals("bytes=1024-2047", CustomUtil.getRequestHeader(reader));
|
||||
}
|
||||
|
||||
/** 没有 Range 头(全新下载)时返回 null,调用方据此从头开始传。 */
|
||||
@Test
|
||||
void returnsNullWhenRangeHeaderIsAbsent() throws Exception {
|
||||
BufferedReader reader = new BufferedReader(new StringReader(
|
||||
"Host: lionwebsite.xyz\r\nUser-Agent: Mozilla/5.0\r\n\r\n"));
|
||||
|
||||
assertNull(CustomUtil.getRequestHeader(reader));
|
||||
}
|
||||
|
||||
/** 从指定端口开始探测,且返回的端口确实可以立刻绑定。 */
|
||||
@Test
|
||||
void returnsBindableIdlePort() throws Exception {
|
||||
int start = freePort();
|
||||
int found = CustomUtil._findIdlePort(start);
|
||||
|
||||
assertTrue(found >= start, "应在起始端口或其之后找到空闲端口");
|
||||
try (ServerSocket socket = new ServerSocket(found)) {
|
||||
assertEquals(found, socket.getLocalPort(), "返回的端口必须可以绑定");
|
||||
}
|
||||
}
|
||||
|
||||
/** 起始端口被占用时不得返回它,必须继续往后找。 */
|
||||
@Test
|
||||
void skipsOccupiedPort() throws Exception {
|
||||
try (ServerSocket occupied = new ServerSocket(0)) {
|
||||
int busy = occupied.getLocalPort();
|
||||
|
||||
int found = CustomUtil._findIdlePort(busy);
|
||||
|
||||
assertTrue(found > busy, "被占用的端口必须跳过");
|
||||
}
|
||||
}
|
||||
|
||||
private static int freePort() throws Exception {
|
||||
try (ServerSocket socket = new ServerSocket(0)) {
|
||||
return socket.getLocalPort();
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,264 @@
|
||||
package lion.Externel;
|
||||
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
import lion.CustomUtil;
|
||||
import lion.Message.Main.SubscriptionAccountSnapshot;
|
||||
import lion.Message.Main.SubscriptionBindingSnapshot;
|
||||
import lion.Message.Main.SubscriptionSnapshotMessage;
|
||||
import lion.Message.Main.SubscriptionSnapshotPayload;
|
||||
import lion.Service.SubscriptionSnapshotStore;
|
||||
import org.junit.jupiter.api.BeforeAll;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import org.junit.jupiter.api.io.TempDir;
|
||||
|
||||
import javax.crypto.Mac;
|
||||
import javax.crypto.spec.SecretKeySpec;
|
||||
import java.io.ByteArrayOutputStream;
|
||||
import java.io.IOException;
|
||||
import java.net.ServerSocket;
|
||||
import java.net.Socket;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.nio.file.Path;
|
||||
import java.security.MessageDigest;
|
||||
import java.util.Arrays;
|
||||
import java.util.Base64;
|
||||
import java.util.HashMap;
|
||||
import java.util.HexFormat;
|
||||
import java.util.List;
|
||||
import java.util.Locale;
|
||||
import java.util.Map;
|
||||
import java.util.zip.GZIPOutputStream;
|
||||
|
||||
import static org.junit.jupiter.api.Assertions.*;
|
||||
|
||||
/**
|
||||
* 备机订阅分发 HTTP 服务(端口 8889)。
|
||||
*
|
||||
* <p>这是订阅用户的唯一入口,此前完全没有测试。这里起真实端口,按线路字节校验
|
||||
* 状态码、正文与响应头:按公开 Key 命中、两种订阅格式、Range 断点续传、HEAD、
|
||||
* 未知 Key、无快照时的 503、健康检查不泄露、方法不被允许以及不可满足的 Range。
|
||||
*/
|
||||
class BackupSubServerTest {
|
||||
|
||||
private static final String SECRET = "backup-sub-test-secret";
|
||||
private static final String PUBLIC_KEY = "public-key-abcdef";
|
||||
private static final String V2_CONTENT = "v2ray subscription body";
|
||||
private static final String CLASH_CONTENT = "clash subscription body";
|
||||
|
||||
@TempDir
|
||||
static Path tempDir;
|
||||
|
||||
static int readyPort;
|
||||
static int emptyPort;
|
||||
|
||||
@BeforeAll
|
||||
static void startServers() throws Exception {
|
||||
// 有有效快照的备机:正常分发。
|
||||
SubscriptionSnapshotStore ready = store(tempDir.resolve("ready"));
|
||||
assertEquals(SubscriptionSnapshotStore.APPLY_SUCCESS, ready.apply(snapshot()).code());
|
||||
readyPort = startServer(ready);
|
||||
|
||||
// 从未收到过快照的备机:必须回 503,而不是把请求静默挂起。
|
||||
emptyPort = startServer(store(tempDir.resolve("empty")));
|
||||
}
|
||||
|
||||
/** 命中公开 Key:v2 与 clash 各自返回对应正文。 */
|
||||
@Test
|
||||
void servesBothSubscriptionFormatsByPublicKey() throws Exception {
|
||||
Response v2 = get(readyPort, "/sub/v2/" + PUBLIC_KEY);
|
||||
assertEquals(200, v2.status());
|
||||
assertEquals(V2_CONTENT, v2.bodyAsString());
|
||||
assertTrue(v2.header("content-type").startsWith("text/plain"));
|
||||
|
||||
Response clash = get(readyPort, "/sub/cat/" + PUBLIC_KEY);
|
||||
assertEquals(200, clash.status());
|
||||
assertEquals(CLASH_CONTENT, clash.bodyAsString());
|
||||
assertTrue(clash.header("content-type").startsWith("text/yaml"));
|
||||
}
|
||||
|
||||
/** Range 请求必须按断点续传返回 206,并给出正确的 Content-Range。 */
|
||||
@Test
|
||||
void supportsPartialContentForResumedDownloads() throws Exception {
|
||||
Response response = request(readyPort, "GET", "/sub/v2/" + PUBLIC_KEY, "Range: bytes=0-3");
|
||||
|
||||
assertEquals(206, response.status());
|
||||
assertEquals("v2ra", response.bodyAsString());
|
||||
assertEquals("bytes 0-3/" + V2_CONTENT.length(), response.header("content-range"));
|
||||
assertEquals("4", response.header("content-length"));
|
||||
}
|
||||
|
||||
/** HEAD 必须有正确的 Content-Length,但不回正文。 */
|
||||
@Test
|
||||
void headReturnsHeadersWithoutBody() throws Exception {
|
||||
Response response = request(readyPort, "HEAD", "/sub/v2/" + PUBLIC_KEY);
|
||||
|
||||
assertEquals(200, response.status());
|
||||
assertEquals(String.valueOf(V2_CONTENT.length()), response.header("content-length"));
|
||||
assertEquals(0, response.body().length, "HEAD 不得返回正文");
|
||||
}
|
||||
|
||||
/** 未知 Key 回 404。 */
|
||||
@Test
|
||||
void unknownKeyIsNotFound() throws Exception {
|
||||
assertEquals(404, get(readyPort, "/sub/v2/unknown-public-key").status());
|
||||
}
|
||||
|
||||
/** Key 过短属于非法请求,直接 404 而不是去查表。 */
|
||||
@Test
|
||||
void tooShortKeyIsRejected() throws Exception {
|
||||
assertEquals(404, get(readyPort, "/sub/v2/abc").status());
|
||||
}
|
||||
|
||||
/** 没有有效快照时必须回 503,让客户端重试,而不是假装有内容。 */
|
||||
@Test
|
||||
void returnsServiceUnavailableWhenNoSnapshotExists() throws Exception {
|
||||
assertEquals(503, get(emptyPort, "/sub/v2/" + PUBLIC_KEY).status());
|
||||
}
|
||||
|
||||
/** 健康检查只报快照状态,不得包含 Key 或订阅正文。 */
|
||||
@Test
|
||||
void healthEndpointReportsStateWithoutLeakingKeyOrContent() throws Exception {
|
||||
Response response = get(readyPort, "/health/subscription");
|
||||
|
||||
assertEquals(200, response.status());
|
||||
assertTrue(response.header("content-type").startsWith("application/json"));
|
||||
String body = response.bodyAsString();
|
||||
assertTrue(body.contains("ready"), "有快照且新鲜时应报告 ready");
|
||||
assertFalse(body.contains(PUBLIC_KEY), "健康检查不得泄露公开 Key");
|
||||
assertFalse(body.contains(V2_CONTENT), "健康检查不得泄露订阅正文");
|
||||
}
|
||||
|
||||
/** 非 GET/HEAD 方法回 405。 */
|
||||
@Test
|
||||
void rejectsUnsupportedMethods() throws Exception {
|
||||
assertEquals(405, request(readyPort, "POST", "/sub/v2/" + PUBLIC_KEY).status());
|
||||
}
|
||||
|
||||
/** 起点越界的 Range 回 416,而不是返回空 200。 */
|
||||
@Test
|
||||
void unsatisfiableRangeIsRejected() throws Exception {
|
||||
assertEquals(416, request(readyPort, "GET", "/sub/v2/" + PUBLIC_KEY, "Range: bytes=999-1000").status());
|
||||
}
|
||||
|
||||
// ---- 测试脚手架 ----
|
||||
|
||||
private static SubscriptionSnapshotStore store(Path root) {
|
||||
return new SubscriptionSnapshotStore(root, SECRET, 3600, 1024 * 1024);
|
||||
}
|
||||
|
||||
private static int startServer(SubscriptionSnapshotStore store) throws IOException {
|
||||
int port;
|
||||
try (ServerSocket probe = new ServerSocket(0)) {
|
||||
port = probe.getLocalPort();
|
||||
}
|
||||
Thread thread = new Thread(new BackupSubServer(store, port, 2), "backup-sub-test-" + port);
|
||||
thread.setDaemon(true);
|
||||
thread.start();
|
||||
return port;
|
||||
}
|
||||
|
||||
private static Response get(int port, String path) throws IOException {
|
||||
return request(port, "GET", path);
|
||||
}
|
||||
|
||||
private static Response request(int port, String method, String path, String... headers) throws IOException {
|
||||
try (Socket socket = new Socket("127.0.0.1", port)) {
|
||||
socket.setSoTimeout(5_000);
|
||||
StringBuilder request = new StringBuilder()
|
||||
.append(method).append(' ').append(path).append(" HTTP/1.1\r\n")
|
||||
.append("Host: 127.0.0.1\r\n");
|
||||
for (String header : headers)
|
||||
request.append(header).append("\r\n");
|
||||
request.append("Connection: close\r\n\r\n");
|
||||
socket.getOutputStream().write(request.toString().getBytes(StandardCharsets.US_ASCII));
|
||||
socket.getOutputStream().flush();
|
||||
|
||||
byte[] raw = socket.getInputStream().readAllBytes();
|
||||
int split = indexOfHeaderEnd(raw);
|
||||
assertTrue(split > 0, "响应必须含完整的头部结束标记");
|
||||
String head = new String(raw, 0, split, StandardCharsets.ISO_8859_1);
|
||||
byte[] body = Arrays.copyOfRange(raw, split + 4, raw.length);
|
||||
|
||||
String[] lines = head.split("\r\n");
|
||||
int status = Integer.parseInt(lines[0].split(" ")[1]);
|
||||
Map<String, String> parsed = new HashMap<>();
|
||||
for (int i = 1; i < lines.length; i++) {
|
||||
int colon = lines[i].indexOf(':');
|
||||
if (colon > 0)
|
||||
parsed.put(lines[i].substring(0, colon).trim().toLowerCase(Locale.ROOT),
|
||||
lines[i].substring(colon + 1).trim());
|
||||
}
|
||||
return new Response(status, parsed, body);
|
||||
}
|
||||
}
|
||||
|
||||
private static int indexOfHeaderEnd(byte[] raw) {
|
||||
for (int i = 0; i + 3 < raw.length; i++)
|
||||
if (raw[i] == '\r' && raw[i + 1] == '\n' && raw[i + 2] == '\r' && raw[i + 3] == '\n')
|
||||
return i;
|
||||
return -1;
|
||||
}
|
||||
|
||||
private record Response(int status, Map<String, String> headers, byte[] body) {
|
||||
String header(String name) {
|
||||
return headers.get(name);
|
||||
}
|
||||
|
||||
String bodyAsString() {
|
||||
return new String(body, StandardCharsets.UTF_8);
|
||||
}
|
||||
}
|
||||
|
||||
private static SubscriptionSnapshotMessage snapshot() throws Exception {
|
||||
ObjectMapper mapper = CustomUtil.objectMapper;
|
||||
byte[] v2 = V2_CONTENT.getBytes(StandardCharsets.UTF_8);
|
||||
byte[] clash = CLASH_CONTENT.getBytes(StandardCharsets.UTF_8);
|
||||
|
||||
SubscriptionAccountSnapshot account = new SubscriptionAccountSnapshot();
|
||||
account.setAccountId(1);
|
||||
account.setEnabled(true);
|
||||
account.setV2ContentBase64(Base64.getEncoder().encodeToString(v2));
|
||||
account.setV2Sha256(sha256(v2));
|
||||
account.setClashContentBase64(Base64.getEncoder().encodeToString(clash));
|
||||
account.setClashSha256(sha256(clash));
|
||||
|
||||
SubscriptionBindingSnapshot binding = new SubscriptionBindingSnapshot();
|
||||
binding.setPublicKeySha256(sha256(PUBLIC_KEY.getBytes(StandardCharsets.UTF_8)));
|
||||
binding.setAccountId(1);
|
||||
|
||||
SubscriptionSnapshotPayload payload = new SubscriptionSnapshotPayload();
|
||||
payload.setSchemaVersion(1);
|
||||
payload.setAccounts(List.of(account));
|
||||
payload.setBindings(List.of(binding));
|
||||
byte[] json = mapper.writeValueAsBytes(payload);
|
||||
byte[] compressed = gzip(json);
|
||||
|
||||
SubscriptionSnapshotMessage message = new SubscriptionSnapshotMessage();
|
||||
message.setSchemaVersion(1);
|
||||
message.setRevision(sha256(json));
|
||||
message.setGeneratedAt(System.currentTimeMillis());
|
||||
message.setPayloadBase64(Base64.getEncoder().encodeToString(compressed));
|
||||
message.setPayloadSha256(sha256(compressed));
|
||||
String input = "1\n" + message.getRevision() + "\n" + message.getGeneratedAt() + "\n" + message.getPayloadSha256();
|
||||
Mac mac = Mac.getInstance("HmacSHA256");
|
||||
mac.init(new SecretKeySpec(SECRET.getBytes(StandardCharsets.UTF_8), "HmacSHA256"));
|
||||
message.setSignature(hex(mac.doFinal(input.getBytes(StandardCharsets.UTF_8))));
|
||||
return message;
|
||||
}
|
||||
|
||||
private static byte[] gzip(byte[] bytes) throws Exception {
|
||||
ByteArrayOutputStream output = new ByteArrayOutputStream();
|
||||
try (GZIPOutputStream gzip = new GZIPOutputStream(output)) {
|
||||
gzip.write(bytes);
|
||||
}
|
||||
return output.toByteArray();
|
||||
}
|
||||
|
||||
private static String sha256(byte[] bytes) throws Exception {
|
||||
return hex(MessageDigest.getInstance("SHA-256").digest(bytes));
|
||||
}
|
||||
|
||||
private static String hex(byte[] bytes) {
|
||||
return HexFormat.of().formatHex(bytes);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,232 @@
|
||||
package lion;
|
||||
|
||||
import io.netty.buffer.ByteBuf;
|
||||
import io.netty.buffer.Unpooled;
|
||||
import io.netty.channel.embedded.EmbeddedChannel;
|
||||
import io.netty.handler.codec.LengthFieldBasedFrameDecoder;
|
||||
import lion.Domain.GalleryTask;
|
||||
import lion.Message.AbstractMessage;
|
||||
import lion.Message.MessageCodec;
|
||||
import lion.Message.Main.AvailableCheckMessage;
|
||||
import lion.Message.Main.DeleteGalleryMessage;
|
||||
import lion.Message.Main.DownloadPostMessage;
|
||||
import lion.Message.Main.DownloadStatusMessage;
|
||||
import lion.Message.Main.IdentityMessage;
|
||||
import lion.Message.Main.MaintainMessage;
|
||||
import lion.Message.Main.ResponseMessage;
|
||||
import lion.Message.Main.SubscriptionSnapshotMessage;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
import java.nio.charset.StandardCharsets;
|
||||
|
||||
import static org.junit.jupiter.api.Assertions.*;
|
||||
|
||||
/**
|
||||
* 自定义 TCP 协议 {@code [1 字节类型][4 字节长度][JSON]} 的编解码。
|
||||
*
|
||||
* <p>这是节点与主站之间唯一的线路格式:类型字节决定反序列化成哪个消息类,
|
||||
* 长度前缀决定帧边界。两边任何一处不一致都是掉线或错消息,而此前完全无覆盖。
|
||||
*/
|
||||
class MessageCodecTest {
|
||||
|
||||
/** 每种消息类型都要能按自身类型字节往返,且解出的类型正确。 */
|
||||
@Test
|
||||
void roundTripsEveryMessageType() {
|
||||
assertRoundTrip(new ResponseMessage(7, (byte) 3), AbstractMessage.RESPONSE_MESSAGE);
|
||||
assertRoundTrip(downloadPost(), AbstractMessage.DOWNLOAD_POST_MESSAGE);
|
||||
assertRoundTrip(downloadStatus(), AbstractMessage.DOWNLOAD_STATUS_MESSAGE);
|
||||
assertRoundTrip(deleteGallery(), AbstractMessage.DELETE_GALLERY_MESSAGE);
|
||||
assertRoundTrip(identity(), AbstractMessage.IDENTITY_MESSAGE);
|
||||
assertRoundTrip(new MaintainMessage(), AbstractMessage.MAINTAIN_MESSAGE);
|
||||
assertRoundTrip(new AvailableCheckMessage(), AbstractMessage.AVAILABLE_CHECK_MESSAGE);
|
||||
assertRoundTrip(snapshot(), AbstractMessage.SUBSCRIPTION_SNAPSHOT_MESSAGE);
|
||||
}
|
||||
|
||||
/** 帧头就是「类型字节 + 大端 4 字节长度」,长度必须等于 JSON 负载的字节数。 */
|
||||
@Test
|
||||
void writesTypeByteAndBigEndianLengthPrefix() {
|
||||
ByteBuf frame = encode(new ResponseMessage(7, (byte) 3));
|
||||
try {
|
||||
assertEquals(AbstractMessage.RESPONSE_MESSAGE, frame.readByte(), "首字节必须是消息类型");
|
||||
int length = frame.readInt();
|
||||
int jsonLength = frame.readableBytes();
|
||||
assertEquals(jsonLength, length, "长度前缀必须等于 JSON 负载字节数");
|
||||
assertTrue(jsonLength > 0);
|
||||
} finally {
|
||||
frame.release();
|
||||
}
|
||||
}
|
||||
|
||||
/** 未知类型字节必须被安静丢弃(不产出消息),而不是抛出或产出错误类型的消息。 */
|
||||
@Test
|
||||
void unknownTypeByteIsDroppedWithoutThrowing() {
|
||||
EmbeddedChannel channel = new EmbeddedChannel(new MessageCodec());
|
||||
try {
|
||||
byte[] payload = "{}".getBytes(StandardCharsets.UTF_8);
|
||||
ByteBuf frame = Unpooled.buffer();
|
||||
frame.writeByte(99);
|
||||
frame.writeInt(payload.length);
|
||||
frame.writeBytes(payload);
|
||||
|
||||
channel.writeInbound(frame);
|
||||
|
||||
assertNull(channel.readInbound(), "未知类型不应产出任何消息");
|
||||
} finally {
|
||||
channel.finishAndReleaseAll();
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 生产管线是「先分帧、再解码」:{@link LengthFieldBasedFrameDecoder} 负责等齐整帧,
|
||||
* 紧跟其后的 {@link MessageCodec} 才假设长度前缀完整。半个帧到达时必须按兵不动、
|
||||
* 等剩余字节,而不能抛异常或产出半条消息。
|
||||
*/
|
||||
@Test
|
||||
void incompleteFrameIsHeldByFrameDecoderUntilComplete() {
|
||||
EmbeddedChannel channel = new EmbeddedChannel(
|
||||
new LengthFieldBasedFrameDecoder(100_000_000, 1, 4), new MessageCodec());
|
||||
try {
|
||||
AvailableCheckMessage probe = new AvailableCheckMessage();
|
||||
probe.messageId = 11;
|
||||
|
||||
ByteBuf frame = encode(probe);
|
||||
byte[] bytes = new byte[frame.readableBytes()];
|
||||
frame.getBytes(0, bytes);
|
||||
frame.release();
|
||||
|
||||
int half = bytes.length / 2;
|
||||
ByteBuf firstHalf = Unpooled.copiedBuffer(bytes, 0, half);
|
||||
ByteBuf secondHalf = Unpooled.copiedBuffer(bytes, half, bytes.length - half);
|
||||
|
||||
assertFalse(channel.writeInbound(firstHalf), "半个帧不应产出消息");
|
||||
assertNull(channel.readInbound());
|
||||
|
||||
channel.writeInbound(secondHalf);
|
||||
AbstractMessage decoded = channel.readInbound();
|
||||
assertInstanceOf(AvailableCheckMessage.class, decoded);
|
||||
assertEquals(11, decoded.messageId);
|
||||
} finally {
|
||||
channel.finishAndReleaseAll();
|
||||
}
|
||||
}
|
||||
|
||||
/** 长度前缀来自线路,必须设上限;否则一个报文就能让对端按声明的长度申请内存。 */
|
||||
@Test
|
||||
void rejectsAbsurdDeclaredLength() {
|
||||
EmbeddedChannel channel = new EmbeddedChannel(new MessageCodec());
|
||||
try {
|
||||
ByteBuf frame = Unpooled.buffer();
|
||||
frame.writeByte(AbstractMessage.RESPONSE_MESSAGE);
|
||||
frame.writeInt(Integer.MAX_VALUE);
|
||||
|
||||
assertThrows(Exception.class, () -> channel.writeInbound(frame),
|
||||
"超出上限的帧长度必须被拒绝,而不是按声明分配字节数组");
|
||||
} finally {
|
||||
channel.finishAndReleaseAll();
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 序列化失败时绝不能写出半帧。
|
||||
*
|
||||
* <p>原实现先写类型字节、再序列化,失败时只记日志,于是留下「有类型字节、没有长度和
|
||||
* JSON」的残帧,对端的分帧器会永远等长度前缀。现在必须先序列化成功再写帧头。
|
||||
*/
|
||||
@Test
|
||||
void serializationFailureDoesNotEmitHalfFrame() {
|
||||
EmbeddedChannel channel = new EmbeddedChannel(new MessageCodec());
|
||||
try {
|
||||
assertThrows(Exception.class, () -> channel.writeOutbound(new UnserializableMessage()),
|
||||
"序列化失败必须抛错,而不是安静地留下半帧");
|
||||
assertNull(channel.readOutbound(), "失败时不得产出任何字节");
|
||||
} finally {
|
||||
channel.finishAndReleaseAll();
|
||||
}
|
||||
}
|
||||
|
||||
/** 故意让 Jackson 在序列化属性时抛异常。 */
|
||||
static class UnserializableMessage extends AbstractMessage {
|
||||
UnserializableMessage() {
|
||||
messageType = AbstractMessage.RESPONSE_MESSAGE;
|
||||
}
|
||||
|
||||
@SuppressWarnings("unused")
|
||||
public String getExplode() {
|
||||
throw new IllegalStateException("boom");
|
||||
}
|
||||
}
|
||||
|
||||
private static void assertRoundTrip(AbstractMessage message, byte expectedType) {
|
||||
ByteBuf frame = encode(message);
|
||||
assertEquals(expectedType, frame.getByte(0), "线路上的类型字节必须与消息类匹配");
|
||||
|
||||
EmbeddedChannel channel = new EmbeddedChannel(
|
||||
new LengthFieldBasedFrameDecoder(100_000_000, 1, 4), new MessageCodec());
|
||||
try {
|
||||
channel.writeInbound(frame);
|
||||
AbstractMessage decoded = channel.readInbound();
|
||||
assertNotNull(decoded, "整帧必须被解码出来");
|
||||
assertEquals(message.getClass(), decoded.getClass(), "解码类型必须与编码类型一致");
|
||||
assertEquals(expectedType, decoded.messageType);
|
||||
} finally {
|
||||
channel.finishAndReleaseAll();
|
||||
}
|
||||
}
|
||||
|
||||
private static ByteBuf encode(AbstractMessage message) {
|
||||
EmbeddedChannel channel = new EmbeddedChannel(new MessageCodec());
|
||||
try {
|
||||
channel.writeOutbound(message);
|
||||
ByteBuf out = channel.readOutbound();
|
||||
assertNotNull(out, "编码后必须产出字节");
|
||||
return out;
|
||||
} finally {
|
||||
channel.finishAndReleaseAll();
|
||||
}
|
||||
}
|
||||
|
||||
private static DownloadPostMessage downloadPost() {
|
||||
GalleryTask task = new GalleryTask();
|
||||
task.setGid(4242);
|
||||
task.setName("sample [4242]");
|
||||
task.setStatus(GalleryTask.DOWNLOADING);
|
||||
DownloadPostMessage message = new DownloadPostMessage();
|
||||
message.messageId = 1;
|
||||
message.setGalleryTask(task);
|
||||
return message;
|
||||
}
|
||||
|
||||
private static DownloadStatusMessage downloadStatus() {
|
||||
GalleryTask task = new GalleryTask();
|
||||
task.setGid(4242);
|
||||
task.setName("sample [4242]");
|
||||
task.setStatus(GalleryTask.COMPRESS_COMPLETE);
|
||||
DownloadStatusMessage message = new DownloadStatusMessage();
|
||||
message.setGalleryTasks(new GalleryTask[]{task});
|
||||
return message;
|
||||
}
|
||||
|
||||
private static DeleteGalleryMessage deleteGallery() {
|
||||
DeleteGalleryMessage message = new DeleteGalleryMessage();
|
||||
message.messageId = 5;
|
||||
message.setGalleryName("sample [4242]");
|
||||
return message;
|
||||
}
|
||||
|
||||
private static IdentityMessage identity() {
|
||||
IdentityMessage message = new IdentityMessage();
|
||||
message.setIdentity("lionwebsite");
|
||||
return message;
|
||||
}
|
||||
|
||||
private static SubscriptionSnapshotMessage snapshot() {
|
||||
SubscriptionSnapshotMessage message = new SubscriptionSnapshotMessage();
|
||||
message.setSchemaVersion(1);
|
||||
message.setRevision("a".repeat(64));
|
||||
message.setGeneratedAt(1_700_000_000_000L);
|
||||
message.setPayloadBase64("ZmFrZQ==");
|
||||
message.setPayloadSha256("b".repeat(64));
|
||||
message.setSignature("c".repeat(64));
|
||||
return message;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,255 @@
|
||||
package lion;
|
||||
|
||||
import org.junit.jupiter.api.Test;
|
||||
import org.junit.jupiter.api.io.TempDir;
|
||||
|
||||
import java.io.IOException;
|
||||
import java.net.ServerSocket;
|
||||
import java.net.Socket;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.nio.file.Files;
|
||||
import java.nio.file.Path;
|
||||
import java.util.Map;
|
||||
import java.util.concurrent.atomic.AtomicReference;
|
||||
|
||||
import static org.junit.jupiter.api.Assertions.*;
|
||||
|
||||
/**
|
||||
* 下载 HTTP 服务的请求行解析。
|
||||
* 这条链路只接受主站 IP 的请求,解析细节直接决定能否按 gid 找到正确的压缩包。
|
||||
*/
|
||||
class MultiThreadedHTTPServerTest {
|
||||
|
||||
/** 普通请求:path 与各查询参数都要被拆出来。 */
|
||||
@Test
|
||||
void parsesPathAndQueryParameters() {
|
||||
Map<String, String> params = MultiThreadedHTTPServer.parseRequestLine(
|
||||
"/download?AuthCode=abc&gid=12345");
|
||||
|
||||
assertEquals("/download", params.get("path"));
|
||||
assertEquals("abc", params.get("AuthCode"));
|
||||
assertEquals("12345", params.get("gid"));
|
||||
}
|
||||
|
||||
/** 无查询串时只回 path,且不应抛异常。 */
|
||||
@Test
|
||||
void parsesPathWithoutQueryString() {
|
||||
Map<String, String> params = MultiThreadedHTTPServer.parseRequestLine("/archive/file.zip");
|
||||
assertEquals("/archive/file.zip", params.get("path"));
|
||||
assertNull(params.get("gid"));
|
||||
}
|
||||
|
||||
/**
|
||||
* 参数值里含 '=' 时必须完整保留。
|
||||
* 回归:split("=") 无 limit 会在第一个 '=' 处截断,
|
||||
* base64 padding(结尾的 '=')等取值会被悄悄改短。
|
||||
*/
|
||||
@Test
|
||||
void keepsEqualsSignInsideParameterValue() {
|
||||
Map<String, String> params = MultiThreadedHTTPServer.parseRequestLine(
|
||||
"/download?AuthCode=alone&token=YWJjZA==");
|
||||
|
||||
assertEquals("YWJjZA==", params.get("token"), "值内的 '=' 不得被截断");
|
||||
assertEquals("alone", params.get("AuthCode"));
|
||||
}
|
||||
|
||||
/** URL 编码的值必须解码回原文。 */
|
||||
@Test
|
||||
void decodesPercentEncodedValues() {
|
||||
Map<String, String> params = MultiThreadedHTTPServer.parseRequestLine(
|
||||
"/download?AuthCode=alone&name=a%20b");
|
||||
assertEquals("a b", params.get("name"));
|
||||
}
|
||||
|
||||
/** null 请求行返回 null,交由调用方兜底。 */
|
||||
@Test
|
||||
void returnsNullForNullRequestLine() {
|
||||
assertNull(MultiThreadedHTTPServer.parseRequestLine(null));
|
||||
}
|
||||
|
||||
// ---- 请求处理(真实 socket) ----
|
||||
|
||||
/**
|
||||
* 畸形请求行(没有空格)不得打死处理线程,更不得泄漏连接。
|
||||
*
|
||||
* <p>原实现直接取 {@code requestParts[1]},空行会抛 ArrayIndexOutOfBoundsException;
|
||||
* 它不是 IOException,会被线程池静默吞掉,socket 也关不掉。现在应回 400 并关闭连接。
|
||||
*/
|
||||
@Test
|
||||
void malformedRequestLineIsRejectedWithoutCrashing() throws Exception {
|
||||
try (ServerSocket server = new ServerSocket(0); Socket client = new Socket("127.0.0.1", server.getLocalPort())) {
|
||||
client.setSoTimeout(5_000);
|
||||
Thread handler = startHandler(server);
|
||||
|
||||
client.getOutputStream().write("GARBAGE\r\n\r\n".getBytes(StandardCharsets.US_ASCII));
|
||||
client.getOutputStream().flush();
|
||||
|
||||
String head = readHead(client);
|
||||
assertTrue(head.startsWith("HTTP/1.1 400"), "畸形请求行应回 400,实际:" + head.lines().findFirst().orElse(""));
|
||||
handler.join(5_000);
|
||||
assertFalse(handler.isAlive(), "处理器必须结束并关闭连接,不得挂住线程");
|
||||
}
|
||||
}
|
||||
|
||||
/** 只有空行的连接应被安静关闭,不产生响应也不抛异常。 */
|
||||
@Test
|
||||
void blankRequestLineClosesConnectionQuietly() throws Exception {
|
||||
try (ServerSocket server = new ServerSocket(0); Socket client = new Socket("127.0.0.1", server.getLocalPort())) {
|
||||
client.setSoTimeout(5_000);
|
||||
AtomicReference<Throwable> failure = new AtomicReference<>();
|
||||
Thread handler = startHandler(server, failure);
|
||||
|
||||
client.getOutputStream().write("\r\n".getBytes(StandardCharsets.US_ASCII));
|
||||
client.getOutputStream().flush();
|
||||
|
||||
assertEquals(-1, client.getInputStream().read(), "应直接关闭连接而不回正文");
|
||||
handler.join(5_000);
|
||||
assertFalse(handler.isAlive(), "处理器必须结束");
|
||||
assertNull(failure.get(), "空行不得抛异常: " + failure.get());
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 请求的 Range 超过文件末端时必须夹到文件末尾。
|
||||
*
|
||||
* <p>原实现按未夹取的区间写 Content-Length,客户端拿到一个远大于实际发送量的长度,
|
||||
* 会一直等下去。现在 Content-Range / Content-Length / 正文都必须只覆盖到文件末尾。
|
||||
*/
|
||||
@Test
|
||||
void oversizedRangeIsClampedToFileEnd(@TempDir Path root) throws Exception {
|
||||
byte[] payload = "0123456789".getBytes(StandardCharsets.UTF_8);
|
||||
Path archive = root.resolve("archive.zip");
|
||||
Files.write(archive, payload);
|
||||
|
||||
String head = fetchRange(root, archive, "bytes=4-999999");
|
||||
|
||||
assertTrue(head.contains("Content-Range: bytes 4-9/10"), "Content-Range 必须夹到文件末尾:" + head);
|
||||
assertTrue(head.contains("Content-Length: 6"), "Content-Length 必须与实际发送量一致:" + head);
|
||||
}
|
||||
|
||||
/** Range 起点超过文件长度时无可发送内容,应回 416,而不是假装有长度。 */
|
||||
@Test
|
||||
void rangeStartingBeyondFileIsNotSatisfiable(@TempDir Path root) throws Exception {
|
||||
Path archive = root.resolve("archive.zip");
|
||||
Files.write(archive, "0123456789".getBytes(StandardCharsets.UTF_8));
|
||||
|
||||
String head = fetchRange(root, archive, "bytes=999-1000");
|
||||
|
||||
assertTrue(head.startsWith("HTTP/1.1 416"), "越界起点应回 416:" + head.lines().findFirst().orElse(""));
|
||||
}
|
||||
|
||||
// ---- 管理员直取文件的口令与目录约束 ----
|
||||
|
||||
/** 未配置口令时,管理员能力必须关闭,任何 AuthCode 都不能直取文件。 */
|
||||
@Test
|
||||
void adminCapabilityIsOffUntilCodeConfigured() {
|
||||
lion.Config.Config.adminDownloadCode = "";
|
||||
|
||||
assertFalse(MultiThreadedHTTPServer.isAdminRequest("alone"));
|
||||
assertFalse(MultiThreadedHTTPServer.isAdminRequest(""));
|
||||
}
|
||||
|
||||
/** 配置口令后只认该口令,旧写死值与其他值都不再放行。 */
|
||||
@Test
|
||||
void onlyConfiguredAdminCodeIsAccepted() {
|
||||
lion.Config.Config.adminDownloadCode = "s3cret-code";
|
||||
try {
|
||||
assertTrue(MultiThreadedHTTPServer.isAdminRequest("s3cret-code"));
|
||||
assertFalse(MultiThreadedHTTPServer.isAdminRequest("alone"), "写死的旧值不应再有效");
|
||||
assertFalse(MultiThreadedHTTPServer.isAdminRequest("s3cret"));
|
||||
assertFalse(MultiThreadedHTTPServer.isAdminRequest(null));
|
||||
} finally {
|
||||
lion.Config.Config.adminDownloadCode = "";
|
||||
}
|
||||
}
|
||||
|
||||
/** 管理员直取文件只允许根目录之下的真实文件。 */
|
||||
@Test
|
||||
void adminDownloadIsConfinedToConfiguredRoot(@TempDir Path root) throws Exception {
|
||||
Path galleryRoot = Files.createDirectories(root.resolve("gallery"));
|
||||
Path archive = Files.write(galleryRoot.resolve("sample.zip"), "zip".getBytes(StandardCharsets.UTF_8));
|
||||
Path outside = Files.write(root.resolve("secret.txt"), "top secret".getBytes(StandardCharsets.UTF_8));
|
||||
lion.Config.Config.adminDownloadRoot = galleryRoot.toString();
|
||||
try {
|
||||
assertEquals(archive.toRealPath().toFile(), MultiThreadedHTTPServer.resolveAdminFile(archive.toString()));
|
||||
assertEquals(archive.toRealPath().toFile(), MultiThreadedHTTPServer.resolveAdminFile("sample.zip"));
|
||||
|
||||
assertNull(MultiThreadedHTTPServer.resolveAdminFile("../secret.txt"), "../ 越界必须被拒");
|
||||
assertNull(MultiThreadedHTTPServer.resolveAdminFile(outside.toString()), "根目录之外的绝对路径必须被拒");
|
||||
assertNull(MultiThreadedHTTPServer.resolveAdminFile("/etc/passwd"), "任意系统文件必须被拒");
|
||||
assertNull(MultiThreadedHTTPServer.resolveAdminFile(galleryRoot.toString()), "目录本身不是可下载文件");
|
||||
} finally {
|
||||
lion.Config.Config.adminDownloadRoot = "/root/gallery/gallery";
|
||||
}
|
||||
}
|
||||
|
||||
/** 指向根目录之外的符号链接也不得把文件泄漏出去。 */
|
||||
@Test
|
||||
void adminDownloadRejectsSymlinkEscape(@TempDir Path root) throws Exception {
|
||||
Path galleryRoot = Files.createDirectories(root.resolve("gallery"));
|
||||
Path outside = Files.write(root.resolve("secret.txt"), "top secret".getBytes(StandardCharsets.UTF_8));
|
||||
Path link = galleryRoot.resolve("link.zip");
|
||||
try {
|
||||
Files.createSymbolicLink(link, outside);
|
||||
} catch (UnsupportedOperationException | IOException e) {
|
||||
return; // 平台不支持符号链接时跳过
|
||||
}
|
||||
lion.Config.Config.adminDownloadRoot = galleryRoot.toString();
|
||||
try {
|
||||
assertNull(MultiThreadedHTTPServer.resolveAdminFile("link.zip"), "指向根目录外的符号链接必须被拒");
|
||||
} finally {
|
||||
lion.Config.Config.adminDownloadRoot = "/root/gallery/gallery";
|
||||
}
|
||||
}
|
||||
|
||||
// ---- 测试脚手架 ----
|
||||
|
||||
/** 在守护线程里接受一个连接并交给真实的请求处理器。 */
|
||||
private static Thread startHandler(ServerSocket server) {
|
||||
return startHandler(server, new AtomicReference<>());
|
||||
}
|
||||
|
||||
private static Thread startHandler(ServerSocket server, AtomicReference<Throwable> failure) {
|
||||
Thread handler = new Thread(() -> {
|
||||
try {
|
||||
MultiThreadedHTTPServer.handleClientRequest(server.accept());
|
||||
} catch (Throwable t) {
|
||||
failure.set(t);
|
||||
}
|
||||
});
|
||||
handler.setDaemon(true);
|
||||
handler.start();
|
||||
return handler;
|
||||
}
|
||||
|
||||
/** 走 AuthCode=alone 路径请求指定文件,返回响应头。 */
|
||||
private static String fetchRange(Path root, Path archive, String range) throws IOException {
|
||||
lion.Config.Config.adminDownloadCode = "alone";
|
||||
// 管理员直取已限定在 adminDownloadRoot 之下,测试把根目录指到临时目录。
|
||||
lion.Config.Config.adminDownloadRoot = root.toString();
|
||||
try (ServerSocket server = new ServerSocket(0); Socket client = new Socket("127.0.0.1", server.getLocalPort())) {
|
||||
client.setSoTimeout(5_000);
|
||||
Thread handler = startHandler(server);
|
||||
|
||||
String request = "GET " + archive + "?AuthCode=alone HTTP/1.1\r\n"
|
||||
+ "Host: 127.0.0.1\r\nRange: " + range + "\r\n\r\n";
|
||||
client.getOutputStream().write(request.getBytes(StandardCharsets.US_ASCII));
|
||||
client.getOutputStream().flush();
|
||||
|
||||
return readHead(client);
|
||||
}
|
||||
}
|
||||
|
||||
private static String readHead(Socket socket) throws IOException {
|
||||
java.io.ByteArrayOutputStream buffer = new java.io.ByteArrayOutputStream();
|
||||
int b;
|
||||
while ((b = socket.getInputStream().read()) != -1) {
|
||||
buffer.write(b);
|
||||
byte[] bytes = buffer.toByteArray();
|
||||
if (bytes.length >= 4 && bytes[bytes.length - 4] == '\r' && bytes[bytes.length - 3] == '\n'
|
||||
&& bytes[bytes.length - 2] == '\r' && bytes[bytes.length - 1] == '\n')
|
||||
break;
|
||||
}
|
||||
return buffer.toString(StandardCharsets.ISO_8859_1);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,88 @@
|
||||
package lion;
|
||||
|
||||
import io.netty.channel.embedded.EmbeddedChannel;
|
||||
import lion.Message.AbstractMessage;
|
||||
import lion.Message.Main.AvailableCheckMessage;
|
||||
import lion.Message.Main.IdentityMessage;
|
||||
import lion.Service.PrimaryChannelTracker;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
import static org.junit.jupiter.api.Assertions.*;
|
||||
|
||||
/**
|
||||
* 复现并锁定线上事故的通道恢复语义。
|
||||
*
|
||||
* <p>事故经过:主站重连期间节点先后认证了多条通道,引用被最后一条覆盖;那条通道断开后
|
||||
* 引用被清空,而更早建立、仍然可用的通道继续发送可用性探测。节点因引用为空不再上报
|
||||
* 任何任务状态,主站又能在旧通道上收到探活响应,双方都判定连接正常,未完成任务
|
||||
* 便永久停在「已提交」,直到人工触发重连。
|
||||
*/
|
||||
class PrimaryChannelRecoveryTest {
|
||||
|
||||
@Test
|
||||
void availableCheckOnSurvivingChannelRestoresReporting() {
|
||||
PrimaryChannelTracker tracker = new PrimaryChannelTracker();
|
||||
EmbeddedChannel survivor = new EmbeddedChannel();
|
||||
EmbeddedChannel overwriter = new EmbeddedChannel();
|
||||
try {
|
||||
// 主站两次上线:引用最终落在后建立的 overwriter 上。
|
||||
tracker.onPrimaryMessage(survivor);
|
||||
tracker.onPrimaryMessage(overwriter);
|
||||
|
||||
// overwriter 断开,引用被清空——事故的起点。
|
||||
tracker.unregister(overwriter);
|
||||
|
||||
// 仍然可用的 survivor 发来探活消息:引用应当立即恢复。
|
||||
AvailableCheckMessage probe = new AvailableCheckMessage();
|
||||
assertEquals(AbstractMessage.AVAILABLE_CHECK_MESSAGE, probe.messageType);
|
||||
tracker.onPrimaryMessage(survivor);
|
||||
|
||||
assertSame(survivor, tracker.current(), "探活消息应恢复引用");
|
||||
assertTrue(tracker.usable(), "恢复后必须能继续上报任务状态");
|
||||
} finally {
|
||||
survivor.finishAndReleaseAll();
|
||||
overwriter.finishAndReleaseAll();
|
||||
}
|
||||
}
|
||||
|
||||
/** 首选通道断开但仍有其它已认证通道时,不得出现「无通道可用」的空窗。 */
|
||||
@Test
|
||||
void referenceNeverDropsWhileAnotherAuthenticatedChannelLives() {
|
||||
PrimaryChannelTracker tracker = new PrimaryChannelTracker();
|
||||
EmbeddedChannel first = new EmbeddedChannel();
|
||||
EmbeddedChannel second = new EmbeddedChannel();
|
||||
try {
|
||||
tracker.onPrimaryMessage(first);
|
||||
tracker.onPrimaryMessage(second);
|
||||
|
||||
tracker.unregister(second);
|
||||
|
||||
assertTrue(tracker.usable(), "仍有已认证通道时必须可用");
|
||||
assertSame(first, tracker.current());
|
||||
} finally {
|
||||
first.finishAndReleaseAll();
|
||||
second.finishAndReleaseAll();
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 备机的身份消息不得夺走主站引用。
|
||||
*
|
||||
* <p>备机在启动时同样会自我介绍,若按消息类型无条件认领,备机通道会被误当成主站,
|
||||
* 任务状态将上报到错误的连接上。
|
||||
*/
|
||||
@Test
|
||||
void sidecarIdentityDoesNotStealPrimaryReference() {
|
||||
assertTrue(storageNode.isPrimaryEvidence(identity("lionwebsite")), "主站身份应被认作证据");
|
||||
assertFalse(storageNode.isPrimaryEvidence(identity("lionwebsiteside")),
|
||||
"备机身份不是主站证据,否则任务状态会上报到错误的连接");
|
||||
assertFalse(storageNode.isPrimaryEvidence(new lion.Message.Main.ResponseMessage()),
|
||||
"响应消息是节点自己的出站类型,不能用来认领主站引用");
|
||||
}
|
||||
|
||||
private static IdentityMessage identity(String value) {
|
||||
IdentityMessage message = new IdentityMessage();
|
||||
message.setIdentity(value);
|
||||
return message;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,82 @@
|
||||
package lion.Service;
|
||||
|
||||
import lion.ErrorCode.ErrorCode;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import org.junit.jupiter.api.io.TempDir;
|
||||
|
||||
import java.nio.file.Files;
|
||||
import java.nio.file.Path;
|
||||
|
||||
import static org.junit.jupiter.api.Assertions.*;
|
||||
|
||||
/** 画廊目录删除:成功归零,失败按 ErrorCode 区分「路径不存在」与其它 IO 错误。 */
|
||||
class DeleteServiceTest {
|
||||
|
||||
@Test
|
||||
void deletesExistingDirectoryAndReturnsSuccess(@TempDir Path root) throws Exception {
|
||||
Path gallery = Files.createDirectories(root.resolve("sample [123]"));
|
||||
Files.writeString(gallery.resolve("1.jpg"), "image bytes");
|
||||
|
||||
assertEquals(0, DeleteService.deleteAll(gallery.toString()));
|
||||
assertFalse(Files.exists(gallery), "目录应被整棵删除");
|
||||
}
|
||||
|
||||
/** 路径不存在时必须回 FILE_NOT_FOUND,让主站能区分「本来就没有」。 */
|
||||
@Test
|
||||
void missingPathReportsFileNotFound(@TempDir Path root) {
|
||||
assertEquals(ErrorCode.FILE_NOT_FOUND, DeleteService.deleteAll(root.resolve("absent").toString()));
|
||||
}
|
||||
|
||||
/** 目标是普通文件(而非目录)时同样按 FILE_NOT_FOUND 处理,不得误删文件。 */
|
||||
@Test
|
||||
void plainFileIsNotTreatedAsDirectory(@TempDir Path root) throws Exception {
|
||||
Path file = root.resolve("not-a-directory.txt");
|
||||
Files.writeString(file, "keep me");
|
||||
|
||||
assertEquals(ErrorCode.FILE_NOT_FOUND, DeleteService.deleteAll(file.toString()));
|
||||
assertTrue(Files.exists(file), "普通文件不应被删除");
|
||||
}
|
||||
|
||||
/** 正常的子目录名应在根目录内被删除。 */
|
||||
@Test
|
||||
void deleteWithinRemovesChildOfRoot(@TempDir Path root) throws Exception {
|
||||
Path gallery = Files.createDirectories(root.resolve("sample [123]"));
|
||||
|
||||
assertEquals(0, DeleteService.deleteWithin(root.toString(), "sample [123]"));
|
||||
assertFalse(Files.exists(gallery));
|
||||
}
|
||||
|
||||
/**
|
||||
* 画廊名来自主站下发的消息,带 {@code ../} 时必须被拒绝。
|
||||
* 否则路径会被规范化到根目录之外,删掉节点上的任意目录。
|
||||
*/
|
||||
@Test
|
||||
void deleteWithinRejectsPathTraversal(@TempDir Path root) throws Exception {
|
||||
Path outside = Files.createDirectories(root.resolve("important"));
|
||||
Path galleryRoot = Files.createDirectories(root.resolve("gallery"));
|
||||
|
||||
byte result = DeleteService.deleteWithin(galleryRoot.toString(), "../important");
|
||||
|
||||
assertEquals(ErrorCode.FILE_NOT_FOUND, result, "越界路径必须按找不到处理");
|
||||
assertTrue(Files.exists(outside), "根目录之外的内容绝不能被删除");
|
||||
}
|
||||
|
||||
/** 绝对路径同样不得越过根目录。 */
|
||||
@Test
|
||||
void deleteWithinRejectsAbsolutePath(@TempDir Path root) throws Exception {
|
||||
Path outside = Files.createDirectories(root.resolve("important"));
|
||||
Path galleryRoot = Files.createDirectories(root.resolve("gallery"));
|
||||
|
||||
assertEquals(ErrorCode.FILE_NOT_FOUND, DeleteService.deleteWithin(galleryRoot.toString(), outside.toString()));
|
||||
assertTrue(Files.exists(outside));
|
||||
}
|
||||
|
||||
/** 空名会导致规范化结果等于根目录本身,必须拒绝。 */
|
||||
@Test
|
||||
void deleteWithinRejectsEmptyName(@TempDir Path root) throws Exception {
|
||||
Path galleryRoot = Files.createDirectories(root.resolve("gallery"));
|
||||
|
||||
assertEquals(ErrorCode.FILE_NOT_FOUND, DeleteService.deleteWithin(galleryRoot.toString(), " "));
|
||||
assertTrue(Files.exists(galleryRoot), "根目录本身不得被删除");
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,113 @@
|
||||
package lion.Service;
|
||||
|
||||
import lion.Domain.GalleryTask;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import org.junit.jupiter.api.io.TempDir;
|
||||
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.nio.file.Files;
|
||||
import java.nio.file.Path;
|
||||
import java.util.concurrent.ConcurrentHashMap;
|
||||
import java.util.zip.ZipEntry;
|
||||
import java.util.zip.ZipOutputStream;
|
||||
|
||||
import static org.junit.jupiter.api.Assertions.*;
|
||||
|
||||
/**
|
||||
* 入队时按 gid 定位磁盘内容。
|
||||
*
|
||||
* <p>gid 稳定、目录名不稳定(下载器会追加分辨率等后缀),因此匹配不能只看名字相等,
|
||||
* 而要按 {@code [gid]} / {@code [gid-...]} 标记判断;同时绝不能把 {@code [1234]}
|
||||
* 误判成 gid 123,否则会把下载链接指向错误的画廊。
|
||||
*/
|
||||
class DownloadCheckServiceGidMatchingTest {
|
||||
|
||||
/** 目录名带后缀时也要能按 gid 命中已有归档,并判为压缩完成。 */
|
||||
@Test
|
||||
void matchesStoredArchiveWhenDirectoryNameHasSuffix(@TempDir Path root) throws Exception {
|
||||
DownloadCheckService service = service(root);
|
||||
Path directory = Files.createDirectories(
|
||||
Path.of(service.storagePath, "sample [123-456]"));
|
||||
writeValidZip(directory.resolve("sample [123-456].zip"));
|
||||
|
||||
GalleryTask result = service.addToQueue(task(123));
|
||||
|
||||
assertTrue(result.is_compress_complete(), "gid 命中已有归档应判为已完成");
|
||||
assertEquals("sample [123-456]", result.getName(), "名称应取实际目录名");
|
||||
}
|
||||
|
||||
/** 尚未出现 galleryinfo.txt:判为下载中,并统计已下载页数。 */
|
||||
@Test
|
||||
void marksTaskDownloadingWhileGalleryInfoMissing(@TempDir Path root) throws Exception {
|
||||
DownloadCheckService service = service(root);
|
||||
Path directory = Files.createDirectories(Path.of(service.downloadPath, "sample [123-456]"));
|
||||
Files.writeString(directory.resolve("1.jpg"), "x");
|
||||
Files.writeString(directory.resolve("2.jpg"), "y");
|
||||
|
||||
GalleryTask result = service.addToQueue(task(123));
|
||||
|
||||
assertEquals(GalleryTask.DOWNLOADING, result.getStatus());
|
||||
assertEquals(2, result.getProceeding(), "应统计除 galleryinfo.txt 外的页数");
|
||||
assertEquals("sample [123-456]", result.getName());
|
||||
}
|
||||
|
||||
/** 出现 galleryinfo.txt:判为下载完成,并记录源目录用于后续压缩。 */
|
||||
@Test
|
||||
void marksTaskCompleteWhenGalleryInfoPresent(@TempDir Path root) throws Exception {
|
||||
DownloadCheckService service = service(root);
|
||||
Path directory = Files.createDirectories(Path.of(service.downloadPath, "sample [123]"));
|
||||
Files.writeString(directory.resolve("1.jpg"), "x");
|
||||
Files.writeString(directory.resolve("galleryinfo.txt"), "meta");
|
||||
|
||||
GalleryTask result = service.addToQueue(task(123));
|
||||
|
||||
assertEquals(GalleryTask.DOWNLOAD_COMPLETE, result.getStatus());
|
||||
assertEquals(directory.toString(), result.getPath());
|
||||
}
|
||||
|
||||
/** gid 前缀相同但实际不同的目录不得被匹配。 */
|
||||
@Test
|
||||
void doesNotMatchLongerGidSharingTheSamePrefix(@TempDir Path root) throws Exception {
|
||||
DownloadCheckService service = service(root);
|
||||
Files.createDirectories(Path.of(service.downloadPath, "sample [1234]"));
|
||||
|
||||
GalleryTask result = service.addToQueue(task(123));
|
||||
|
||||
assertEquals(0, result.getStatus(), "[1234] 不得被当成 gid 123 的目录");
|
||||
assertNull(result.getName(), "未命中时不应改写任务名称");
|
||||
}
|
||||
|
||||
/** 归档校验:空文件与不存在的路径都不算有效归档。 */
|
||||
@Test
|
||||
void rejectsEmptyAndMissingArchives(@TempDir Path root) throws Exception {
|
||||
Path empty = Files.createFile(root.resolve("empty.zip"));
|
||||
assertFalse(DownloadCheckService.isValidArchive(empty.toFile()));
|
||||
assertFalse(DownloadCheckService.isValidArchive(root.resolve("missing.zip").toFile()));
|
||||
|
||||
Path valid = root.resolve("valid.zip");
|
||||
writeValidZip(valid);
|
||||
assertTrue(DownloadCheckService.isValidArchive(valid.toFile()));
|
||||
}
|
||||
|
||||
private static DownloadCheckService service(Path root) throws Exception {
|
||||
DownloadCheckService service = new DownloadCheckService(new ConcurrentHashMap<>(), false);
|
||||
service.downloadPath = Files.createDirectories(root.resolve("download")).toString();
|
||||
service.storagePath = Files.createDirectories(root.resolve("gallery")).toString();
|
||||
return service;
|
||||
}
|
||||
|
||||
private static GalleryTask task(int gid) {
|
||||
GalleryTask task = new GalleryTask();
|
||||
task.setGid(gid);
|
||||
return task;
|
||||
}
|
||||
|
||||
private static void writeValidZip(Path zip) throws Exception {
|
||||
Files.createDirectories(zip.getParent());
|
||||
try (ZipOutputStream out = new ZipOutputStream(Files.newOutputStream(zip))) {
|
||||
out.putNextEntry(new ZipEntry("1.jpg"));
|
||||
out.write("image bytes".getBytes(StandardCharsets.UTF_8));
|
||||
out.closeEntry();
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,78 @@
|
||||
package lion.Service;
|
||||
|
||||
import lion.Domain.GalleryTask;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import org.junit.jupiter.api.io.TempDir;
|
||||
import java.nio.file.*;
|
||||
import java.util.concurrent.ConcurrentHashMap;
|
||||
import static org.junit.jupiter.api.Assertions.*;
|
||||
|
||||
class DownloadCheckServiceTest {
|
||||
@Test
|
||||
void failedCompressionRetainsSourceAndCanRetry(@TempDir Path root) throws Exception {
|
||||
var queue = new ConcurrentHashMap<Integer, GalleryTask>();
|
||||
var service = new DownloadCheckService(queue, false);
|
||||
Path downloads = Files.createDirectory(root.resolve("downloads"));
|
||||
Path source = Files.createDirectory(downloads.resolve("sample [123]"));
|
||||
Files.writeString(source.resolve("galleryinfo.txt"), "metadata");
|
||||
Files.writeString(source.resolve("1.jpg"), "image bytes");
|
||||
Path storage = root.resolve("storage");
|
||||
Files.writeString(storage, "block mkdir");
|
||||
service.downloadPath = downloads.toString();
|
||||
service.storagePath = storage.toString();
|
||||
GalleryTask task = new GalleryTask();
|
||||
task.setGid(123);
|
||||
service.addToQueue(task);
|
||||
service.downloadCheck();
|
||||
service.compress();
|
||||
assertEquals(GalleryTask.DOWNLOAD_COMPLETE, task.getStatus());
|
||||
assertTrue(Files.exists(source.resolve("1.jpg")));
|
||||
Files.delete(storage);
|
||||
Files.createDirectory(storage);
|
||||
service.addToQueue(task); // manual retry removes the backoff
|
||||
service.downloadCheck();
|
||||
service.compress();
|
||||
assertEquals(GalleryTask.COMPRESS_COMPLETE, task.getStatus());
|
||||
assertTrue(DownloadCheckService.isValidArchive(storage.resolve("sample [123]/sample [123].zip").toFile()));
|
||||
assertFalse(Files.exists(source));
|
||||
}
|
||||
|
||||
@Test
|
||||
void corruptArchiveIsNotTreatedAsCompleted(@TempDir Path root) throws Exception {
|
||||
Path stored = Files.createDirectories(root.resolve("stored/sample [123]"));
|
||||
Files.writeString(stored.resolve("sample [123].zip"), "partial zip");
|
||||
var service = new DownloadCheckService(new ConcurrentHashMap<>(), false);
|
||||
service.storagePath = root.resolve("stored").toString();
|
||||
service.downloadPath = root.resolve("downloads").toString();
|
||||
GalleryTask task = new GalleryTask();
|
||||
task.setGid(123);
|
||||
assertFalse(service.addToQueue(task).is_compress_complete());
|
||||
}
|
||||
|
||||
/**
|
||||
* 已有有效归档时必须判为完成,且重复添加(命中校验缓存)行为保持一致。
|
||||
* 校验缓存按「路径 + 大小」复用,目的是避免每次 addToQueue 都把整包读一遍,
|
||||
* 但它绝不能让有效归档被误判为未完成。
|
||||
*/
|
||||
@Test
|
||||
void validArchiveIsRecognizedAcrossRepeatedQueueAdds(@TempDir Path root) throws Exception {
|
||||
Path stored = Files.createDirectories(root.resolve("stored/sample [123]"));
|
||||
Path zip = stored.resolve("sample [123].zip");
|
||||
try (var out = new java.util.zip.ZipOutputStream(Files.newOutputStream(zip))) {
|
||||
out.putNextEntry(new java.util.zip.ZipEntry("1.jpg"));
|
||||
out.write("image bytes".getBytes(java.nio.charset.StandardCharsets.UTF_8));
|
||||
out.closeEntry();
|
||||
}
|
||||
var service = new DownloadCheckService(new ConcurrentHashMap<>(), false);
|
||||
service.storagePath = root.resolve("stored").toString();
|
||||
service.downloadPath = root.resolve("downloads").toString();
|
||||
|
||||
GalleryTask first = new GalleryTask();
|
||||
first.setGid(123);
|
||||
assertTrue(service.addToQueue(first).is_compress_complete(), "存在有效归档时应判为完成");
|
||||
|
||||
GalleryTask second = new GalleryTask();
|
||||
second.setGid(123);
|
||||
assertTrue(service.addToQueue(second).is_compress_complete(), "第二次应命中校验缓存且结论一致");
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,117 @@
|
||||
package lion.Service;
|
||||
|
||||
import io.netty.channel.Channel;
|
||||
import io.netty.channel.embedded.EmbeddedChannel;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
import static org.junit.jupiter.api.Assertions.*;
|
||||
|
||||
class PrimaryChannelTrackerTest {
|
||||
|
||||
/**
|
||||
* 未认证通道退出时不得清空引用。
|
||||
*
|
||||
* <p>这正是线上事故的成因:并发认证让引用先指向 X 再被 Y 覆盖,
|
||||
* Y 退出时把引用清空,而仍在工作的 X 从此不再被用于上报任务状态。
|
||||
*/
|
||||
@Test
|
||||
void unregisterOfNonPrimaryChannelKeepsReference() {
|
||||
PrimaryChannelTracker tracker = new PrimaryChannelTracker();
|
||||
EmbeddedChannel primary = new EmbeddedChannel();
|
||||
EmbeddedChannel other = new EmbeddedChannel();
|
||||
try {
|
||||
tracker.onPrimaryMessage(primary);
|
||||
tracker.onPrimaryMessage(other);
|
||||
tracker.onPrimaryMessage(primary); // 引用回到仍在工作的那条通道
|
||||
|
||||
tracker.unregister(other);
|
||||
|
||||
assertSame(primary, tracker.current());
|
||||
assertTrue(tracker.usable());
|
||||
} finally {
|
||||
primary.finishAndReleaseAll();
|
||||
other.finishAndReleaseAll();
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 首选通道退出时立即回退到其它仍可用的已认证通道。
|
||||
*
|
||||
* <p>事故中节点引用的那条通道退出后引用被清空,而另一条已认证通道仍然活跃,
|
||||
* 任务状态就此无人上报。回退保证「有已认证通道在,就一定能上报」。
|
||||
*/
|
||||
@Test
|
||||
void unregisterOfPrimaryFallsBackToSurvivingChannel() {
|
||||
PrimaryChannelTracker tracker = new PrimaryChannelTracker();
|
||||
EmbeddedChannel survivor = new EmbeddedChannel();
|
||||
EmbeddedChannel transientChannel = new EmbeddedChannel();
|
||||
try {
|
||||
tracker.onPrimaryMessage(survivor);
|
||||
tracker.onPrimaryMessage(transientChannel);
|
||||
tracker.unregister(transientChannel);
|
||||
|
||||
assertSame(survivor, tracker.current(), "应回退到仍可用的已认证通道");
|
||||
assertTrue(tracker.usable());
|
||||
} finally {
|
||||
survivor.finishAndReleaseAll();
|
||||
transientChannel.finishAndReleaseAll();
|
||||
}
|
||||
}
|
||||
|
||||
/** 实际在收发消息的已认证通道应当接管引用,即使原引用仍处于 active 状态。 */
|
||||
@Test
|
||||
void authenticatedChannelTakesOverOnMessage() {
|
||||
PrimaryChannelTracker tracker = new PrimaryChannelTracker();
|
||||
EmbeddedChannel stalePrimary = new EmbeddedChannel();
|
||||
EmbeddedChannel live = new EmbeddedChannel();
|
||||
try {
|
||||
tracker.onPrimaryMessage(stalePrimary);
|
||||
tracker.onPrimaryMessage(live);
|
||||
tracker.onPrimaryMessage(stalePrimary); // 引用先落在 stalePrimary 上
|
||||
|
||||
tracker.onPrimaryMessage(live);
|
||||
|
||||
assertSame(live, tracker.current(), "说话的通道应成为引用");
|
||||
} finally {
|
||||
stalePrimary.finishAndReleaseAll();
|
||||
live.finishAndReleaseAll();
|
||||
}
|
||||
}
|
||||
|
||||
/** 未经认证的通道不得占用引用,避免陌生连接抢走状态上报通道。 */
|
||||
@Test
|
||||
void unauthenticatedChannelCannotClaimReference() {
|
||||
PrimaryChannelTracker tracker = new PrimaryChannelTracker();
|
||||
EmbeddedChannel authenticated = new EmbeddedChannel();
|
||||
EmbeddedChannel stranger = new EmbeddedChannel();
|
||||
try {
|
||||
tracker.onPrimaryMessage(authenticated);
|
||||
|
||||
// 未认证通道不会进入已认证集合,因此不能用它认领。
|
||||
|
||||
assertSame(authenticated, tracker.current());
|
||||
} finally {
|
||||
authenticated.finishAndReleaseAll();
|
||||
stranger.finishAndReleaseAll();
|
||||
}
|
||||
}
|
||||
|
||||
/** 已下线的通道即使曾被认证,也不能再占用引用。 */
|
||||
@Test
|
||||
void inactiveChannelIsNotUsable() {
|
||||
PrimaryChannelTracker tracker = new PrimaryChannelTracker();
|
||||
EmbeddedChannel channel = new EmbeddedChannel();
|
||||
tracker.onPrimaryMessage(channel);
|
||||
channel.finishAndReleaseAll();
|
||||
|
||||
assertFalse(tracker.usable());
|
||||
}
|
||||
|
||||
@Test
|
||||
void emptyTrackerHasNoUsableChannel() {
|
||||
PrimaryChannelTracker tracker = new PrimaryChannelTracker();
|
||||
|
||||
assertNull(tracker.current());
|
||||
assertFalse(tracker.usable());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,47 @@
|
||||
package lion.Service;
|
||||
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
import static org.junit.jupiter.api.Assertions.*;
|
||||
|
||||
class RekickPolicyTest {
|
||||
|
||||
@Test
|
||||
void noPendingReportsNeverRekicks() {
|
||||
RekickPolicy policy = new RekickPolicy();
|
||||
|
||||
assertFalse(policy.shouldRekick(false, false, 1_000L));
|
||||
}
|
||||
|
||||
@Test
|
||||
void usableChannelNeverRekicks() {
|
||||
RekickPolicy policy = new RekickPolicy();
|
||||
|
||||
assertFalse(policy.shouldRekick(true, true, 1_000L));
|
||||
}
|
||||
|
||||
@Test
|
||||
void pendingWithoutChannelRekicksThenRateLimits() {
|
||||
RekickPolicy policy = new RekickPolicy();
|
||||
long now = 1_000_000L;
|
||||
|
||||
assertTrue(policy.shouldRekick(true, false, now), "首次应允许重连");
|
||||
assertFalse(policy.shouldRekick(true, false, now + 1), "间隔内应被限流");
|
||||
assertFalse(policy.shouldRekick(true, false, now + RekickPolicy.MIN_INTERVAL_MILLIS - 1));
|
||||
assertTrue(policy.shouldRekick(true, false, now + RekickPolicy.MIN_INTERVAL_MILLIS),
|
||||
"超过最小间隔后应再次允许");
|
||||
}
|
||||
|
||||
/** 限流只抑制连击,不应把「已限流」永久锁死后续尝试。 */
|
||||
@Test
|
||||
void rateLimitDoesNotBlockForever() {
|
||||
RekickPolicy policy = new RekickPolicy();
|
||||
long now = 500_000L;
|
||||
|
||||
policy.shouldRekick(true, false, now);
|
||||
for (int i = 0; i < 5; i++)
|
||||
policy.shouldRekick(true, false, now + i * 1_000L);
|
||||
|
||||
assertTrue(policy.shouldRekick(true, false, now + 10 * RekickPolicy.MIN_INTERVAL_MILLIS));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,181 @@
|
||||
package lion.Service;
|
||||
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
import lion.CustomUtil;
|
||||
import lion.Message.Main.SubscriptionAccountSnapshot;
|
||||
import lion.Message.Main.SubscriptionBindingSnapshot;
|
||||
import lion.Message.Main.SubscriptionSnapshotMessage;
|
||||
import lion.Message.Main.SubscriptionSnapshotPayload;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import org.junit.jupiter.api.io.TempDir;
|
||||
|
||||
import javax.crypto.Mac;
|
||||
import javax.crypto.spec.SecretKeySpec;
|
||||
import java.io.ByteArrayOutputStream;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.nio.file.Path;
|
||||
import java.security.MessageDigest;
|
||||
import java.util.Base64;
|
||||
import java.util.HexFormat;
|
||||
import java.util.zip.GZIPOutputStream;
|
||||
|
||||
import static org.junit.jupiter.api.Assertions.*;
|
||||
|
||||
class SubscriptionSnapshotStoreTest {
|
||||
private static final String SECRET = "snapshot-test-secret";
|
||||
private final ObjectMapper mapper = CustomUtil.objectMapper;
|
||||
|
||||
@Test
|
||||
void appliesSnapshotAndServesByHashedPublicKey(@TempDir Path directory) throws Exception {
|
||||
SubscriptionSnapshotStore store = new SubscriptionSnapshotStore(directory, SECRET, 3600, 1024 * 1024);
|
||||
SubscriptionSnapshotMessage message = message("public-key-1", "v2-content", "clash-content", System.currentTimeMillis());
|
||||
|
||||
assertEquals(SubscriptionSnapshotStore.APPLY_SUCCESS, store.apply(message).code());
|
||||
SubscriptionSnapshotStore.Lookup v2 = store.lookup("v2", "public-key-1");
|
||||
SubscriptionSnapshotStore.Lookup clash = store.lookup("cat", "public-key-1");
|
||||
assertArrayEquals("v2-content".getBytes(StandardCharsets.UTF_8), v2.content());
|
||||
assertArrayEquals("clash-content".getBytes(StandardCharsets.UTF_8), clash.content());
|
||||
assertEquals(SubscriptionSnapshotStore.APPLY_OLD, store.apply(message).code());
|
||||
}
|
||||
|
||||
@Test
|
||||
void rejectsTamperedPayloadAndKeepsPreviousSnapshot(@TempDir Path directory) throws Exception {
|
||||
SubscriptionSnapshotStore store = new SubscriptionSnapshotStore(directory, SECRET, 3600, 1024 * 1024);
|
||||
SubscriptionSnapshotMessage message = message("public-key-1", "v2-content", "clash-content", System.currentTimeMillis());
|
||||
assertEquals(SubscriptionSnapshotStore.APPLY_SUCCESS, store.apply(message).code());
|
||||
message.setPayloadBase64(Base64.getEncoder().encodeToString("tampered".getBytes(StandardCharsets.UTF_8)));
|
||||
assertEquals(SubscriptionSnapshotStore.APPLY_INVALID, store.apply(message).code());
|
||||
assertArrayEquals("v2-content".getBytes(StandardCharsets.UTF_8), store.lookup("v2", "public-key-1").content());
|
||||
}
|
||||
|
||||
@Test
|
||||
void loadsLastGoodSnapshotAfterRestart(@TempDir Path directory) throws Exception {
|
||||
SubscriptionSnapshotMessage message = message("public-key-1", "v2-content", "clash-content", System.currentTimeMillis());
|
||||
SubscriptionSnapshotStore first = new SubscriptionSnapshotStore(directory, SECRET, 3600, 1024 * 1024);
|
||||
assertEquals(SubscriptionSnapshotStore.APPLY_SUCCESS, first.apply(message).code());
|
||||
SubscriptionSnapshotStore second = new SubscriptionSnapshotStore(directory, SECRET, 3600, 1024 * 1024);
|
||||
second.load();
|
||||
assertArrayEquals("clash-content".getBytes(StandardCharsets.UTF_8), second.lookup("cat", "public-key-1").content());
|
||||
}
|
||||
|
||||
/**
|
||||
* 回归:节点曾按「内容里的 generatedAt」判过期,而 revision 是内容寻址的——
|
||||
* 内容长期不变时主站每分钟重发同一 revision 会被判为 APPLY_OLD 直接丢弃,
|
||||
* 新鲜度永远停在首次接收那天,于是主站在线且持续同步,备机仍在第 7 天开始 503。
|
||||
* 现在过期只取决于「最近一次成功接收主站快照的时刻」。
|
||||
*/
|
||||
@Test
|
||||
void repeatedSameRevisionRefreshesFreshnessAndNeverExpires(@TempDir Path directory) throws Exception {
|
||||
// 有效期 1 秒,便于在用例内观察「不续期会过期、续期后恢复」。
|
||||
SubscriptionSnapshotStore store = new SubscriptionSnapshotStore(directory, SECRET, 1, 1024 * 1024);
|
||||
SubscriptionSnapshotMessage message = message("public-key-1", "v2-content", "clash-content", System.currentTimeMillis());
|
||||
|
||||
assertEquals(SubscriptionSnapshotStore.APPLY_SUCCESS, store.apply(message).code());
|
||||
assertEquals("ready", store.status().state());
|
||||
|
||||
// 超过有效期且期间没有任何推送 → 过期(证明判定确实生效,不是恒 ready)
|
||||
Thread.sleep(1_200);
|
||||
assertEquals("expired", store.status().state(), "超过有效期且无推送应判过期");
|
||||
|
||||
// 同 revision 重发:仍是 APPLY_OLD(内容幂等),但必须刷新新鲜度
|
||||
assertEquals(SubscriptionSnapshotStore.APPLY_OLD, store.apply(message).code());
|
||||
assertEquals("ready", store.status().state(), "重新收到同 revision 后应恢复可用");
|
||||
assertNotNull(store.lookup("v2", "public-key-1"), "持续同步期间必须能取到订阅");
|
||||
}
|
||||
|
||||
/** 超过有效期后仍未收到任何快照,才判过期并停止分发。 */
|
||||
@Test
|
||||
void expiresOnlyAfterNoSnapshotArrivesWithinMaxAge(@TempDir Path directory) throws Exception {
|
||||
SubscriptionSnapshotStore store = new SubscriptionSnapshotStore(directory, SECRET, 3600, 1024 * 1024);
|
||||
// 内容生成于 2 小时前,且重启后再没收到主站推送 → 超期。
|
||||
SubscriptionSnapshotMessage message = message("public-key-1", "v2-content", "clash-content",
|
||||
System.currentTimeMillis() - 7_200_000L);
|
||||
assertEquals(SubscriptionSnapshotStore.APPLY_SUCCESS, store.apply(message).code());
|
||||
|
||||
SubscriptionSnapshotStore reloaded = new SubscriptionSnapshotStore(directory, SECRET, 3600, 1024 * 1024);
|
||||
reloaded.load();
|
||||
assertEquals("expired", reloaded.status().state(), "重启后未再收到快照,超期应判过期");
|
||||
assertNull(reloaded.lookup("v2", "public-key-1"), "过期快照不得继续分发");
|
||||
}
|
||||
|
||||
/**
|
||||
* 主站的常规存活信号(例如每 30 分钟的可用性检查)必须刷新新鲜度。
|
||||
*
|
||||
* <p>这是「过期 ⇔ 主站失联」的关键:快照是内容寻址的,内容长期不变时主站没有理由
|
||||
* 重发整份快照;若新鲜度只由「收到快照」驱动,备机会在内容不动的第 7 天误判过期。
|
||||
*/
|
||||
@Test
|
||||
void primaryContactWithoutNewSnapshotKeepsSubscriptionFresh(@TempDir Path directory) throws Exception {
|
||||
SubscriptionSnapshotStore store = new SubscriptionSnapshotStore(directory, SECRET, 1, 1024 * 1024);
|
||||
SubscriptionSnapshotMessage message = message("public-key-1", "v2-content", "clash-content", System.currentTimeMillis());
|
||||
assertEquals(SubscriptionSnapshotStore.APPLY_SUCCESS, store.apply(message).code());
|
||||
|
||||
// 超过有效期(1 秒),期间只有主站存活信号、没有新快照
|
||||
Thread.sleep(1_200);
|
||||
assertEquals("expired", store.status().state(), "有效期已过且无任何主站信号,应先判过期");
|
||||
|
||||
store.markPrimaryContact();
|
||||
assertEquals("ready", store.status().state(), "收到主站存活信号后应恢复可用");
|
||||
assertNotNull(store.lookup("v2", "public-key-1"), "主站在线期间必须能取到订阅");
|
||||
// 内容仍是最初那份,说明续期靠的是存活信号,而不是内容变化
|
||||
assertArrayEquals("v2-content".getBytes(StandardCharsets.UTF_8), store.lookup("v2", "public-key-1").content());
|
||||
}
|
||||
|
||||
/** 主站失联(长时间没有任何信号)后仍必须判过期,不能因加入存活信号而永不失效。 */
|
||||
@Test
|
||||
void staysExpiredWhenPrimaryContactStops(@TempDir Path directory) throws Exception {
|
||||
SubscriptionSnapshotStore store = new SubscriptionSnapshotStore(directory, SECRET, 1, 1024 * 1024);
|
||||
SubscriptionSnapshotMessage message = message("public-key-1", "v2-content", "clash-content", System.currentTimeMillis());
|
||||
assertEquals(SubscriptionSnapshotStore.APPLY_SUCCESS, store.apply(message).code());
|
||||
store.markPrimaryContact();
|
||||
assertEquals("ready", store.status().state());
|
||||
|
||||
Thread.sleep(1_200); // 之后主站再无任何消息
|
||||
assertEquals("expired", store.status().state(), "主站失联超过有效期必须判过期");
|
||||
assertNull(store.lookup("v2", "public-key-1"), "失联后不得继续分发");
|
||||
}
|
||||
|
||||
private SubscriptionSnapshotMessage message(String publicKey, String v2, String clash, long generatedAt) throws Exception {
|
||||
byte[] v2Bytes = v2.getBytes(StandardCharsets.UTF_8);
|
||||
byte[] clashBytes = clash.getBytes(StandardCharsets.UTF_8);
|
||||
SubscriptionAccountSnapshot account = new SubscriptionAccountSnapshot();
|
||||
account.setAccountId(1);
|
||||
account.setEnabled(true);
|
||||
account.setV2ContentBase64(Base64.getEncoder().encodeToString(v2Bytes));
|
||||
account.setV2Sha256(sha256(v2Bytes));
|
||||
account.setClashContentBase64(Base64.getEncoder().encodeToString(clashBytes));
|
||||
account.setClashSha256(sha256(clashBytes));
|
||||
SubscriptionBindingSnapshot binding = new SubscriptionBindingSnapshot();
|
||||
binding.setPublicKeySha256(sha256(publicKey.getBytes(StandardCharsets.UTF_8)));
|
||||
binding.setAccountId(1);
|
||||
SubscriptionSnapshotPayload payload = new SubscriptionSnapshotPayload();
|
||||
payload.setSchemaVersion(1);
|
||||
payload.setAccounts(java.util.List.of(account));
|
||||
payload.setBindings(java.util.List.of(binding));
|
||||
byte[] json = mapper.writeValueAsBytes(payload);
|
||||
byte[] compressed = gzip(json);
|
||||
SubscriptionSnapshotMessage message = new SubscriptionSnapshotMessage();
|
||||
message.setSchemaVersion(1);
|
||||
message.setRevision(sha256(json));
|
||||
message.setGeneratedAt(generatedAt);
|
||||
message.setPayloadBase64(Base64.getEncoder().encodeToString(compressed));
|
||||
message.setPayloadSha256(sha256(compressed));
|
||||
String input = "1\n" + message.getRevision() + "\n" + generatedAt + "\n" + message.getPayloadSha256();
|
||||
Mac mac = Mac.getInstance("HmacSHA256");
|
||||
mac.init(new SecretKeySpec(SECRET.getBytes(StandardCharsets.UTF_8), "HmacSHA256"));
|
||||
message.setSignature(hex(mac.doFinal(input.getBytes(StandardCharsets.UTF_8))));
|
||||
return message;
|
||||
}
|
||||
|
||||
private static byte[] gzip(byte[] bytes) throws Exception {
|
||||
ByteArrayOutputStream output = new ByteArrayOutputStream();
|
||||
try (GZIPOutputStream gzip = new GZIPOutputStream(output)) { gzip.write(bytes); }
|
||||
return output.toByteArray();
|
||||
}
|
||||
|
||||
private static String sha256(byte[] bytes) throws Exception {
|
||||
return hex(MessageDigest.getInstance("SHA-256").digest(bytes));
|
||||
}
|
||||
|
||||
private static String hex(byte[] bytes) { return HexFormat.of().formatHex(bytes); }
|
||||
}
|
||||
@@ -0,0 +1,297 @@
|
||||
package lion;
|
||||
|
||||
import io.netty.channel.embedded.EmbeddedChannel;
|
||||
import lion.Config.Config;
|
||||
import lion.Domain.GalleryTask;
|
||||
import lion.ErrorCode.ErrorCode;
|
||||
import lion.Message.Main.AvailableCheckMessage;
|
||||
import lion.Message.Main.DeleteGalleryMessage;
|
||||
import lion.Message.Main.DownloadPostMessage;
|
||||
import lion.Message.Main.DownloadStatusMessage;
|
||||
import lion.Message.Main.IdentityMessage;
|
||||
import lion.Message.Main.ResponseMessage;
|
||||
import lion.Message.Main.SubscriptionSnapshotMessage;
|
||||
import lion.Service.SubscriptionSnapshotStore;
|
||||
import org.junit.jupiter.api.AfterAll;
|
||||
import org.junit.jupiter.api.BeforeAll;
|
||||
import org.junit.jupiter.api.BeforeEach;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import org.junit.jupiter.api.io.TempDir;
|
||||
|
||||
import java.nio.file.Files;
|
||||
import java.nio.file.Path;
|
||||
|
||||
import static org.junit.jupiter.api.Assertions.*;
|
||||
|
||||
/**
|
||||
* 节点消息处理与状态上报循环。
|
||||
*
|
||||
* <p>{@link storageNode.MyChannelInboundHandlerAdapter} 与 {@link storageNode#mainThread()}
|
||||
* 约 250 行,承载任务下发、删除、探活、订阅快照分发与队列重放,此前只有静态方法
|
||||
* {@code isPrimaryEvidence} 被间接覆盖。这里用 {@link EmbeddedChannel} 把入站消息直接
|
||||
* 喂给真实处理器,断言回包与队列副作用,同时锁定「无可用通道时不得丢弃任务」的语义。
|
||||
*/
|
||||
class StorageNodeMessageHandlingTest {
|
||||
|
||||
@TempDir
|
||||
static Path tempDir;
|
||||
|
||||
static storageNode node;
|
||||
|
||||
@BeforeAll
|
||||
static void startNode() throws Exception {
|
||||
Path storageRoot = Files.createDirectories(tempDir.resolve("gallery"));
|
||||
Path subscriptionRoot = Files.createDirectories(tempDir.resolve("sub"));
|
||||
|
||||
storageNode.storagePath = storageRoot + "/";
|
||||
SubscriptionSnapshotStore store =
|
||||
new SubscriptionSnapshotStore(subscriptionRoot, "test-secret", 3600, 1024 * 1024);
|
||||
node = new storageNode(store);
|
||||
// 订阅同步默认关闭:快照消息必须被拒绝,也不触发异步落盘。
|
||||
Config.subscriptionSyncEnabled = false;
|
||||
}
|
||||
|
||||
@AfterAll
|
||||
static void stopNode() {
|
||||
node.subscriptionApplyExecutor.shutdownNow();
|
||||
node.rekickExecutor.shutdownNow();
|
||||
}
|
||||
|
||||
@BeforeEach
|
||||
void resetState() {
|
||||
node.queue.clear();
|
||||
node.tempQueue.clear();
|
||||
}
|
||||
|
||||
/**
|
||||
* 任务下发:先按 gid 入队并立即回传当前状态,再回执响应。
|
||||
*
|
||||
* <p>回传顺序不能反——主站的单任务重试接口依赖「先收到状态、再收到响应」。
|
||||
*/
|
||||
@Test
|
||||
void downloadPostEnqueuesTaskAndReportsStatusBeforeResponse() {
|
||||
EmbeddedChannel channel = nodeChannel();
|
||||
try {
|
||||
DownloadPostMessage post = new DownloadPostMessage();
|
||||
post.setGalleryTask(task(90001));
|
||||
post.messageId = 7;
|
||||
|
||||
channel.writeInbound(post);
|
||||
|
||||
DownloadStatusMessage status = channel.readOutbound();
|
||||
assertNotNull(status, "应先把当前任务状态回传给主站");
|
||||
assertEquals(1, status.getGalleryTasks().length);
|
||||
assertEquals(90001, status.getGalleryTasks()[0].getGid());
|
||||
|
||||
ResponseMessage response = channel.readOutbound();
|
||||
assertNotNull(response, "随后应回执响应");
|
||||
assertEquals(7, response.messageId, "响应必须带上下发方的 messageId");
|
||||
assertEquals(0, response.getResult());
|
||||
|
||||
assertTrue(node.queue.containsKey(90001), "任务应已进入待处理队列");
|
||||
assertTrue(node.primaryChannel.usable(), "主站消息应认领通道引用");
|
||||
} finally {
|
||||
close(channel);
|
||||
}
|
||||
}
|
||||
|
||||
/** 删除任务:既清理待上报队列,也删掉磁盘目录,并回执成功。 */
|
||||
@Test
|
||||
void deleteGalleryRemovesQueuedTaskAndDirectory() throws Exception {
|
||||
EmbeddedChannel channel = nodeChannel();
|
||||
try {
|
||||
Path directory = Files.createDirectories(Path.of(storageNode.storagePath, "sample [90002]"));
|
||||
node.queue.put(90002, task(90002));
|
||||
|
||||
DeleteGalleryMessage delete = new DeleteGalleryMessage();
|
||||
delete.setGalleryName("sample [90002]");
|
||||
delete.messageId = 3;
|
||||
channel.writeInbound(delete);
|
||||
|
||||
ResponseMessage response = channel.readOutbound();
|
||||
assertNotNull(response);
|
||||
assertEquals(3, response.messageId);
|
||||
assertEquals(0, response.getResult(), "目录删除成功应回 0");
|
||||
|
||||
assertFalse(node.queue.containsKey(90002), "已删除的任务不得继续上报状态");
|
||||
assertFalse(Files.exists(directory), "磁盘目录应被删除");
|
||||
} finally {
|
||||
close(channel);
|
||||
}
|
||||
}
|
||||
|
||||
/** 删除不存在的目录时仍要回包,并如实返回 FILE_NOT_FOUND。 */
|
||||
@Test
|
||||
void deleteGalleryMissingDirectoryReportsFileNotFound() {
|
||||
EmbeddedChannel channel = nodeChannel();
|
||||
try {
|
||||
DeleteGalleryMessage delete = new DeleteGalleryMessage();
|
||||
delete.setGalleryName("never-existed [90003]");
|
||||
delete.messageId = 4;
|
||||
|
||||
channel.writeInbound(delete);
|
||||
|
||||
ResponseMessage response = channel.readOutbound();
|
||||
assertNotNull(response, "无论删除成败都必须回执,否则主站会一直等待");
|
||||
assertEquals(4, response.messageId);
|
||||
assertEquals(ErrorCode.FILE_NOT_FOUND, response.getResult());
|
||||
} finally {
|
||||
close(channel);
|
||||
}
|
||||
}
|
||||
|
||||
/** 探活:必须原样带回 messageId,供主站配对。 */
|
||||
@Test
|
||||
void availableCheckEchoesMessageId() {
|
||||
EmbeddedChannel channel = nodeChannel();
|
||||
try {
|
||||
AvailableCheckMessage probe = new AvailableCheckMessage();
|
||||
probe.messageId = 42;
|
||||
|
||||
channel.writeInbound(probe);
|
||||
|
||||
ResponseMessage response = channel.readOutbound();
|
||||
assertNotNull(response);
|
||||
assertEquals(42, response.messageId);
|
||||
assertEquals(0, response.getResult());
|
||||
} finally {
|
||||
close(channel);
|
||||
}
|
||||
}
|
||||
|
||||
/** 主站身份消息认领通道引用;备机身份消息不得认领,否则状态会上报到错误连接。 */
|
||||
@Test
|
||||
void onlyPrimaryIdentityClaimsChannelReference() {
|
||||
EmbeddedChannel primary = nodeChannel();
|
||||
EmbeddedChannel sidecar = nodeChannel();
|
||||
try {
|
||||
sidecar.writeInbound(identity("lionwebsiteside"));
|
||||
assertFalse(node.primaryChannel.usable(), "备机身份不得占用主站引用");
|
||||
assertSame(sidecar, node.node, "备机通道应登记为 node");
|
||||
|
||||
primary.writeInbound(identity("lionwebsite"));
|
||||
assertTrue(node.primaryChannel.usable(), "主站身份应认领引用");
|
||||
assertSame(primary, node.primaryChannel.current());
|
||||
} finally {
|
||||
close(primary);
|
||||
close(sidecar);
|
||||
}
|
||||
}
|
||||
|
||||
/** 未开启订阅同步时,快照消息必须被拒绝。 */
|
||||
@Test
|
||||
void subscriptionSnapshotRejectedWhenSyncDisabled() {
|
||||
EmbeddedChannel channel = nodeChannel();
|
||||
try {
|
||||
SubscriptionSnapshotMessage snapshot = new SubscriptionSnapshotMessage();
|
||||
snapshot.messageId = 9;
|
||||
snapshot.setSchemaVersion(1);
|
||||
snapshot.setRevision("a".repeat(64));
|
||||
|
||||
channel.writeInbound(snapshot);
|
||||
|
||||
ResponseMessage response = channel.readOutbound();
|
||||
assertNotNull(response);
|
||||
assertEquals(9, response.messageId);
|
||||
assertEquals(SubscriptionSnapshotStore.APPLY_INVALID, response.getResult());
|
||||
} finally {
|
||||
close(channel);
|
||||
}
|
||||
}
|
||||
|
||||
/** 有待上报任务且通道可用时,mainThread 应上报并清空已完成的压缩任务。 */
|
||||
@Test
|
||||
void mainThreadReportsCompletedTaskAndDrainsQueue() {
|
||||
EmbeddedChannel channel = nodeChannel();
|
||||
try {
|
||||
channel.writeInbound(identity("lionwebsite"));
|
||||
drainOutbound(channel);
|
||||
|
||||
GalleryTask done = task(90004);
|
||||
done.setStatus(GalleryTask.COMPRESS_COMPLETE);
|
||||
node.queue.put(90004, done);
|
||||
|
||||
node.mainThread();
|
||||
|
||||
DownloadStatusMessage status = readStatusMessage(channel);
|
||||
assertNotNull(status, "通道可用时必须上报任务状态");
|
||||
assertEquals(1, status.getGalleryTasks().length);
|
||||
assertEquals(90004, status.getGalleryTasks()[0].getGid());
|
||||
assertFalse(node.queue.containsKey(90004), "压缩完成的任务上报后应出队");
|
||||
} finally {
|
||||
close(channel);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 无可用通道时不得上报,也绝不能丢弃任务——队列要保留到主站重连后重放。
|
||||
* 这是「状态永久卡在已提交」的另一半成因。
|
||||
*/
|
||||
@Test
|
||||
void mainThreadKeepsTasksWhenNoChannelIsUsable() {
|
||||
assertFalse(node.primaryChannel.usable(), "前置条件:本用例开始时不应有已认证通道");
|
||||
|
||||
GalleryTask done = task(90005);
|
||||
done.setStatus(GalleryTask.COMPRESS_COMPLETE);
|
||||
node.queue.put(90005, done);
|
||||
|
||||
node.mainThread();
|
||||
|
||||
assertTrue(node.queue.containsKey(90005), "无通道时任务必须保留在队列中等待重放");
|
||||
assertNull(node.primaryChannel.current());
|
||||
}
|
||||
|
||||
/** 队列为空时不得发送任务状态消息(心跳不算)。 */
|
||||
@Test
|
||||
void mainThreadSendsNoStatusWhenQueueIsEmpty() {
|
||||
EmbeddedChannel channel = nodeChannel();
|
||||
try {
|
||||
channel.writeInbound(identity("lionwebsite"));
|
||||
drainOutbound(channel);
|
||||
|
||||
node.mainThread();
|
||||
|
||||
assertNull(readStatusMessage(channel), "没有任务可报时不应发送状态消息");
|
||||
} finally {
|
||||
close(channel);
|
||||
}
|
||||
}
|
||||
|
||||
private static EmbeddedChannel nodeChannel() {
|
||||
return new EmbeddedChannel(node.new MyChannelInboundHandlerAdapter(node.tempQueue));
|
||||
}
|
||||
|
||||
private static GalleryTask task(int gid) {
|
||||
GalleryTask task = new GalleryTask();
|
||||
task.setGid(gid);
|
||||
task.setName("sample [" + gid + "]");
|
||||
return task;
|
||||
}
|
||||
|
||||
private static IdentityMessage identity(String value) {
|
||||
IdentityMessage message = new IdentityMessage();
|
||||
message.setIdentity(value);
|
||||
return message;
|
||||
}
|
||||
|
||||
/** 读出并丢弃全部出站消息,用于清场。 */
|
||||
private static int drainOutbound(EmbeddedChannel channel) {
|
||||
int count = 0;
|
||||
while (channel.readOutbound() != null)
|
||||
count++;
|
||||
return count;
|
||||
}
|
||||
|
||||
/** 找出第一条任务状态消息;容忍调度线程可能插入的心跳等消息。 */
|
||||
private static DownloadStatusMessage readStatusMessage(EmbeddedChannel channel) {
|
||||
Object outbound;
|
||||
while ((outbound = channel.readOutbound()) != null)
|
||||
if (outbound instanceof DownloadStatusMessage status)
|
||||
return status;
|
||||
return null;
|
||||
}
|
||||
|
||||
private static void close(EmbeddedChannel channel) {
|
||||
channel.finishAndReleaseAll();
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user