diff --git a/docs/prod-api-v1.md b/docs/prod-api-v1.md index 7639612..6c847f2 100644 --- a/docs/prod-api-v1.md +++ b/docs/prod-api-v1.md @@ -1,424 +1,272 @@ -# 生产端配置同步接口文档 V1 +# 生产端配置同步接口文档 V1 ## 1. 文档目的 -本文档定义生产端提供给同步工具使用的接口协议，用于支持以下两类能力： +本文档基于 `testapi.txt` 中给出的正式接口协议，定义生产端配置同步相关接口，供 Git 直连同步工具使用。 -- 接收开发侧下发的配置包并导入生产环境 -- 向同步工具提供当前生产环境配置快照 +本文档覆盖三类接口： -本文档面向： +1. `pushConfig`：把 Git 配置推送到生产 +2. `pullConfig`：把生产配置拉回到 Git +3. `login`：获取调用接口所需 token -- 生产端接口开发人员 -- 同步工具开发人员 -- 联调与运维人员 +## 2. 统一约定 -## 2. 适用范围 +### 2.1 编码与格式 -本文档适用于“基于 FTP 中转的配置双向同步工具”中的生产端接口。 +- 编码：`UTF-8` +- 返回格式：`JSON` +- 鉴权方式：请求头中携带 `token` -当前接口版本： +### 2.2 成功判定 -- `V1` +接口成功时返回： -生产端建议提供两个接口： +- `code = "0"` +- `msg = "ok"` -- `POST /api/config/push` -- `GET /api/config/pull` +### 2.3 失败响应格式 -## 3. 设计原则 - -- 接口必须可被自动化任务调用，不依赖人工交互 -- `push` 必须具备幂等能力，避免重复导入 -- `pull` 必须返回当前生产正在生效的配置快照 -- 返回结果要足够明确，便于同步工具决定是否写 ACK -- 接口协议尽量简单，优先保证稳定性和可追踪性 - -## 4. 认证与安全 - -### 4.1 认证方式 - -建议使用： - -- `Bearer Token` - -请求头示例： - -```http -Authorization: Bearer xxxxxxxxxxxxx -``` - -### 4.2 传输协议 - -建议使用： - -- `HTTPS` - -### 4.3 调用方 - -仅允许生产环境内部部署的 `prod-agent` 调用。 - -## 5. 公共约定 - -### 5.1 公共请求头 - -建议所有请求携带以下 Header： - -| Header | 必填 | 说明 | -| --- | --- | --- | -| `Authorization` | 是 | Bearer Token | -| `Content-Type` | `push` 必填 | `multipart/form-data` | -| `Accept` | 否 | 推荐 `application/json` | -| `X-Trace-Id` | 否 | 同步链路跟踪号，可与报文中的 `traceId` 一致 | - -### 5.2 公共响应体 - -除 `pull` 成功场景外，建议统一返回 JSON： +所有接口失败时的响应格式统一如下： ```json { - "code": "SUCCESS", - "message": "Operation succeeded", - "traceId": "4ec9506b4edf438ab6b6f7764e2d1d28", - "timestamp": "2026-04-16T10:20:30+08:00", - "data": {} + "code": "XXX-00-00-XXX", + "data": null, + "msg": "errmsg!", + "timestamp": "1776735560594" +} +``` + +## 3. 接口一：推送 Git 配置到生产 + +### 3.1 基本信息 + +- 地址：`http://ip:port/pic_bus_manage_monitor/configSync/pushConfig` +- 方法：`POST` +- 内容格式：`JSON` +- 鉴权：Header 中携带 `token` + +### 3.2 请求参数 + +无 URL 参数。 + +### 3.3 请求体 + +请求体为 JSON 数组，每个数组元素代表一个配置文件： + +```json +[ + { + "airportId": "test", + "appName": "XXX", + "configVersion": "R_XXX_V3.0.3_XXX", + "configContent": "配置内容", + "fileName": "配置文件名" + } +] +``` + +字段说明： + +| 字段 | 必填 | 说明 | +| --- | --- | --- | +| `airportId` | 是 | 机场编码 | +| `appName` | 是 | 应用名称 | +| `configVersion` | 是 | 配置版本号 | +| `configContent` | 是 | 配置内容 | +| `fileName` | 是 | 配置文件名 | + +### 3.4 响应体 + +```json +{ + "code": "0", + "data": { + "ackFail": [ + { + "airportId": "test", + "appName": "XXXx", + "configVersion": "R_XXX_V3.0.35.6.1_XXX", + "fileName": "配置文件名" + } + ] + }, + "msg": "ok" } ``` 字段说明： -| 字段 | 类型 | 说明 | -| --- | --- | --- | -| `code` | string | 业务码 | -| `message` | string | 说明信息 | -| `traceId` | string | 链路追踪号 | -| `timestamp` | string | 接口返回时间，ISO-8601 格式 | -| `data` | object | 业务数据 | +- `ackFail`：本次推送失败的配置项列表 +- 如果 `ackFail` 为空或不存在，可视为本次推送成功 +- 如果 `ackFail` 非空，应视为部分失败或失败 -### 5.3 公共业务码 +### 3.5 业务说明 -| code | 说明 | -| --- | --- | -| `SUCCESS` | 处理成功 | -| `ACCEPTED` | 已接收，异步处理中 | -| `INVALID_PARAM` | 参数错误 | -| `UNAUTHORIZED` | 认证失败 | -| `FORBIDDEN` | 无权限 | -| `DUPLICATE_REQUEST` | 重复请求 | -| `PACKAGE_VALIDATE_FAILED` | 包校验失败 | -| `CONFIG_IMPORT_FAILED` | 配置导入失败 | -| `CONFIG_NOT_FOUND` | 当前无可导出的生产配置 | -| `INTERNAL_ERROR` | 系统内部异常 | +- 第一次推送为全量 +- 之后按增量推送 +- 如果全量配置过大，需要视情况拆分多次推送 +- `configContent` 需要加密 +- **加密方式当前留为 `TODO`** -说明： +## 4. 接口二：从生产拉取配置到 Git -- 当前同步工具实现更适合同步处理，因此 `push` 推荐返回 `SUCCESS`，不建议首版做异步。 - -## 6. 配置包约定 - -`push` 接口接收的文件建议为 zip 包，结构如下： - -```text -package.zip - |- manifest.json - |- config/ - |- sha256.txt -``` - -### 6.1 manifest.json 示例 - -```json -{ - "traceId": "4ec9506b4edf438ab6b6f7764e2d1d28", - "direction": "DEV_TO_PROD", - "sourceEnv": "DEV", - "sourceVersion": "c1d2e3f4", - "contentHash": "b0f8f1d0ef7b7f0b8cb72a1cbf877f49d2d4073c8444a64eb8fd2e684cb7fe53", - "createdAt": "2026-04-16T10:18:00+08:00", - "packageName": "dev_to_prod-c1d2e3f4-4ec9506b4edf438ab6b6f7764e2d1d28.zip" -} -``` - -### 6.2 生产端校验要求 - -生产端在导入前建议至少执行以下校验： - -1. zip 包可正常解压 -2. `manifest.json` 存在且字段完整 -3. `config/` 目录存在 -4. `manifest.contentHash` 与实际内容哈希一致 -5. `direction` 必须为 `DEV_TO_PROD` -6. 同一个 `traceId` 或同一个 `sourceVersion + contentHash` 不重复导入 - -## 7. 接口一：推送配置到生产 - -### 7.1 接口信息 - -- 方法：`POST` -- 路径：`/api/config/push` -- 说明：接收开发侧配置包并导入生产环境 - -### 7.2 请求格式 - -`Content-Type`： - -- `multipart/form-data` - -表单字段定义： - -| 字段 | 类型 | 必填 | 说明 | -| --- | --- | --- | --- | -| `file` | file | 是 | 配置 zip 包 | -| `traceId` | string | 是 | 同步链路唯一标识 | -| `direction` | string | 是 | 固定为 `DEV_TO_PROD` | -| `sourceVersion` | string | 是 | 开发侧来源版本号，通常为 Git Commit ID | -| `contentHash` | string | 是 | 配置内容哈希 | - -### 7.3 curl 示例 - -```bash -curl -X POST "https://prod.example.com/api/config/push" \ - -H "Authorization: Bearer xxxxxxxxxxxxx" \ - -H "Accept: application/json" \ - -F "file=@dev_to_prod-c1d2e3f4-4ec9506b4edf438ab6b6f7764e2d1d28.zip" \ - -F "traceId=4ec9506b4edf438ab6b6f7764e2d1d28" \ - -F "direction=DEV_TO_PROD" \ - -F "sourceVersion=c1d2e3f4" \ - -F "contentHash=b0f8f1d0ef7b7f0b8cb72a1cbf877f49d2d4073c8444a64eb8fd2e684cb7fe53" -``` - -### 7.4 成功响应示例 - -```json -{ - "code": "SUCCESS", - "message": "Configuration imported successfully", - "traceId": "4ec9506b4edf438ab6b6f7764e2d1d28", - "timestamp": "2026-04-16T10:20:30+08:00", - "data": { - "applied": true, - "sourceVersion": "c1d2e3f4", - "contentHash": "b0f8f1d0ef7b7f0b8cb72a1cbf877f49d2d4073c8444a64eb8fd2e684cb7fe53", - "importTime": "2026-04-16T10:20:30+08:00" - } -} -``` - -### 7.5 重复请求响应示例 - -```json -{ - "code": "DUPLICATE_REQUEST", - "message": "The same package has already been applied", - "traceId": "4ec9506b4edf438ab6b6f7764e2d1d28", - "timestamp": "2026-04-16T10:20:35+08:00", - "data": { - "applied": true, - "sourceVersion": "c1d2e3f4", - "contentHash": "b0f8f1d0ef7b7f0b8cb72a1cbf877f49d2d4073c8444a64eb8fd2e684cb7fe53" - } -} -``` - -说明： - -- 对同步工具而言，`DUPLICATE_REQUEST` 可以按成功处理。 - -### 7.6 失败响应示例 - -```json -{ - "code": "PACKAGE_VALIDATE_FAILED", - "message": "Package content hash mismatch", - "traceId": "4ec9506b4edf438ab6b6f7764e2d1d28", - "timestamp": "2026-04-16T10:20:40+08:00", - "data": {} -} -``` - -### 7.7 HTTP 状态码建议 - -| HTTP 状态码 | 场景 | -| --- | --- | -| `200` | 导入成功或重复导入 | -| `400` | 参数缺失、方向错误、文件格式错误 | -| `401` | Token 无效 | -| `403` | 无权限 | -| `409` | 幂等冲突或状态冲突 | -| `422` | 包内容校验失败、配置业务校验失败 | -| `500` | 服务端异常 | - -### 7.8 处理语义 - -`push` 接口建议采用同步处理语义： - -1. 接收请求 -2. 校验包 -3. 导入并应用配置 -4. 成功后返回 `200` - -说明： - -- 同步工具当前在收到 `2xx` 后会回写 FTP ACK，因此生产端若采用异步处理，会导致工具提前判定成功。 -- 如后续要改异步，需要同时调整同步工具 ACK 逻辑。 - -## 8. 接口二：拉取生产配置快照 - -### 8.1 接口信息 +### 4.1 基本信息 +- 地址：`http://ip:port/pic_bus_manage_monitor/configSync/pullConfig` - 方法：`GET` -- 路径：`/api/config/pull` -- 说明：返回当前生产环境已生效配置的快照 +- 内容格式：`JSON` +- 鉴权：Header 中携带 `token` -### 8.2 请求格式 +### 4.2 请求参数 -当前 V1 版不要求请求参数。 - -curl 示例： - -```bash -curl -X GET "https://prod.example.com/api/config/pull" \ - -H "Authorization: Bearer xxxxxxxxxxxxx" -``` - -### 8.3 成功响应约定 - -成功时返回： - -- `HTTP 200` -- `Content-Type: application/json;charset=UTF-8` - -推荐响应头： - -| Header | 必填 | 说明 | -| --- | --- | --- | -| `X-Config-Version` | 推荐 | 当前生产配置版本号 | -| `X-Config-Hash` | 推荐 | 当前配置内容哈希 | -| `ETag` | 可选 | 可作为版本标识备用 | - -说明： - -- 当前同步工具优先读取 `X-Config-Version`，其次读取 `ETag`，如果都没有则退化为以内容哈希作为版本号。 - -### 8.4 成功响应体示例 +接口文档给出的请求参数如下： ```json { - "systemCode": "PAY_CENTER", - "version": "2026.04.16.01", - "profiles": [ + "airportId": "test", + "appName": "XXXx", + "configVersion": "R_XXX_V3.0.35.6.1_XXX", + "fileName": "配置文件名", + "ackSuc": "id,id", + "ackFail": "id,id" +} +``` + +说明： + +- 由于该接口是 `GET`，当前实现按查询参数方式理解这些字段 +- 当前代码只使用 `airportId`、`appName` 做基础过滤 +- `ackSuc`、`ackFail` 暂未纳入当前实现 +- 这部分如果后续要完全对齐，可继续补充 + +### 4.3 响应体 + +```json +{ + "code": "0", + "data": [ { - "name": "default", - "items": [ - { - "key": "feature.switch.payment", - "value": "true" - }, - { - "key": "payment.timeout.seconds", - "value": "30" - } - ] + "id": "1", + "airportId": "test", + "appName": "XXXx", + "configVersion": "R_XXX_V3.0.35.6.1_XXX", + "configContent": "配置内容（加密）", + "fileName": "配置文件名" } - ] + ], + "msg": "ok" } ``` -### 8.5 成功响应头示例 +字段说明： -```http -HTTP/1.1 200 OK -Content-Type: application/json;charset=UTF-8 -X-Config-Version: 2026.04.16.01 -X-Config-Hash: 3d96f8d4c6d6d7cfcb1dc3d2c335ad426376cf2f774ab5b5567f7f8e9b30b5d1 -``` +| 字段 | 说明 | +| --- | --- | +| `id` | 配置记录标识 | +| `airportId` | 机场编码 | +| `appName` | 应用名称 | +| `configVersion` | 配置版本号 | +| `configContent` | 配置内容 | +| `fileName` | 配置文件名 | -### 8.6 无配置场景示例 +### 4.4 业务说明 + +- 不带请求参数时，返回所有“未同步到 Git 且已审核通过”的配置 +- `configContent` 为加密内容 +- **解密方式当前留为 `TODO`** + +## 5. 接口三：获取 token + +### 5.1 基本信息 + +- 地址：`http://ip:port/pic_bus_manage_monitor/pam-monitor/login` +- 方法：`POST` +- 内容格式：`JSON` +- 说明：该接口已存在，用于获取配置同步接口所需 token + +### 5.2 请求体 ```json { - "code": "CONFIG_NOT_FOUND", - "message": "No active production configuration found", - "traceId": "server-generated-trace-id", - "timestamp": "2026-04-16T10:30:00+08:00", - "data": {} + "name": "XXXxx", + "password": "" } ``` -### 8.7 HTTP 状态码建议 +### 5.3 响应体 -| HTTP 状态码 | 场景 | +```json +{ + "code": "0", + "data": { + "token": "tetttwe", + "expireTime": "2026-07-11 11:11:11" + }, + "msg": "ok" +} +``` + +字段说明： + +| 字段 | 说明 | | --- | --- | -| `200` | 成功返回配置快照 | -| `204` | 无可导出配置 | -| `401` | Token 无效 | -| `403` | 无权限 | -| `500` | 服务端异常 | +| `token` | 鉴权 token | +| `expireTime` | token 过期时间 | -说明： +### 5.4 业务说明 -- 当前同步工具对 `204` 尚未做专门分支处理，如果生产端预计长期存在“无配置”场景，建议工具侧后续补充兼容逻辑。 +- 如果 token 过期，需要重新调用登录接口获取新 token -## 9. 幂等规则 +## 6. 当前代码实现对齐情况 -### 9.1 push 幂等建议 - -生产端建议以下任一方式作为幂等判断条件： - -1. `traceId` -2. `sourceVersion + contentHash` - -建议优先： - -- `traceId` - -补充校验： - -- 如果 `traceId` 相同但 `contentHash` 不同，应返回冲突错误，不允许覆盖 - -## 10. 审计建议 - -生产端建议记录以下审计信息： - -- `traceId` -- 调用时间 -- 调用方 IP -- 来源版本号 -- 内容哈希 -- 导入结果 -- 失败原因 - -## 11. 联调建议 - -联调时建议优先确认以下内容： - -1. `Authorization` 认证是否打通 -2. `push` 是否按 `multipart/form-data` 接收 -3. `pull` 是否返回原始 JSON 字节流 -4. `X-Config-Version` 是否稳定返回 -5. 重复请求是否能幂等处理 - -## 12. 与当前同步工具实现的对应关系 - -当前同步工具中的生产接口调用实现文件为： +当前代码实现文件： - [ProdConfigApiService.java](e:/AIcoding/FtpTool/src/main/java/com/ftptool/sync/service/ProdConfigApiService.java) -目前工具侧已按以下假设实现： +当前实现已经对齐到以下协议： -- `push` 使用 `POST + multipart/form-data` -- `pull` 使用 `GET` -- `pull` 成功后将响应体直接保存为本地文件 `prod-config.json` -- `pull` 优先从 `X-Config-Version` 或 `ETag` 中提取版本号 +- `pushConfig` 使用 `POST + JSON 数组` +- `pullConfig` 使用 `GET + JSON 响应` +- `login` 使用 `POST + JSON` +- 请求头默认使用 `token` 作为 token 头名称 +- `token` 未静态配置时，会自动调用登录接口获取并缓存 -因此本接口文档与当前工具实现是兼容的。 +当前仍保留的 `TODO`： -## 13. 后续可扩展项 +- `configContent` 推送前加密 +- `configContent` 拉取后解密 +- `pullConfig` 中 `ackSuc/ackFail` 参数的完整处理 -后续如需增强，可在 V2 中考虑： +## 7. 当前配置项 -- `push` 增加签名校验 -- `pull` 支持增量拉取 -- `pull` 增加查询参数，如 `systemCode`、`profile`、`version` -- `push` 增加灰度导入、预校验模式 -- 增加 `/api/config/validate` 预校验接口 +为配合上述接口，当前配置项建议如下： + +```properties +prod.api.base-url=https://prod.example.com +prod.api.push-path=/pic_bus_manage_monitor/configSync/pushConfig +prod.api.pull-path=/pic_bus_manage_monitor/configSync/pullConfig +prod.api.login-path=/pic_bus_manage_monitor/pam-monitor/login +prod.api.token=replace-me +prod.api.token-header-name=token +prod.api.airport-id=replace-me +prod.api.app-name=replace-me +prod.api.login-name= +prod.api.login-password= +``` + +说明： + +- 如果 `prod.api.token` 已直接配置，则当前实现优先使用静态 token +- 如果没有配置 token，则走 `login` 接口获取 token + +## 8. 联调建议 + +联调时建议优先确认以下几点： + +1. `token` 请求头名称是否就是 `token` +2. `pullConfig` 的 GET 请求参数是否确实按 query string 传递 +3. `ackFail` 非空时是否要整体视为失败 +4. `configContent` 加密/解密算法何时补齐 +5. `fileName` 是否可能包含目录层级 diff --git a/src/main/java/com/ftptool/sync/config/ProdApiProperties.java b/src/main/java/com/ftptool/sync/config/ProdApiProperties.java index b04b142..4146158 100644 --- a/src/main/java/com/ftptool/sync/config/ProdApiProperties.java +++ b/src/main/java/com/ftptool/sync/config/ProdApiProperties.java @@ -14,8 +14,20 @@ public class ProdApiProperties { private String pushPath; /** 生产快照导出接口路径。 */ private String pullPath; + /** 获取 token 的登录接口路径。 */ + private String loginPath; /** 访问生产接口的 Bearer Token。 */ private String token; + /** 放置 token 的请求头名称，默认按对方接口约定使用 token。 */ + private String tokenHeaderName = "token"; + /** 业务归属机场编码。 */ + private String airportId; + /** 业务应用名称。 */ + private String appName; + /** 登录接口用户名。 */ + private String loginName; + /** 登录接口密码。 */ + private String loginPassword; /** HTTP 连接超时。 */ private int connectTimeoutMs = 10000; /** HTTP 读取超时。 */ @@ -45,6 +57,14 @@ public class ProdApiProperties { this.pullPath = pullPath; } + public String getLoginPath() { + return loginPath; + } + + public void setLoginPath(String loginPath) { + this.loginPath = loginPath; + } + public String getToken() { return token; } @@ -53,6 +73,46 @@ public class ProdApiProperties { this.token = token; } + public String getTokenHeaderName() { + return tokenHeaderName; + } + + public void setTokenHeaderName(String tokenHeaderName) { + this.tokenHeaderName = tokenHeaderName; + } + + public String getAirportId() { + return airportId; + } + + public void setAirportId(String airportId) { + this.airportId = airportId; + } + + public String getAppName() { + return appName; + } + + public void setAppName(String appName) { + this.appName = appName; + } + + public String getLoginName() { + return loginName; + } + + public void setLoginName(String loginName) { + this.loginName = loginName; + } + + public String getLoginPassword() { + return loginPassword; + } + + public void setLoginPassword(String loginPassword) { + this.loginPassword = loginPassword; + } + public int getConnectTimeoutMs() { return connectTimeoutMs; } diff --git a/src/main/java/com/ftptool/sync/entity/ProdPullAckRecord.java b/src/main/java/com/ftptool/sync/entity/ProdPullAckRecord.java new file mode 100644 index 0000000..21bc9ea --- /dev/null +++ b/src/main/java/com/ftptool/sync/entity/ProdPullAckRecord.java @@ -0,0 +1,107 @@ +package com.ftptool.sync.entity; + +import com.ftptool.sync.model.ProdPullAckStatus; + +import javax.persistence.Column; +import javax.persistence.Entity; +import javax.persistence.EnumType; +import javax.persistence.Enumerated; +import javax.persistence.GeneratedValue; +import javax.persistence.GenerationType; +import javax.persistence.Id; +import javax.persistence.PrePersist; +import javax.persistence.PreUpdate; +import javax.persistence.Table; +import javax.persistence.UniqueConstraint; +import java.time.LocalDateTime; + +/** + * pullConfig 回执记录。 + * 用于记录需要在下一次 pull 请求里回传给生产端的 ackSuc/ackFail。 + */ +@Entity +@Table(name = "prod_pull_ack", uniqueConstraints = { + @UniqueConstraint(name = "uk_prod_pull_ack_remote_id", columnNames = "remote_config_id") +}) +public class ProdPullAckRecord { + + @Id + @GeneratedValue(strategy = GenerationType.IDENTITY) + private Long id; + + /** 生产端返回的配置项 id。 */ + @Column(name = "remote_config_id", nullable = false, length = 128) + private String remoteConfigId; + + /** 回执状态。 */ + @Enumerated(EnumType.STRING) + @Column(name = "ack_status", nullable = false, length = 16) + private ProdPullAckStatus ackStatus; + + /** 是否已经在 pullConfig 请求里成功回传给生产端。 */ + @Column(name = "reported", nullable = false) + private Boolean reported; + + /** 创建时间。 */ + @Column(name = "created_at", nullable = false) + private LocalDateTime createdAt; + + /** 更新时间。 */ + @Column(name = "updated_at", nullable = false) + private LocalDateTime updatedAt; + + @PrePersist + public void prePersist() { + LocalDateTime now = LocalDateTime.now(); + this.createdAt = now; + this.updatedAt = now; + if (this.reported == null) { + this.reported = Boolean.FALSE; + } + } + + @PreUpdate + public void preUpdate() { + this.updatedAt = LocalDateTime.now(); + } + + public Long getId() { + return id; + } + + public void setId(Long id) { + this.id = id; + } + + public String getRemoteConfigId() { + return remoteConfigId; + } + + public void setRemoteConfigId(String remoteConfigId) { + this.remoteConfigId = remoteConfigId; + } + + public ProdPullAckStatus getAckStatus() { + return ackStatus; + } + + public void setAckStatus(ProdPullAckStatus ackStatus) { + this.ackStatus = ackStatus; + } + + public Boolean getReported() { + return reported; + } + + public void setReported(Boolean reported) { + this.reported = reported; + } + + public LocalDateTime getCreatedAt() { + return createdAt; + } + + public LocalDateTime getUpdatedAt() { + return updatedAt; + } +} diff --git a/src/main/java/com/ftptool/sync/model/ProdPullAckStatus.java b/src/main/java/com/ftptool/sync/model/ProdPullAckStatus.java new file mode 100644 index 0000000..5582400 --- /dev/null +++ b/src/main/java/com/ftptool/sync/model/ProdPullAckStatus.java @@ -0,0 +1,11 @@ +package com.ftptool.sync.model; + +/** + * pullConfig 回执状态。 + */ +public enum ProdPullAckStatus { + /** 上一批拉取配置已成功回写到 Git。 */ + SUCCESS, + /** 上一批拉取配置在本地处理失败。 */ + FAILED +} diff --git a/src/main/java/com/ftptool/sync/model/ProdPullResult.java b/src/main/java/com/ftptool/sync/model/ProdPullResult.java index 987a78d..cf15caf 100644 --- a/src/main/java/com/ftptool/sync/model/ProdPullResult.java +++ b/src/main/java/com/ftptool/sync/model/ProdPullResult.java @@ -1,6 +1,7 @@ package com.ftptool.sync.model; import java.nio.file.Path; +import java.util.List; /** * 生产 pull 接口返回结果在本地落盘后的封装对象。 @@ -13,11 +14,14 @@ public class ProdPullResult { private final String sourceVersion; /** 响应体内容哈希。 */ private final String contentHash; + /** 本次 pull 返回的生产配置项 id 列表。 */ + private final List pulledConfigIds; - public ProdPullResult(Path contentDirectory, String sourceVersion, String contentHash) { + public ProdPullResult(Path contentDirectory, String sourceVersion, String contentHash, List pulledConfigIds) { this.contentDirectory = contentDirectory; this.sourceVersion = sourceVersion; this.contentHash = contentHash; + this.pulledConfigIds = pulledConfigIds; } public Path getContentDirectory() { @@ -31,4 +35,8 @@ public class ProdPullResult { public String getContentHash() { return contentHash; } + + public List getPulledConfigIds() { + return pulledConfigIds; + } } diff --git a/src/main/java/com/ftptool/sync/orchestrator/ProdSyncCoordinator.java b/src/main/java/com/ftptool/sync/orchestrator/ProdSyncCoordinator.java index 450806b..58c1470 100644 --- a/src/main/java/com/ftptool/sync/orchestrator/ProdSyncCoordinator.java +++ b/src/main/java/com/ftptool/sync/orchestrator/ProdSyncCoordinator.java @@ -4,14 +4,15 @@ import com.ftptool.sync.config.GitRepoProperties; import com.ftptool.sync.config.ProdApiProperties; import com.ftptool.sync.config.SyncProperties; import com.ftptool.sync.entity.SyncTask; -import com.ftptool.sync.model.PackageBuildResult; import com.ftptool.sync.model.PackageManifest; +import com.ftptool.sync.model.ProdPullAckStatus; import com.ftptool.sync.model.ProdPullResult; import com.ftptool.sync.model.SyncDirection; import com.ftptool.sync.model.SyncStatus; import com.ftptool.sync.service.CheckpointService; import com.ftptool.sync.service.GitClientService; import com.ftptool.sync.service.PackageService; +import com.ftptool.sync.service.ProdPullAckService; import com.ftptool.sync.service.ProdConfigApiService; import com.ftptool.sync.service.SyncMetadataService; import com.ftptool.sync.service.SyncTaskService; @@ -21,17 +22,29 @@ import org.slf4j.LoggerFactory; import org.springframework.context.annotation.Profile; import org.springframework.stereotype.Service; +import java.io.IOException; +import java.nio.file.Files; import java.nio.file.Path; +import java.util.ArrayList; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; import java.util.Optional; +import java.util.Set; +import java.util.stream.Collectors; +import java.util.stream.Stream; + +import com.ftptool.sync.util.FileHashUtils; +import com.ftptool.sync.util.FileTreeUtils; -@Service -@Profile("prod-agent") /** * 生产侧同步协调器。 * 串联两条核心链路： * 1. Git -> PROD * 2. PROD -> Git */ +@Service +@Profile("prod-agent") public class ProdSyncCoordinator { private static final Logger log = LoggerFactory.getLogger(ProdSyncCoordinator.class); @@ -43,6 +56,7 @@ public class ProdSyncCoordinator { private final GitClientService gitClientService; private final PackageService packageService; private final ProdConfigApiService prodConfigApiService; + private final ProdPullAckService prodPullAckService; private final SyncTaskService syncTaskService; private final CheckpointService checkpointService; private final SyncMetadataService syncMetadataService; @@ -55,6 +69,7 @@ public class ProdSyncCoordinator { GitClientService gitClientService, PackageService packageService, ProdConfigApiService prodConfigApiService, + ProdPullAckService prodPullAckService, SyncTaskService syncTaskService, CheckpointService checkpointService, SyncMetadataService syncMetadataService @@ -66,6 +81,7 @@ public class ProdSyncCoordinator { this.gitClientService = gitClientService; this.packageService = packageService; this.prodConfigApiService = prodConfigApiService; + this.prodPullAckService = prodPullAckService; this.syncTaskService = syncTaskService; this.checkpointService = checkpointService; this.syncMetadataService = syncMetadataService; @@ -83,9 +99,11 @@ public class ProdSyncCoordinator { gitRepoProperties.getScanBranch(), prodApiProperties.getPushPath() ); + String branch = gitRepoProperties.getScanBranch(); String sourceVersion = gitClientService.prepareRepositoryAndGetHead(branch); Path exportDirectory = workDirectoryService.getDevToProdStagingDir().resolve("git-" + sourceVersion); + // 先导出 Git 工作树快照，再计算内容哈希，避免直接拿工作目录参与后续修改。 gitClientService.exportBranchSnapshot(branch, exportDirectory); String contentHash = packageService.calculateDirectoryHash(exportDirectory); @@ -112,27 +130,33 @@ public class ProdSyncCoordinator { manifest.setPackageName(existing.get().getPackageName()); } - PackageBuildResult packageBuildResult = packageService.buildPackageFromDirectory(exportDirectory, manifest); SyncTask task = syncTaskService.createOrLoadTask( SyncDirection.DEV_TO_PROD, sourceVersion, - packageBuildResult.getContentHash(), - packageBuildResult.getPackageName(), + contentHash, + manifest.getPackageName(), traceId ); + // Git 提交哈希 + 内容哈希作为业务幂等键，避免同一版本重复推送到生产。 syncTaskService.markStatus(task.getTraceId(), SyncStatus.CONSUMING, null); - prodConfigApiService.pushPackage(manifest, packageBuildResult.getZipFile()); + Path pushDirectory = preparePushDirectory(exportDirectory, sourceVersion); + prodConfigApiService.pushPackage(manifest, pushDirectory); syncTaskService.markStatus(task.getTraceId(), SyncStatus.SUCCESS, null); checkpointService.saveCheckpoint(task.getDirection(), task.getSourceVersion(), task.getContentHash()); + refreshGitToProdBaseline(exportDirectory); log.info("Git version pushed to prod successfully. traceId={}, version={}", task.getTraceId(), task.getSourceVersion()); } catch (Exception e) { handleFailure(traceId, "PROD git->prod sync failed", e); } } + /** + * 主链路二：从生产拉取当前配置并回写到 Git 快照分支。 + */ public void syncProdSnapshotToGit() { String traceId = null; + ProdPullResult pullResult = null; try { log.info( "PROD prod->git tick. apiBaseUrl={}, pullPath={}, snapshotBranch={}", @@ -140,7 +164,8 @@ public class ProdSyncCoordinator { prodApiProperties.getPullPath(), gitRepoProperties.getSnapshotBranch() ); - ProdPullResult pullResult = prodConfigApiService.pullConfigSnapshot(); + + pullResult = prodConfigApiService.pullConfigSnapshot(); Optional existing = syncTaskService.findByBusinessKey( SyncDirection.PROD_TO_DEV, pullResult.getSourceVersion(), @@ -174,6 +199,7 @@ public class ProdSyncCoordinator { syncTaskService.markStatus(task.getTraceId(), SyncStatus.SUCCESS, null); checkpointService.saveCheckpoint(task.getDirection(), task.getSourceVersion(), task.getContentHash()); + prodPullAckService.recordAckResult(pullResult.getPulledConfigIds(), ProdPullAckStatus.SUCCESS); log.info( "Production snapshot synced to Git. traceId={}, version={}, gitPushed={}", task.getTraceId(), @@ -181,14 +207,94 @@ public class ProdSyncCoordinator { pushed ); } catch (Exception e) { + if (pullResult != null) { + prodPullAckService.recordAckResult(pullResult.getPulledConfigIds(), ProdPullAckStatus.FAILED); + } handleFailure(traceId, "PROD prod->git sync failed", e); } } + /** + * 已成功的同版本任务直接跳过，避免重复同步。 + */ private boolean shouldSkip(Optional existing) { return existing.isPresent() && existing.get().getStatus() == SyncStatus.SUCCESS; } + /** + * 生成本次推送目录： + * 1. 首次推送走全量 + * 2. 检测到删除时回退全量 + * 3. 其余场景仅推送变更文件 + */ + private Path preparePushDirectory(Path exportDirectory, String sourceVersion) throws IOException { + Path baselineDirectory = workDirectoryService.getGitToProdBaselineDir(); + if (Files.notExists(baselineDirectory) || isDirectoryEmpty(baselineDirectory)) { + return exportDirectory; + } + + Map currentFileHashes = collectFileHashes(exportDirectory); + Map baselineFileHashes = collectFileHashes(baselineDirectory); + + Set deletedFiles = baselineFileHashes.keySet().stream() + .filter(path -> !currentFileHashes.containsKey(path)) + .collect(Collectors.toSet()); + if (!deletedFiles.isEmpty()) { + log.info("Git->PROD 检测到文件删除，回退为全量推送。deletedCount={}", deletedFiles.size()); + return exportDirectory; + } + + List changedFiles = new ArrayList(); + for (Map.Entry entry : currentFileHashes.entrySet()) { + String baselineHash = baselineFileHashes.get(entry.getKey()); + if (baselineHash == null || !baselineHash.equals(entry.getValue())) { + changedFiles.add(entry.getKey()); + } + } + + if (changedFiles.isEmpty() || changedFiles.size() == currentFileHashes.size()) { + return exportDirectory; + } + + Path incrementalDirectory = workDirectoryService.getDevToProdStagingDir().resolve("git-delta-" + sourceVersion); + FileTreeUtils.deleteRecursively(incrementalDirectory); + FileTreeUtils.ensureDirectory(incrementalDirectory); + FileTreeUtils.copySelectedFiles(exportDirectory, incrementalDirectory, changedFiles); + log.info("Git->PROD 本次采用最小增量推送。changedCount={}", changedFiles.size()); + return incrementalDirectory; + } + + /** + * 成功推送后刷新本地基线目录，作为下一次增量比较的依据。 + */ + private void refreshGitToProdBaseline(Path exportDirectory) throws IOException { + Path baselineDirectory = workDirectoryService.getGitToProdBaselineDir(); + FileTreeUtils.ensureDirectory(baselineDirectory); + try (Stream stream = Files.list(baselineDirectory)) { + for (Path child : stream.collect(Collectors.toList())) { + FileTreeUtils.deleteRecursively(child); + } + } + FileTreeUtils.copyDirectory(exportDirectory, baselineDirectory); + } + + private boolean isDirectoryEmpty(Path directory) throws IOException { + try (Stream stream = Files.list(directory)) { + return !stream.findFirst().isPresent(); + } + } + + private Map collectFileHashes(Path rootDirectory) throws IOException { + Map fileHashes = new LinkedHashMap(); + try (Stream stream = Files.walk(rootDirectory)) { + List files = stream.filter(Files::isRegularFile).sorted().collect(Collectors.toList()); + for (Path file : files) { + fileHashes.put(rootDirectory.relativize(file), FileHashUtils.sha256(file)); + } + } + return fileHashes; + } + /** * 统一失败处理逻辑。 */ @@ -197,6 +303,7 @@ public class ProdSyncCoordinator { if (traceId == null) { return; } + // 只有达到最大重试次数后才把任务标记为失败，之前保留为可重试状态。 syncTaskService.increaseRetryCount(traceId, summarizeException(e)); Optional task = syncTaskService.findByTraceId(traceId); diff --git a/src/main/java/com/ftptool/sync/repository/ProdPullAckRecordRepository.java b/src/main/java/com/ftptool/sync/repository/ProdPullAckRecordRepository.java new file mode 100644 index 0000000..6ffd48e --- /dev/null +++ b/src/main/java/com/ftptool/sync/repository/ProdPullAckRecordRepository.java @@ -0,0 +1,20 @@ +package com.ftptool.sync.repository; + +import com.ftptool.sync.entity.ProdPullAckRecord; +import com.ftptool.sync.model.ProdPullAckStatus; +import org.springframework.data.jpa.repository.JpaRepository; + +import java.util.List; +import java.util.Optional; + +/** + * pullConfig 回执记录仓储。 + */ +public interface ProdPullAckRecordRepository extends JpaRepository { + + Optional findByRemoteConfigId(String remoteConfigId); + + List findByReportedFalseOrderByUpdatedAtAsc(); + + List findByAckStatusAndReportedFalseOrderByUpdatedAtAsc(ProdPullAckStatus ackStatus); +} diff --git a/src/main/java/com/ftptool/sync/service/ProdConfigApiService.java b/src/main/java/com/ftptool/sync/service/ProdConfigApiService.java index 169e902..a25267f 100644 --- a/src/main/java/com/ftptool/sync/service/ProdConfigApiService.java +++ b/src/main/java/com/ftptool/sync/service/ProdConfigApiService.java @@ -3,117 +3,335 @@ package com.ftptool.sync.service; import com.ftptool.sync.config.ProdApiProperties; import com.ftptool.sync.config.SyncProperties; import com.ftptool.sync.model.PackageManifest; +import com.ftptool.sync.model.ProdPullAckStatus; import com.ftptool.sync.model.ProdPullResult; import com.ftptool.sync.util.FileHashUtils; import com.ftptool.sync.util.FileTreeUtils; import org.slf4j.Logger; import org.slf4j.LoggerFactory; -import org.springframework.core.io.FileSystemResource; +import org.springframework.core.ParameterizedTypeReference; import org.springframework.http.HttpEntity; import org.springframework.http.HttpHeaders; import org.springframework.http.HttpMethod; import org.springframework.http.MediaType; import org.springframework.http.ResponseEntity; import org.springframework.stereotype.Service; -import org.springframework.util.LinkedMultiValueMap; -import org.springframework.util.MultiValueMap; +import org.springframework.util.StringUtils; import org.springframework.web.client.RestTemplate; +import org.springframework.web.util.UriComponentsBuilder; import java.io.IOException; +import java.nio.charset.StandardCharsets; import java.nio.file.Files; import java.nio.file.Path; +import java.time.LocalDateTime; +import java.time.format.DateTimeFormatter; +import java.util.ArrayList; +import java.util.LinkedHashSet; +import java.util.List; +import java.util.Set; +import java.util.stream.Collectors; +import java.util.stream.Stream; -@Service /** * 生产接口访问服务。 - * 封装对生产 push/pull 接口的 HTTP 调用。 + * 封装对生产 pushConfig / pullConfig / login 接口的 HTTP 调用。 */ +@Service public class ProdConfigApiService { private static final Logger log = LoggerFactory.getLogger(ProdConfigApiService.class); + private static final String SUCCESS_CODE = "0"; + private static final DateTimeFormatter EXPIRE_TIME_FORMATTER = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss"); private final ProdApiProperties prodApiProperties; private final SyncProperties syncProperties; private final RestTemplate restTemplate; private final WorkDirectoryService workDirectoryService; + private final ProdPullAckService prodPullAckService; + + private volatile String cachedToken; + private volatile LocalDateTime cachedTokenExpireTime; public ProdConfigApiService( ProdApiProperties prodApiProperties, SyncProperties syncProperties, RestTemplate restTemplate, - WorkDirectoryService workDirectoryService + WorkDirectoryService workDirectoryService, + ProdPullAckService prodPullAckService ) { this.prodApiProperties = prodApiProperties; this.syncProperties = syncProperties; this.restTemplate = restTemplate; this.workDirectoryService = workDirectoryService; + this.prodPullAckService = prodPullAckService; } /** - * 调用生产 push 接口，导入一个标准同步包。 + * 调用 pushConfig 接口，把目录中的配置内容按文件维度推送到生产。 */ - public void pushPackage(PackageManifest manifest, Path zipFile) { + public void pushPackage(PackageManifest manifest, Path sourceDirectory) throws IOException { String url = buildUrl(prodApiProperties.getPushPath()); - HttpHeaders headers = defaultHeaders(); - headers.setContentType(MediaType.MULTIPART_FORM_DATA); + HttpHeaders headers = defaultJsonHeaders(); - // 当前协议约定 push 使用 multipart/form-data 上传标准同步包。 - MultiValueMap body = new LinkedMultiValueMap(); - body.add("file", new FileSystemResource(zipFile.toFile())); - body.add("traceId", manifest.getTraceId()); - body.add("direction", manifest.getDirection().name()); - body.add("sourceVersion", manifest.getSourceVersion()); - body.add("contentHash", manifest.getContentHash()); + List requestBody = buildPushRequest(manifest, sourceDirectory); + ResponseEntity> response = restTemplate.exchange( + url, + HttpMethod.POST, + new HttpEntity>(requestBody, headers), + new ParameterizedTypeReference>() { + } + ); - ResponseEntity response = restTemplate.postForEntity(url, new HttpEntity>(body, headers), String.class); - if (!response.getStatusCode().is2xxSuccessful()) { - throw new IllegalStateException("Prod push API failed with status " + response.getStatusCodeValue()); + ProdApiResponse body = response.getBody(); + validateSuccess(body, "生产 pushConfig 接口调用失败"); + + List ackFail = body.getData() == null ? null : body.getData().getAckFail(); + if (ackFail != null && !ackFail.isEmpty()) { + String failedFiles = ackFail.stream() + .map(ProdPushAckItem::getFileName) + .collect(Collectors.joining(",")); + throw new IllegalStateException("生产 pushConfig 返回部分失败，失败文件：" + failedFiles); } - log.info("Prod push API finished. traceId={}, status={}", manifest.getTraceId(), response.getStatusCodeValue()); + + log.info("Prod pushConfig finished. traceId={}, itemCount={}", manifest.getTraceId(), requestBody.size()); } + /** + * 调用 pullConfig 接口，拉取生产快照并按文件恢复到本地目录。 + */ public ProdPullResult pullConfigSnapshot() throws IOException { String url = buildUrl(prodApiProperties.getPullPath()); - HttpHeaders headers = defaultHeaders(); - // 当前协议约定 pull 直接返回原始配置字节流，由同步工具落成本地文件后再回写 Git。 - ResponseEntity response = restTemplate.exchange( - url, + HttpHeaders headers = defaultJsonHeaders(); + + UriComponentsBuilder builder = UriComponentsBuilder.fromHttpUrl(url); + ProdPullAckService.PendingAckSummary pendingAckSummary = prodPullAckService.getPendingAckSummary(); + + // 当前先按应用维度过滤。 + if (StringUtils.hasText(prodApiProperties.getAirportId()) && !isPlaceholder(prodApiProperties.getAirportId())) { + builder.queryParam("airportId", prodApiProperties.getAirportId()); + } + if (StringUtils.hasText(prodApiProperties.getAppName()) && !isPlaceholder(prodApiProperties.getAppName())) { + builder.queryParam("appName", prodApiProperties.getAppName()); + } + if (pendingAckSummary.getAckSucIds() != null && !pendingAckSummary.getAckSucIds().isEmpty()) { + builder.queryParam("ackSuc", StringUtils.collectionToCommaDelimitedString(pendingAckSummary.getAckSucIds())); + } + if (pendingAckSummary.getAckFailIds() != null && !pendingAckSummary.getAckFailIds().isEmpty()) { + builder.queryParam("ackFail", StringUtils.collectionToCommaDelimitedString(pendingAckSummary.getAckFailIds())); + } + + ResponseEntity>> response = restTemplate.exchange( + builder.build(true).toUri(), HttpMethod.GET, new HttpEntity(headers), - byte[].class + new ParameterizedTypeReference>>() { + } ); - if (!response.getStatusCode().is2xxSuccessful()) { - throw new IllegalStateException("Prod pull API failed with status " + response.getStatusCodeValue()); + + ProdApiResponse> body = response.getBody(); + validateSuccess(body, "生产 pullConfig 接口调用失败"); + if (pendingAckSummary.hasPendingAck()) { + prodPullAckService.markPendingAsReported(); } - byte[] body = response.getBody(); - if (body == null || body.length == 0) { - throw new IllegalStateException("Prod pull API returned empty content"); + List items = body.getData(); + if (items == null || items.isEmpty()) { + throw new IllegalStateException("生产 pullConfig 未返回可同步配置"); } Path tempDir = Files.createTempDirectory(workDirectoryService.getProdToDevStagingDir(), "pull-"); FileTreeUtils.ensureDirectory(tempDir); - Path targetFile = tempDir.resolve(syncProperties.getPullResponseFileName()); - Files.write(targetFile, body); + for (ProdPulledConfigItem item : items) { + writePulledConfigItem(tempDir, item); + } - String contentHash = FileHashUtils.sha256(body); - // 优先取服务端显式版本号；如果服务端没给，就退化为内容哈希做幂等判断。 - String sourceVersion = firstNonBlank( - response.getHeaders().getFirst("X-Config-Version"), - response.getHeaders().getETag(), - contentHash - ); - return new ProdPullResult(tempDir, sourceVersion, contentHash); + String contentHash = FileHashUtils.sha256Directory(tempDir); + String sourceVersion = resolvePullVersion(items, contentHash); + return new ProdPullResult(tempDir, sourceVersion, contentHash, collectPulledIds(items)); } - private HttpHeaders defaultHeaders() { + /** + * 统一构造 JSON 调用头，并补齐 token 鉴权。 + */ + private HttpHeaders defaultJsonHeaders() { HttpHeaders headers = new HttpHeaders(); headers.setAccept(java.util.Collections.singletonList(MediaType.APPLICATION_JSON)); - if (prodApiProperties.getToken() != null && !prodApiProperties.getToken().trim().isEmpty()) { - headers.setBearerAuth(prodApiProperties.getToken().trim()); + headers.setContentType(MediaType.APPLICATION_JSON); + + String token = resolveToken(); + if (StringUtils.hasText(prodApiProperties.getTokenHeaderName())) { + headers.set(prodApiProperties.getTokenHeaderName(), token); + } else { + headers.set("token", token); } return headers; } + /** + * 优先使用静态 token；如果未配置，则走登录接口获取并缓存。 + */ + private String resolveToken() { + if (StringUtils.hasText(prodApiProperties.getToken()) && !isPlaceholder(prodApiProperties.getToken())) { + return prodApiProperties.getToken().trim(); + } + return loginAndGetToken(); + } + + /** + * 调用登录接口获取 token，并根据过期时间做简单缓存。 + */ + private synchronized String loginAndGetToken() { + if (StringUtils.hasText(cachedToken) + && cachedTokenExpireTime != null + && cachedTokenExpireTime.isAfter(LocalDateTime.now().plusSeconds(60))) { + return cachedToken; + } + + validateLoginConfig(); + String url = buildUrl(prodApiProperties.getLoginPath()); + HttpHeaders headers = new HttpHeaders(); + headers.setAccept(java.util.Collections.singletonList(MediaType.APPLICATION_JSON)); + headers.setContentType(MediaType.APPLICATION_JSON); + + ProdLoginRequest requestBody = new ProdLoginRequest(); + requestBody.setName(prodApiProperties.getLoginName()); + requestBody.setPassword(prodApiProperties.getLoginPassword()); + + ResponseEntity> response = restTemplate.exchange( + url, + HttpMethod.POST, + new HttpEntity(requestBody, headers), + new ParameterizedTypeReference>() { + } + ); + + ProdApiResponse body = response.getBody(); + validateSuccess(body, "生产登录接口调用失败"); + if (body.getData() == null || !StringUtils.hasText(body.getData().getToken())) { + throw new IllegalStateException("生产登录接口未返回有效 token"); + } + + cachedToken = body.getData().getToken().trim(); + if (StringUtils.hasText(body.getData().getExpireTime())) { + cachedTokenExpireTime = LocalDateTime.parse(body.getData().getExpireTime(), EXPIRE_TIME_FORMATTER); + } else { + cachedTokenExpireTime = LocalDateTime.now().plusMinutes(30); + } + return cachedToken; + } + + /** + * 把本地目录转换成 pushConfig 需要的 JSON 数组。 + */ + private List buildPushRequest(PackageManifest manifest, Path sourceDirectory) throws IOException { + validateBusinessConfig(); + + List result = new ArrayList(); + try (Stream stream = Files.walk(sourceDirectory)) { + List files = stream + .filter(Files::isRegularFile) + .sorted() + .collect(Collectors.toList()); + for (Path file : files) { + ProdPushConfigItem item = new ProdPushConfigItem(); + item.setAirportId(prodApiProperties.getAirportId()); + item.setAppName(prodApiProperties.getAppName()); + item.setConfigVersion(manifest.getSourceVersion()); + // TODO: 配置内容需按生产接口约定加密后再发送。 + item.setConfigContent(new String(Files.readAllBytes(file), StandardCharsets.UTF_8)); + item.setFileName(sourceDirectory.relativize(file).toString().replace('\\', '/')); + result.add(item); + } + } + + if (result.isEmpty()) { + throw new IllegalStateException("待推送目录为空，无法构造 pushConfig 请求"); + } + return result; + } + + /** + * 将 pullConfig 返回的单条配置恢复为本地文件。 + */ + private void writePulledConfigItem(Path baseDirectory, ProdPulledConfigItem item) throws IOException { + String fileName = StringUtils.hasText(item.getFileName()) ? item.getFileName() : syncProperties.getPullResponseFileName(); + Path targetFile = baseDirectory.resolve(fileName).normalize(); + if (!targetFile.startsWith(baseDirectory)) { + throw new IOException("pullConfig 返回的 fileName 非法：" + fileName); + } + Files.createDirectories(targetFile.getParent()); + // TODO: configContent 当前直接按返回值落盘，后续需补充解密逻辑。 + Files.write(targetFile, safeString(item.getConfigContent()).getBytes(StandardCharsets.UTF_8)); + } + + /** + * 解析本次 pull 结果的来源版本。 + */ + private String resolvePullVersion(List items, String contentHash) { + Set versions = new LinkedHashSet(); + for (ProdPulledConfigItem item : items) { + if (StringUtils.hasText(item.getConfigVersion())) { + versions.add(item.getConfigVersion().trim()); + } + } + if (versions.size() == 1) { + return versions.iterator().next(); + } + return contentHash; + } + + /** + * 提取 pullConfig 返回项中的 id，供下一次请求回传 ackSuc/ackFail。 + */ + private List collectPulledIds(List items) { + List ids = new ArrayList(); + for (ProdPulledConfigItem item : items) { + if (StringUtils.hasText(item.getId())) { + ids.add(item.getId().trim()); + } + } + return ids; + } + + /** + * 校验接口返回是否成功。 + */ + private void validateSuccess(ProdApiResponse response, String failureMessage) { + if (response == null) { + throw new IllegalStateException(failureMessage + "：响应体为空"); + } + if (!SUCCESS_CODE.equals(response.getCode())) { + throw new IllegalStateException(failureMessage + "：" + safeString(response.getMsg()) + "，code=" + safeString(response.getCode())); + } + } + + /** + * 校验 push/pull 业务维度配置是否已提供。 + */ + private void validateBusinessConfig() { + if (!StringUtils.hasText(prodApiProperties.getAirportId()) || isPlaceholder(prodApiProperties.getAirportId())) { + throw new IllegalStateException("未配置 prod.api.airport-id"); + } + if (!StringUtils.hasText(prodApiProperties.getAppName()) || isPlaceholder(prodApiProperties.getAppName())) { + throw new IllegalStateException("未配置 prod.api.app-name"); + } + } + + /** + * 校验登录配置是否已提供。 + */ + private void validateLoginConfig() { + if (!StringUtils.hasText(prodApiProperties.getLoginPath())) { + throw new IllegalStateException("未配置 prod.api.login-path"); + } + if (!StringUtils.hasText(prodApiProperties.getLoginName())) { + throw new IllegalStateException("未配置 prod.api.login-name"); + } + if (!StringUtils.hasText(prodApiProperties.getLoginPassword())) { + throw new IllegalStateException("未配置 prod.api.login-password"); + } + } + /** * 按基础地址和接口路径拼接完整 URL。 */ @@ -128,15 +346,276 @@ public class ProdConfigApiService { return base + path; } + private boolean isPlaceholder(String value) { + return "replace-me".equalsIgnoreCase(safeString(value).trim()); + } + + private String safeString(String value) { + return value == null ? "" : value; + } + /** - * 从多个候选值中选取第一个非空版本号。 + * 通用接口响应包装。 */ - private String firstNonBlank(String... candidates) { - for (String candidate : candidates) { - if (candidate != null && !candidate.trim().isEmpty()) { - return candidate; - } + public static class ProdApiResponse { + + private String code; + private T data; + private String msg; + private String timestamp; + + public String getCode() { + return code; + } + + public void setCode(String code) { + this.code = code; + } + + public T getData() { + return data; + } + + public void setData(T data) { + this.data = data; + } + + public String getMsg() { + return msg; + } + + public void setMsg(String msg) { + this.msg = msg; + } + + public String getTimestamp() { + return timestamp; + } + + public void setTimestamp(String timestamp) { + this.timestamp = timestamp; + } + } + + /** + * pushConfig 请求体项。 + */ + public static class ProdPushConfigItem { + + private String airportId; + private String appName; + private String configVersion; + private String configContent; + private String fileName; + + public String getAirportId() { + return airportId; + } + + public void setAirportId(String airportId) { + this.airportId = airportId; + } + + public String getAppName() { + return appName; + } + + public void setAppName(String appName) { + this.appName = appName; + } + + public String getConfigVersion() { + return configVersion; + } + + public void setConfigVersion(String configVersion) { + this.configVersion = configVersion; + } + + public String getConfigContent() { + return configContent; + } + + public void setConfigContent(String configContent) { + this.configContent = configContent; + } + + public String getFileName() { + return fileName; + } + + public void setFileName(String fileName) { + this.fileName = fileName; + } + } + + /** + * pushConfig 响应 data。 + */ + public static class ProdPushResponseData { + + private List ackFail; + + public List getAckFail() { + return ackFail; + } + + public void setAckFail(List ackFail) { + this.ackFail = ackFail; + } + } + + /** + * pushConfig 失败回执项。 + */ + public static class ProdPushAckItem { + + private String airportId; + private String appName; + private String configVersion; + private String fileName; + + public String getAirportId() { + return airportId; + } + + public void setAirportId(String airportId) { + this.airportId = airportId; + } + + public String getAppName() { + return appName; + } + + public void setAppName(String appName) { + this.appName = appName; + } + + public String getConfigVersion() { + return configVersion; + } + + public void setConfigVersion(String configVersion) { + this.configVersion = configVersion; + } + + public String getFileName() { + return fileName; + } + + public void setFileName(String fileName) { + this.fileName = fileName; + } + } + + /** + * pullConfig 返回的配置项。 + */ + public static class ProdPulledConfigItem { + + private String id; + private String airportId; + private String appName; + private String configVersion; + private String configContent; + private String fileName; + + public String getId() { + return id; + } + + public void setId(String id) { + this.id = id; + } + + public String getAirportId() { + return airportId; + } + + public void setAirportId(String airportId) { + this.airportId = airportId; + } + + public String getAppName() { + return appName; + } + + public void setAppName(String appName) { + this.appName = appName; + } + + public String getConfigVersion() { + return configVersion; + } + + public void setConfigVersion(String configVersion) { + this.configVersion = configVersion; + } + + public String getConfigContent() { + return configContent; + } + + public void setConfigContent(String configContent) { + this.configContent = configContent; + } + + public String getFileName() { + return fileName; + } + + public void setFileName(String fileName) { + this.fileName = fileName; + } + } + + /** + * 登录请求体。 + */ + public static class ProdLoginRequest { + + private String name; + private String password; + + public String getName() { + return name; + } + + public void setName(String name) { + this.name = name; + } + + public String getPassword() { + return password; + } + + public void setPassword(String password) { + this.password = password; + } + } + + /** + * 登录接口返回 data。 + */ + public static class ProdLoginResponseData { + + private String token; + private String expireTime; + + public String getToken() { + return token; + } + + public void setToken(String token) { + this.token = token; + } + + public String getExpireTime() { + return expireTime; + } + + public void setExpireTime(String expireTime) { + this.expireTime = expireTime; } - return null; } } diff --git a/src/main/java/com/ftptool/sync/service/ProdPullAckService.java b/src/main/java/com/ftptool/sync/service/ProdPullAckService.java new file mode 100644 index 0000000..623a325 --- /dev/null +++ b/src/main/java/com/ftptool/sync/service/ProdPullAckService.java @@ -0,0 +1,106 @@ +package com.ftptool.sync.service; + +import com.ftptool.sync.entity.ProdPullAckRecord; +import com.ftptool.sync.model.ProdPullAckStatus; +import com.ftptool.sync.repository.ProdPullAckRecordRepository; +import org.springframework.stereotype.Service; +import org.springframework.transaction.annotation.Transactional; + +import java.util.ArrayList; +import java.util.Collection; +import java.util.List; + +/** + * pullConfig 回执服务。 + */ +@Service +public class ProdPullAckService { + + private final ProdPullAckRecordRepository prodPullAckRecordRepository; + + public ProdPullAckService(ProdPullAckRecordRepository prodPullAckRecordRepository) { + this.prodPullAckRecordRepository = prodPullAckRecordRepository; + } + + /** + * 读取所有尚未回传给生产端的回执状态。 + */ + @Transactional(readOnly = true) + public PendingAckSummary getPendingAckSummary() { + List successIds = toRemoteIds( + prodPullAckRecordRepository.findByAckStatusAndReportedFalseOrderByUpdatedAtAsc(ProdPullAckStatus.SUCCESS) + ); + List failedIds = toRemoteIds( + prodPullAckRecordRepository.findByAckStatusAndReportedFalseOrderByUpdatedAtAsc(ProdPullAckStatus.FAILED) + ); + return new PendingAckSummary(successIds, failedIds); + } + + /** + * 记录一批配置项的处理结果。 + * 若同一 remote id 已存在，则以最新状态覆盖。 + */ + @Transactional + public void recordAckResult(Collection remoteIds, ProdPullAckStatus ackStatus) { + if (remoteIds == null) { + return; + } + for (String remoteId : remoteIds) { + if (remoteId == null || remoteId.trim().isEmpty()) { + continue; + } + ProdPullAckRecord record = prodPullAckRecordRepository.findByRemoteConfigId(remoteId) + .orElseGet(ProdPullAckRecord::new); + record.setRemoteConfigId(remoteId); + record.setAckStatus(ackStatus); + record.setReported(Boolean.FALSE); + prodPullAckRecordRepository.save(record); + } + } + + /** + * 当前批次 pull 请求发送成功后，将待回执记录标记为已回传。 + */ + @Transactional + public void markPendingAsReported() { + List records = prodPullAckRecordRepository.findByReportedFalseOrderByUpdatedAtAsc(); + for (ProdPullAckRecord record : records) { + record.setReported(Boolean.TRUE); + prodPullAckRecordRepository.save(record); + } + } + + private List toRemoteIds(List records) { + List ids = new ArrayList(); + for (ProdPullAckRecord record : records) { + ids.add(record.getRemoteConfigId()); + } + return ids; + } + + /** + * 待回传 ACK 摘要。 + */ + public static class PendingAckSummary { + + private final List ackSucIds; + private final List ackFailIds; + + public PendingAckSummary(List ackSucIds, List ackFailIds) { + this.ackSucIds = ackSucIds; + this.ackFailIds = ackFailIds; + } + + public List getAckSucIds() { + return ackSucIds; + } + + public List getAckFailIds() { + return ackFailIds; + } + + public boolean hasPendingAck() { + return !(ackSucIds == null || ackSucIds.isEmpty()) || !(ackFailIds == null || ackFailIds.isEmpty()); + } + } +} diff --git a/src/main/java/com/ftptool/sync/service/WorkDirectoryService.java b/src/main/java/com/ftptool/sync/service/WorkDirectoryService.java index de1f2a7..8fb1b03 100644 --- a/src/main/java/com/ftptool/sync/service/WorkDirectoryService.java +++ b/src/main/java/com/ftptool/sync/service/WorkDirectoryService.java @@ -31,6 +31,7 @@ public class WorkDirectoryService { FileTreeUtils.ensureDirectory(getPackageTempDir()); FileTreeUtils.ensureDirectory(getDevToProdStagingDir()); FileTreeUtils.ensureDirectory(getProdToDevStagingDir()); + FileTreeUtils.ensureDirectory(getGitToProdBaselineDir()); } /** @@ -60,4 +61,11 @@ public class WorkDirectoryService { public Path getProdToDevStagingDir() { return Paths.get(syncProperties.getProdToDevStagingDir()).toAbsolutePath().normalize(); } + + /** + * Git -> PROD 链路用于比较增量的基线目录。 + */ + public Path getGitToProdBaselineDir() { + return getWorkDir().resolve("baseline").resolve("git-to-prod"); + } } diff --git a/src/main/java/com/ftptool/sync/util/FileTreeUtils.java b/src/main/java/com/ftptool/sync/util/FileTreeUtils.java index 1bf2bde..30d6542 100644 --- a/src/main/java/com/ftptool/sync/util/FileTreeUtils.java +++ b/src/main/java/com/ftptool/sync/util/FileTreeUtils.java @@ -7,6 +7,7 @@ import java.nio.file.Path; import java.nio.file.SimpleFileVisitor; import java.nio.file.StandardCopyOption; import java.nio.file.attribute.BasicFileAttributes; +import java.util.Collection; import java.util.Comparator; import java.util.stream.Stream; @@ -84,4 +85,17 @@ public final class FileTreeUtils { } }); } + + /** + * 按给定的相对路径集合复制部分文件，保留原目录结构。 + */ + public static void copySelectedFiles(Path sourceRoot, Path targetRoot, Collection relativeFiles) throws IOException { + ensureDirectory(targetRoot); + for (Path relativeFile : relativeFiles) { + Path sourceFile = sourceRoot.resolve(relativeFile).normalize(); + Path targetFile = targetRoot.resolve(relativeFile).normalize(); + ensureDirectory(targetFile.getParent()); + Files.copy(sourceFile, targetFile, StandardCopyOption.REPLACE_EXISTING); + } + } } diff --git a/src/main/resources/application-prod-agent.properties b/src/main/resources/application-prod-agent.properties index 04ab81e..8cb2f47 100644 --- a/src/main/resources/application-prod-agent.properties +++ b/src/main/resources/application-prod-agent.properties @@ -10,6 +10,12 @@ sync.jobs.prod-to-git.cron=20 */2 * * * * # Example overrides prod.api.base-url=https://prod.example.com -prod.api.push-path=/api/config/push -prod.api.pull-path=/api/config/pull +prod.api.push-path=/pic_bus_manage_monitor/configSync/pushConfig +prod.api.pull-path=/pic_bus_manage_monitor/configSync/pullConfig +prod.api.login-path=/pic_bus_manage_monitor/pam-monitor/login prod.api.token=change-me +prod.api.token-header-name=token +prod.api.airport-id=replace-me +prod.api.app-name=replace-me +prod.api.login-name= +prod.api.login-password= diff --git a/src/main/resources/application.properties b/src/main/resources/application.properties index 8b58bc2..ad5df0e 100644 --- a/src/main/resources/application.properties +++ b/src/main/resources/application.properties @@ -42,9 +42,15 @@ git.repo.pull-rebase=false # Production API defaults prod.api.base-url=https://prod.example.com -prod.api.push-path=/api/config/push -prod.api.pull-path=/api/config/pull +prod.api.push-path=/pic_bus_manage_monitor/configSync/pushConfig +prod.api.pull-path=/pic_bus_manage_monitor/configSync/pullConfig +prod.api.login-path=/pic_bus_manage_monitor/pam-monitor/login prod.api.token=replace-me +prod.api.token-header-name=token +prod.api.airport-id=replace-me +prod.api.app-name=replace-me +prod.api.login-name= +prod.api.login-password= prod.api.connect-timeout-ms=10000 prod.api.read-timeout-ms=30000 diff --git a/src/main/resources/schema.sql b/src/main/resources/schema.sql index 9074a47..4064992 100644 --- a/src/main/resources/schema.sql +++ b/src/main/resources/schema.sql @@ -25,3 +25,15 @@ create table if not exists sync_task ( create index if not exists idx_sync_task_status on sync_task (status); create index if not exists idx_sync_task_direction on sync_task (direction); + +create table if not exists prod_pull_ack ( + id bigint generated by default as identity primary key, + remote_config_id varchar(128) not null, + ack_status varchar(16) not null, + reported boolean not null default false, + created_at timestamp not null, + updated_at timestamp not null, + constraint uk_prod_pull_ack_remote_id unique (remote_config_id) +); + +create index if not exists idx_prod_pull_ack_reported on prod_pull_ack (reported); diff --git a/src/test/java/com/ftptool/sync/orchestrator/ProdSyncCoordinatorIntegrationTest.java b/src/test/java/com/ftptool/sync/orchestrator/ProdSyncCoordinatorIntegrationTest.java index a79a8f1..2c63adc 100644 --- a/src/test/java/com/ftptool/sync/orchestrator/ProdSyncCoordinatorIntegrationTest.java +++ b/src/test/java/com/ftptool/sync/orchestrator/ProdSyncCoordinatorIntegrationTest.java @@ -1,13 +1,15 @@ package com.ftptool.sync.orchestrator; import com.ftptool.sync.GitDirectSyncToolApplication; +import com.ftptool.sync.entity.ProdPullAckRecord; import com.ftptool.sync.entity.SyncCheckpoint; import com.ftptool.sync.entity.SyncTask; -import com.ftptool.sync.model.PackageBuildResult; import com.ftptool.sync.model.PackageManifest; +import com.ftptool.sync.model.ProdPullAckStatus; import com.ftptool.sync.model.ProdPullResult; import com.ftptool.sync.model.SyncDirection; import com.ftptool.sync.model.SyncStatus; +import com.ftptool.sync.repository.ProdPullAckRecordRepository; import com.ftptool.sync.repository.SyncCheckpointRepository; import com.ftptool.sync.repository.SyncTaskRepository; import com.ftptool.sync.service.GitClientService; @@ -18,18 +20,24 @@ import org.junit.jupiter.api.Test; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.boot.test.context.SpringBootTest; import org.springframework.boot.test.mock.mockito.MockBean; +import org.springframework.test.annotation.DirtiesContext; import org.springframework.test.context.ActiveProfiles; import java.nio.file.Files; import java.nio.file.Path; +import java.util.ArrayList; +import java.util.Arrays; import java.util.List; import java.util.Optional; +import java.util.concurrent.atomic.AtomicInteger; import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; import static org.junit.jupiter.api.Assertions.assertTrue; import static org.mockito.ArgumentMatchers.any; import static org.mockito.ArgumentMatchers.contains; import static org.mockito.ArgumentMatchers.eq; +import static org.mockito.Mockito.doAnswer; import static org.mockito.Mockito.doNothing; import static org.mockito.Mockito.times; import static org.mockito.Mockito.verify; @@ -44,16 +52,18 @@ import static org.mockito.Mockito.when; "spring.datasource.password=", "spring.jpa.hibernate.ddl-auto=none", "spring.sql.init.mode=always", - "sync.work-dir=./target/test-work", - "sync.package-temp-dir=./target/test-work/package", - "sync.dev-to-prod-staging-dir=./target/test-work/dev-to-prod", - "sync.prod-to-dev-staging-dir=./target/test-work/prod-to-dev", + "test.work-root=./target/test-work-${random.uuid}", + "sync.work-dir=${test.work-root}", + "sync.package-temp-dir=${test.work-root}/package", + "sync.dev-to-prod-staging-dir=${test.work-root}/dev-to-prod", + "sync.prod-to-dev-staging-dir=${test.work-root}/prod-to-dev", "git.repo.scan-branch=config-dev-main", "git.repo.snapshot-branch=config-prod-snapshot", - "git.repo.local-path=./target/test-work/git/config-repo" + "git.repo.local-path=${test.work-root}/git/config-repo" } ) @ActiveProfiles("prod-agent") +@DirtiesContext(classMode = DirtiesContext.ClassMode.AFTER_EACH_TEST_METHOD) class ProdSyncCoordinatorIntegrationTest { @Autowired @@ -65,6 +75,9 @@ class ProdSyncCoordinatorIntegrationTest { @Autowired private SyncCheckpointRepository syncCheckpointRepository; + @Autowired + private ProdPullAckRecordRepository prodPullAckRecordRepository; + @MockBean private GitClientService gitClientService; @@ -75,24 +88,19 @@ class ProdSyncCoordinatorIntegrationTest { private ProdConfigApiService prodConfigApiService; @BeforeEach - void setUp() { + void setUp() throws Exception { syncTaskRepository.deleteAll(); syncCheckpointRepository.deleteAll(); + prodPullAckRecordRepository.deleteAll(); } @Test void shouldSyncGitToProdAndKeepItIdempotent() throws Exception { - Path zipFile = Files.createTempFile("git-to-prod-", ".zip"); when(gitClientService.prepareRepositoryAndGetHead("config-dev-main")).thenReturn("commit-a"); when(gitClientService.exportBranchSnapshot(eq("config-dev-main"), any(Path.class))) .thenAnswer(invocation -> invocation.getArgument(1)); when(packageService.calculateDirectoryHash(any(Path.class))).thenReturn("hash-a"); - when(packageService.buildPackageFromDirectory(any(Path.class), any(PackageManifest.class))) - .thenAnswer(invocation -> { - PackageManifest manifest = invocation.getArgument(1); - return new PackageBuildResult(zipFile, manifest.getPackageName(), "hash-a"); - }); - doNothing().when(prodConfigApiService).pushPackage(any(PackageManifest.class), eq(zipFile)); + doNothing().when(prodConfigApiService).pushPackage(any(PackageManifest.class), any(Path.class)); prodSyncCoordinator.syncLatestGitToProd(); prodSyncCoordinator.syncLatestGitToProd(); @@ -110,14 +118,16 @@ class ProdSyncCoordinatorIntegrationTest { assertEquals("commit-a", checkpoint.get().getLastSuccessVersion()); assertEquals("hash-a", checkpoint.get().getLastSuccessHash()); - verify(prodConfigApiService, times(1)).pushPackage(any(PackageManifest.class), eq(zipFile)); + verify(prodConfigApiService, times(1)).pushPackage(any(PackageManifest.class), any(Path.class)); } @Test void shouldSyncProdSnapshotToGitAndKeepItIdempotent() throws Exception { Path contentDirectory = Files.createTempDirectory("prod-to-git-"); Files.write(contentDirectory.resolve("prod-config.json"), "{\"version\":\"prod-v1\"}".getBytes("UTF-8")); - when(prodConfigApiService.pullConfigSnapshot()).thenReturn(new ProdPullResult(contentDirectory, "prod-v1", "hash-b")); + when(prodConfigApiService.pullConfigSnapshot()).thenReturn( + new ProdPullResult(contentDirectory, "prod-v1", "hash-b", Arrays.asList("1", "2")) + ); when(gitClientService.syncDirectoryToBranch( eq(contentDirectory), eq("config-prod-snapshot"), @@ -140,10 +150,79 @@ class ProdSyncCoordinatorIntegrationTest { assertEquals("prod-v1", checkpoint.get().getLastSuccessVersion()); assertEquals("hash-b", checkpoint.get().getLastSuccessHash()); + List ackRecords = prodPullAckRecordRepository.findAll(); + assertEquals(2, ackRecords.size()); + for (ProdPullAckRecord ackRecord : ackRecords) { + assertEquals(ProdPullAckStatus.SUCCESS, ackRecord.getAckStatus()); + assertFalse(ackRecord.getReported()); + } + verify(gitClientService, times(1)).syncDirectoryToBranch( eq(contentDirectory), eq("config-prod-snapshot"), contains("prod-v1") ); } + + @Test + void shouldRecordFailedAckWhenProdSnapshotSyncFails() throws Exception { + Path contentDirectory = Files.createTempDirectory("prod-to-git-fail-"); + Files.write(contentDirectory.resolve("prod-config.json"), "{\"version\":\"prod-v2\"}".getBytes("UTF-8")); + when(prodConfigApiService.pullConfigSnapshot()).thenReturn( + new ProdPullResult(contentDirectory, "prod-v2", "hash-fail", Arrays.asList("9")) + ); + when(gitClientService.syncDirectoryToBranch( + eq(contentDirectory), + eq("config-prod-snapshot"), + contains("prod-v2") + )).thenThrow(new IllegalStateException("git push fail")); + + prodSyncCoordinator.syncProdSnapshotToGit(); + + List ackRecords = prodPullAckRecordRepository.findAll(); + assertEquals(1, ackRecords.size()); + assertEquals("9", ackRecords.get(0).getRemoteConfigId()); + assertEquals(ProdPullAckStatus.FAILED, ackRecords.get(0).getAckStatus()); + assertFalse(ackRecords.get(0).getReported()); + } + + @Test + void shouldUseIncrementalDirectoryForSecondGitToProdPush() throws Exception { + AtomicInteger exportCounter = new AtomicInteger(0); + List pushedDirectories = new ArrayList(); + + when(gitClientService.prepareRepositoryAndGetHead("config-dev-main")) + .thenReturn("commit-base") + .thenReturn("commit-delta"); + when(gitClientService.exportBranchSnapshot(eq("config-dev-main"), any(Path.class))) + .thenAnswer(invocation -> { + Path target = invocation.getArgument(1); + Files.createDirectories(target); + int round = exportCounter.incrementAndGet(); + if (round == 1) { + Files.write(target.resolve("a.txt"), "v1".getBytes("UTF-8")); + Files.write(target.resolve("b.txt"), "same".getBytes("UTF-8")); + } else { + Files.write(target.resolve("a.txt"), "v2".getBytes("UTF-8")); + Files.write(target.resolve("b.txt"), "same".getBytes("UTF-8")); + } + return target; + }); + when(packageService.calculateDirectoryHash(any(Path.class))) + .thenReturn("hash-base") + .thenReturn("hash-delta"); + doAnswer(invocation -> { + pushedDirectories.add(invocation.getArgument(1)); + return null; + }).when(prodConfigApiService).pushPackage(any(PackageManifest.class), any(Path.class)); + + prodSyncCoordinator.syncLatestGitToProd(); + prodSyncCoordinator.syncLatestGitToProd(); + + assertEquals(2, pushedDirectories.size()); + Path secondPushDirectory = pushedDirectories.get(1); + assertTrue(Files.exists(secondPushDirectory.resolve("a.txt"))); + assertFalse(Files.exists(secondPushDirectory.resolve("b.txt"))); + assertTrue(secondPushDirectory.getFileName().toString().startsWith("git-delta-")); + } } diff --git a/src/test/java/com/ftptool/sync/service/ProdConfigApiServiceHttpTest.java b/src/test/java/com/ftptool/sync/service/ProdConfigApiServiceHttpTest.java new file mode 100644 index 0000000..f424f56 --- /dev/null +++ b/src/test/java/com/ftptool/sync/service/ProdConfigApiServiceHttpTest.java @@ -0,0 +1,91 @@ +package com.ftptool.sync.service; + +import com.ftptool.sync.config.ProdApiProperties; +import com.ftptool.sync.config.SyncProperties; +import com.ftptool.sync.model.ProdPullResult; +import org.junit.jupiter.api.Test; +import org.springframework.http.HttpMethod; +import org.springframework.http.MediaType; +import org.springframework.test.web.client.MockRestServiceServer; +import org.springframework.web.client.RestTemplate; + +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.Arrays; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertTrue; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.when; +import static org.springframework.test.web.client.ExpectedCount.once; +import static org.springframework.test.web.client.match.MockRestRequestMatchers.header; +import static org.springframework.test.web.client.match.MockRestRequestMatchers.method; +import static org.springframework.test.web.client.response.MockRestResponseCreators.withSuccess; + +class ProdConfigApiServiceHttpTest { + + @Test + void shouldSendAckParamsWhenPullingConfig() throws Exception { + RestTemplate restTemplate = new RestTemplate(); + MockRestServiceServer server = MockRestServiceServer.bindTo(restTemplate).build(); + + SyncProperties syncProperties = new SyncProperties(); + syncProperties.setWorkDir("./target/http-test-work"); + syncProperties.setPackageTempDir("./target/http-test-work/package"); + syncProperties.setDevToProdStagingDir("./target/http-test-work/dev-to-prod"); + syncProperties.setProdToDevStagingDir("./target/http-test-work/prod-to-dev"); + syncProperties.setPullResponseFileName("prod-config.json"); + + WorkDirectoryService workDirectoryService = new WorkDirectoryService(syncProperties); + workDirectoryService.initialize(); + + ProdApiProperties prodApiProperties = new ProdApiProperties(); + prodApiProperties.setBaseUrl("https://prod.example.com"); + prodApiProperties.setPullPath("/pic_bus_manage_monitor/configSync/pullConfig"); + prodApiProperties.setToken("static-token"); + prodApiProperties.setTokenHeaderName("token"); + prodApiProperties.setAirportId("test-airport"); + prodApiProperties.setAppName("test-app"); + + ProdPullAckService prodPullAckService = mock(ProdPullAckService.class); + when(prodPullAckService.getPendingAckSummary()).thenReturn( + new ProdPullAckService.PendingAckSummary(Arrays.asList("1", "2"), Arrays.asList("9")) + ); + + ProdConfigApiService service = new ProdConfigApiService( + prodApiProperties, + syncProperties, + restTemplate, + workDirectoryService, + prodPullAckService + ); + + server.expect(once(), request -> { + String uri = request.getURI().toString(); + assertTrue(uri.contains("airportId=test-airport")); + assertTrue(uri.contains("appName=test-app")); + assertTrue(uri.contains("ackSuc=1,2") || uri.contains("ackSuc=1%2C2")); + assertTrue(uri.contains("ackFail=9")); + }) + .andExpect(method(HttpMethod.GET)) + .andExpect(header("token", "static-token")) + .andRespond(withSuccess( + "{\"code\":\"0\",\"data\":[{\"id\":\"1\",\"airportId\":\"test-airport\",\"appName\":\"test-app\",\"configVersion\":\"v1\",\"configContent\":\"content\",\"fileName\":\"a.txt\"}],\"msg\":\"ok\"}", + MediaType.APPLICATION_JSON + )); + + ProdPullResult result = service.pullConfigSnapshot(); + + assertEquals("v1", result.getSourceVersion()); + assertEquals(1, result.getPulledConfigIds().size()); + assertEquals("1", result.getPulledConfigIds().get(0)); + Path restoredFile = result.getContentDirectory().resolve("a.txt"); + assertTrue(Files.exists(restoredFile)); + assertEquals("content", new String(Files.readAllBytes(restoredFile), StandardCharsets.UTF_8)); + + verify(prodPullAckService).markPendingAsReported(); + server.verify(); + } +} diff --git a/testapi.txt b/testapi.txt new file mode 100644 index 0000000..4edb913 --- /dev/null +++ b/testapi.txt @@ -0,0 +1,126 @@ +1）git中转服务推送GIT配置到生产的接口 + +接口地址:http://ip:port/pic_bus_manage_monitor/configSync/pushConfig +请求方式:post +内容格式:json +编码:UTF-8 +HEADER里带token用于鉴权 +请求参数:无 +请求体：[{ +"airportId":"test", + +"appName":"XXX", + +"configVersion":"R_XXX_V3.0.3_XXX"， + +"configContent":"配置内容", + +"fileName":"配置文件名" + +}] +响应：{ + + "code":"0", + +"data":{"ackFail":[{ + +"airportId":"test", + +"appName":"XXXx", + +"configVersion":"R_XXX_V3.0.35.6.1_XXX"， + +"fileName":"配置文件名" + +}]}, + +"msg":"ok" + +} + +备注：配置推送第一次全量，之后增量，全量配置较大，需视情况分多次推送。配置需加密,加密方式留成TODO。 + + +2）git中转服务从生产拉取配置到GIT的接口 + +接口地址:http://ip:port/pic_bus_manage_monitor/configSync/pullConfig +请求方式:get +内容格式:json +编码:UTF-8 +HEADER里带token用于鉴权 +请求参数:{ +"airportId":"test", + +"appName":"XXXx", + +"configVersion":"R_XXX_V3.0.35.6.1_XXX", + +"fileName":"配置文件名", + +"ackSuc":"id,id"， + +"ackFail":"id,id" + +} +响应： + +{ + + "code":"0", + +"data":[{ + +"id":"1", + +"airportId":"test", + +"appName":"XXXx", + +"configVersion":"R_XXX_V3.0.35.6.1_XXX"， + +"configContent":"配置内容（加密）", + +"fileName":"配置文件名" + +}], +"msg":"ok" + +} + + 备注：不带参数则返回未同步到git的所有已审核通过的配置。配置需加密,加密方式留成TODO。 +3）git中转服务获取token的接口(已有接口) + +接口地址:http://ip:port/pic_bus_manage_monitor/pam-monitor/login +请求方式:post +内容格式:json +编码:UTF-8 +请求参数:{ +"name":"XXXxx", + +"password":"" + +} +响应： + +{ + + "code":"0", + +"data":{"token"："tetttwe","expireTime":"2026-07-11 11:11:11"}, + +"msg":"ok" + +} + +备注:若token过期，重新调用登录接口获取token + + + +所有接口失败时的响应格式： + +{ + "code": "XXX-00-00-XXX", + "data": null, + "msg": "errmsg！", + "timestamp": "1776735560594" +} \ No newline at end of file