feat:初始化方案和框架

This commit is contained in:
dark 2026-04-15 15:58:23 +08:00
parent f0af9b8aba
commit 8d46e629ad
34 changed files with 4997 additions and 0 deletions

2545
.gitignore vendored Normal file

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,603 @@
# 基于 FTP 中转的配置双向同步工具设计方案
## 1. 文档目的
本文档用于说明一套基于 FTP 中转的配置同步工具设计方案,满足以下目标:
- 开发环境定时从 Git 拉取新配置,并通过生产环境 `push` 接口推送到生产
- 生产环境定时从 `pull` 接口拉取新配置,并同步回开发环境 Git
- 在开发环境与生产环境不能直接互通时,通过 FTP 服务作为中转通道完成双向同步
## 2. 已知约束
### 2.1 技术约束
- JDK`1.8`
- Spring Boot`2.7.18`
- 轻量数据库:`H2` 或同类开源可商用数据库
- 其他依赖必须为开源可商用组件
### 2.2 网络与部署约束
- 无法登录 FTP 所在服务器主机
- 只能访问 FTP 服务:`IP + 端口 + 用户名/密码`
- 网络拓扑如下:
```text
开发环境 <----> FTP A <----> 生产环境
```
说明:
- 开发环境可以访问 FTP A
- 生产环境可以访问 FTP A
- 开发与生产不假设可以直接互通
## 3. 设计原则
- 同一套程序,按不同 `profile` 部署在开发和生产两端
- 通过 FTP 传递标准化同步包,避免环境间直接通信依赖
- 使用本地状态库记录同步任务、检查点、应答信息,保证可追踪、可恢复
- 同步流程必须具备幂等控制,避免重复推送、重复提交
- 开发到生产、生产回开发必须隔离处理,避免双向同步形成死循环
## 4. 总体方案
推荐采用“**双端代理 + FTP 中转 + 本地状态库**”架构:
- `Sync-Agent-Dev`:部署在开发环境
- `Sync-Agent-Prod`:部署在生产环境
- `FTP A`:作为唯一中转通道
- `H2`:记录同步状态、任务、检查点、重试信息
整体结构如下:
```text
开发环境
Sync-Agent-Dev
|- 拉取 Git
|- 上传/下载 FTP A
|- 写入 Git
生产环境
Sync-Agent-Prod
|- 调用生产 pull 接口
|- 调用生产 push 接口
|- 上传/下载 FTP A
中转
FTP A
|- dev-to-prod/
|- prod-to-dev/
|- ack/
|- failed/
```
## 5. 部署模式
建议只维护一套代码,通过 Spring Profile 控制角色:
- `dev-agent`:启用开发侧能力
- `prod-agent`:启用生产侧能力
### 5.1 开发侧职责
- 定时拉取 Git 指定分支的新配置
- 判断是否存在新的有效版本
- 打包配置并上传到 FTP
- 下载生产侧回传的同步包
- 将生产侧回传配置写入 Git
- 提交并推送到远端仓库
### 5.2 生产侧职责
- 轮询 FTP获取开发侧上传的配置包
- 校验后调用生产 `push` 接口导入配置
- 定时调用生产 `pull` 接口拉取最新配置
- 打包并上传回 FTP供开发侧消费
## 6. 技术选型
| 分类 | 选型 | 说明 |
| --- | --- | --- |
| 运行时 | JDK 1.8 | 满足约束 |
| 框架 | Spring Boot 2.7.18 | 主体框架 |
| 调度 | Spring Scheduling | 实现定时任务 |
| 重试 | Spring Retry | 失败重试 |
| 数据库 | H2 File Mode | 轻量、嵌入式、可持久化 |
| Git 操作 | JGit | 纯 Java 实现 |
| FTP 操作 | Apache Commons Net | 主流 FTP 客户端 |
| JSON | Jackson | 标准序列化组件 |
| 日志 | SLF4J + Logback | 默认日志能力 |
### 6.1 数据库模式建议
虽然需求提到“类似 H2 的轻量化内存数据库”,但本场景不建议纯内存模式,原因如下:
- 服务重启后需要保留同步检查点
- 失败任务需要支持补偿和人工追踪
- 需要记录包处理状态,避免重复消费
因此建议使用:
- `H2 File Mode`
即本地文件数据库,仍然轻量,但支持状态持久化。
## 7. 核心业务流程
系统包含两条主链路。
### 7.1 链路一:开发 Git -> 生产 push 接口
用途:将开发环境 Git 中的新配置推送到生产环境。
流程如下:
1. `dev-agent` 定时拉取 Git 指定分支
2. 判断 Git 最新提交是否为新的有效配置版本
3. 将配置目录打包为标准同步包
4. 上传至 FTP 路径 `dev-to-prod/out/`
5. `prod-agent` 轮询 FTP发现新包后下载
6. 校验包完整性、幂等键和来源信息
7. 调用生产环境 `push` 接口导入配置
8. 成功后生成 `ack` 文件上传到 FTP
9. `dev-agent` 读取 `ack`,将任务状态更新为成功
建议时序图如下:
```mermaid
sequenceDiagram
participant G as Git(开发)
participant D as Sync-Agent-Dev
participant F as FTP A
participant P as Sync-Agent-Prod
participant API as 生产Push接口
D->>G: 定时 pull 配置
D->>D: 检查是否有新版本
D->>D: 打包 zip + manifest
D->>F: 上传 dev-to-prod/out/
P->>F: 轮询并下载新包
P->>P: 校验 hash/traceId
P->>API: 调用 push 接口
API-->>P: 返回处理结果
P->>F: 上传 ack
D->>F: 读取 ack
D->>D: 更新状态为成功
```
### 7.2 链路二:生产 pull 接口 -> 开发 Git
用途:将生产环境当前配置回传到开发环境,形成配置镜像或审计记录。
流程如下:
1. `prod-agent` 定时调用生产 `pull` 接口
2. 将返回配置标准化后计算版本标识或内容哈希
3. 如果与上次同步结果不同,则打包上传到 FTP `prod-to-dev/out/`
4. `dev-agent` 轮询 FTP 并下载新包
5. 解包后写入本地 Git 工作目录
6. 提交 commit 并推送到远端 Git
7. 成功后写回 `ack`
建议时序图如下:
```mermaid
sequenceDiagram
participant API as 生产Pull接口
participant P as Sync-Agent-Prod
participant F as FTP A
participant D as Sync-Agent-Dev
participant G as Git(开发)
P->>API: 定时调用 pull 接口
API-->>P: 返回当前配置
P->>P: 标准化并计算 hash
P->>F: 上传 prod-to-dev/out/
D->>F: 轮询并下载新包
D->>D: 解包并写入工作区
D->>G: commit + push
D->>F: 上传 ack
```
## 8. 标准同步包设计
为保证跨环境处理一致,建议所有同步内容封装为统一格式的压缩包。
### 8.1 包结构
```text
package.zip
|- manifest.json
|- config/
|- sha256.txt
```
### 8.2 manifest 字段建议
```json
{
"traceId": "uuid",
"direction": "DEV_TO_PROD",
"sourceEnv": "DEV",
"sourceVersion": "gitCommitId",
"contentHash": "sha256",
"createdAt": "2026-04-15T10:00:00+08:00"
}
```
### 8.3 字段说明
- `traceId`:本次同步唯一流水号
- `direction`:同步方向,例如 `DEV_TO_PROD``PROD_TO_DEV`
- `sourceEnv`:来源环境
- `sourceVersion`:来源版本号,开发侧通常为 Git Commit ID
- `contentHash`:配置内容哈希,便于判断重复包
- `createdAt`:包生成时间
## 9. FTP 目录规划
建议在 FTP A 上使用如下目录结构:
```text
/dev-to-prod/out/
/dev-to-prod/ack/
/prod-to-dev/out/
/prod-to-dev/ack/
/failed/
```
目录说明:
- `/dev-to-prod/out/`:开发侧发往生产侧的同步包
- `/dev-to-prod/ack/`:生产侧返回的处理应答
- `/prod-to-dev/out/`:生产侧发往开发侧的同步包
- `/prod-to-dev/ack/`:开发侧返回的处理应答
- `/failed/`:失败包归档目录
### 9.1 上传规范
为避免消费端读取到半截文件,建议采用临时文件上传策略:
1. 先上传为 `.tmp`
2. 上传完成后重命名为正式 `.zip`
3. 消费端只处理 `.zip` 文件
## 10. Git 分支策略
这是方案中的关键设计点。
不建议将“开发配置推生产”和“生产配置回传开发”写到同一个 Git 分支,否则极易形成循环同步。
建议拆分为两个分支:
- `config-dev-main`:开发主配置分支
- `config-prod-snapshot`:生产配置镜像分支
同步规则:
- `DEV -> PROD` 只消费 `config-dev-main`
- `PROD -> DEV` 只写入 `config-prod-snapshot`
### 10.1 这样设计的好处
- 避免双向同步形成闭环
- 生产回传配置不会覆盖开发主线
- 便于审计“生产当前实际配置”
### 10.2 机器人提交标记
建议同步工具在 commit message 中增加固定前缀,例如:
```text
sync(prod->git): traceId=xxx version=xxx
```
开发侧扫描 Git 时应忽略同步机器人生成的提交,进一步降低环路风险。
## 11. 本地状态库设计
建议至少建立以下 3 张表。
### 11.1 `sync_checkpoint`
用于记录各方向的最后一次成功检查点。
| 字段 | 类型 | 说明 |
| --- | --- | --- |
| id | bigint | 主键 |
| direction | varchar | 同步方向 |
| last_success_version | varchar | 最后成功版本 |
| last_success_hash | varchar | 最后成功内容哈希 |
| updated_at | timestamp | 更新时间 |
### 11.2 `sync_task`
用于记录每次同步任务生命周期。
| 字段 | 类型 | 说明 |
| --- | --- | --- |
| id | bigint | 主键 |
| trace_id | varchar | 流水号 |
| direction | varchar | 同步方向 |
| source_version | varchar | 来源版本 |
| package_name | varchar | 包文件名 |
| status | varchar | 状态 |
| retry_count | int | 重试次数 |
| error_msg | clob | 错误信息 |
| created_at | timestamp | 创建时间 |
| updated_at | timestamp | 更新时间 |
### 11.3 `sync_ack`
用于记录应答信息。
| 字段 | 类型 | 说明 |
| --- | --- | --- |
| id | bigint | 主键 |
| trace_id | varchar | 流水号 |
| ack_side | varchar | 应答方 |
| ack_status | varchar | 应答状态 |
| ack_time | timestamp | 应答时间 |
| remark | varchar | 备注 |
## 12. 幂等与一致性设计
### 12.1 幂等键建议
建议以如下组合作为幂等键:
```text
direction + sourceVersion + contentHash
```
约束效果:
- 已经处理过的包不能重复推送
- 已经提交过的生产快照不能重复写 Git
### 12.2 一致性策略
本方案属于跨系统、跨网络的异步同步,不适合做强一致事务。
建议采用:
- “本地落库 + 外部调用 + 最终一致”模式
- 每一步记录状态
- 失败后允许自动重试或人工补偿
## 13. 失败处理与补偿机制
### 13.1 自动重试
以下场景建议自动重试:
- FTP 上传失败
- FTP 下载失败
- 生产 `push` 接口调用失败
- 生产 `pull` 接口调用失败
- Git push 失败
建议策略:
- 最大重试次数:`3 ~ 5`
- 重试间隔:指数退避,例如 `30s / 60s / 120s`
### 13.2 失败归档
连续失败后建议:
- 将包移动到 FTP 的 `/failed/`
- 将任务状态置为 `FAILED`
- 记录完整错误信息
- 触发告警
### 13.3 人工补偿
后续可以增加一个管理接口,支持:
- 按 `traceId` 重新执行
- 重置任务状态
- 查看失败原因
## 14. 安全设计
### 14.1 传输安全
优先级建议如下:
1. 优先使用 `FTPS`
2. 如果只能使用普通 FTP建议对同步包内容做 AES 加密
### 14.2 凭据管理
以下信息不得写死在代码中:
- FTP 地址、端口、用户名、密码
- Git 用户名、密码或 Token
- 生产接口认证信息
建议通过以下方式外置:
- `application-*.properties`
- 环境变量
- 启动参数
### 14.3 审计日志
建议记录:
- 谁发起了同步
- 同步方向
- 来源版本
- 包名
- 接口调用结果
- 异常原因
## 15. 项目结构建议
有两种实现方式。
### 15.1 方案 A单工程 + Profile 切换
适用于项目规模较小、交付快的场景。
```text
sync-tool
|- src/main/java
| |- config
| |- ftp
| |- git
| |- job
| |- package
| |- repository
| |- service
| |- web
|- src/main/resources
| |- application.properties
| |- application-dev-agent.properties
| |- application-prod-agent.properties
```
### 15.2 方案 B多模块拆分
适用于后续可能演化较多、职责更清晰的场景。
```text
sync-tool
|- common
|- dev-agent
|- prod-agent
```
当前建议优先采用:
- `方案 A单工程 + Profile`
理由:
- 实现成本低
- 运维简单
- 早期更适合快速打通链路
## 16. 核心模块划分
建议按职责拆分以下模块:
- `GitService`
- 拉取仓库
- 检查最新提交
- 提交并推送生产回传配置
- `FtpService`
- 上传、下载、重命名、目录扫描
- `PackageService`
- 生成 zip
- 生成 manifest
- 校验 hash
- `SyncTaskService`
- 任务创建
- 状态变更
- 检查点维护
- `ProdPushService`
- 调用生产 `push` 接口
- `ProdPullService`
- 调用生产 `pull` 接口
- `AckService`
- 生成和消费 ack 文件
- `JobScheduler`
- 各类定时任务调度
## 17. 定时任务建议
### 17.1 开发侧任务
- `GitPullJob`
- 周期拉取 Git 并检查是否有新配置
- `UploadDevPackageJob`
- 将待同步配置上传到 FTP
- `ConsumeProdPackageJob`
- 下载生产回传包并写入 Git
- `AckScanJob`
- 扫描生产侧 ack 并更新任务状态
### 17.2 生产侧任务
- `ConsumeDevPackageJob`
- 下载开发侧同步包并调用生产 `push`
- `PullProdConfigJob`
- 定时调用生产 `pull` 接口
- `UploadProdPackageJob`
- 将拉取结果上传到 FTP
- `AckScanJob`
- 扫描开发侧 ack 并更新任务状态
## 18. 一期 MVP 建议
建议按最小可交付版本分阶段实施。
### 阶段 1打通主链路
- 建立 Spring Boot 工程
- 集成 H2、JGit、FTP
- 实现开发到生产的全量包同步
- 实现生产 `push` 接口调用
### 阶段 2打通回传链路
- 接入生产 `pull` 接口
- 实现生产到开发的 FTP 回传
- 实现开发侧写入 Git 并推送
### 阶段 3增强稳定性
- 增加重试
- 增加 ack 机制
- 增加失败归档
- 增加告警与审计日志
## 19. 风险与注意事项
### 19.1 最大风险:双向同步闭环
如果生产回传配置写入开发主分支,再被开发侧识别为“新配置”,会再次推送到生产,形成无限循环。
规避措施:
- 使用独立镜像分支
- 识别机器人提交
- 使用幂等键
### 19.2 配置冲突风险
如果开发和生产都会修改同一份配置,且要求双向合并,则不能简单用文件覆盖方式处理。
当前建议:
- 将生产回传定义为“镜像/审计”
- 不直接回写开发主配置分支
### 19.3 FTP 能力限制
如果 FTP 不支持原子重命名、目录权限受限或稳定性较差,需要额外做兼容与重试。
## 20. 结论
在当前网络条件下,推荐采用“**开发代理 + 生产代理 + FTP 中转 + H2 状态库**”的双端部署方案。
该方案具备以下特点:
- 不依赖开发与生产直接互通
- 满足开发到生产、生产到开发的双向同步需求
- 支持状态记录、失败重试、幂等控制和审计追踪
- 适合使用 `Java 1.8 + Spring Boot 2.7.18` 快速落地
## 21. 后续可继续细化内容
后续可以基于本方案继续输出:
- `application.properties` 配置项设计
- H2 建表 SQL
- 核心类图与接口设计
- 各定时任务的时序与状态流转
- Spring Boot 工程骨架

View File

@ -0,0 +1,457 @@
# FTP 同步工具详细设计
## 1. 文档说明
本文档是对总体方案的继续细化,重点补充以下内容:
- `application.properties` 配置方案
- H2 表结构与初始化方式
- Spring Boot 2.7.18 工程骨架
- 核心类职责划分
- 启动方式与后续待实现事项
## 2. 配置文件策略
本项目采用 `properties` 配置文件,不使用 `yml`
推荐目录如下:
```text
src/main/resources/
|- application.properties
|- application-dev-agent.properties
|- application-prod-agent.properties
|- schema.sql
```
配置分工如下:
- `application.properties`
- 放公共配置
- 包括数据源、H2、通用路径、FTP 默认项、Git 默认项、生产接口默认项
- `application-dev-agent.properties`
- 放开发环境代理专属配置
- 包括开发侧定时任务表达式、开发侧 FTP 账号、Git 仓库分支
- `application-prod-agent.properties`
- 放生产环境代理专属配置
- 包括生产侧定时任务表达式、生产侧 FTP 账号、生产接口地址与认证
## 3. 当前配置项设计
### 3.1 公共配置
已落地文件:
- [application.properties](e:/AIcoding/FtpTool/src/main/resources/application.properties)
核心配置分组如下:
### `spring.*`
- `spring.application.name`
- `spring.datasource.*`
- `spring.jpa.*`
- `spring.sql.init.*`
- `spring.h2.console.*`
用途:
- 启动 Spring Boot
- 使用 H2 文件数据库
- 通过 `schema.sql` 初始化表结构
### `sync.*`
- `sync.node-id`
- `sync.role`
- `sync.work-dir`
- `sync.package-temp-dir`
- `sync.dev-to-prod-staging-dir`
- `sync.prod-to-dev-staging-dir`
- `sync.max-retry-count`
- `sync.ack-scan-batch-size`
用途:
- 标识当前节点身份
- 控制工作目录和临时目录
- 控制同步重试与 ack 扫描参数
### `ftp.*`
- `ftp.host`
- `ftp.port`
- `ftp.username`
- `ftp.password`
- `ftp.passive-mode`
- `ftp.base-dir`
- `ftp.connect-timeout-ms`
- `ftp.data-timeout-ms`
- `ftp.buffer-size`
用途:
- 定义 FTP 连接参数
- 定义远端根目录和超时策略
### `git.repo.*`
- `git.repo.local-path`
- `git.repo.remote-uri`
- `git.repo.username`
- `git.repo.password`
- `git.repo.scan-branch`
- `git.repo.snapshot-branch`
- `git.repo.commit-author-name`
- `git.repo.commit-author-email`
- `git.repo.commit-message-prefix`
- `git.repo.pull-rebase`
用途:
- 定义开发侧 Git 拉取与提交行为
- 指定开发主分支和生产镜像分支
### `prod.api.*`
- `prod.api.base-url`
- `prod.api.push-path`
- `prod.api.pull-path`
- `prod.api.token`
- `prod.api.connect-timeout-ms`
- `prod.api.read-timeout-ms`
用途:
- 定义生产侧 `push/pull` 接口的连接方式
## 4. Profile 设计
### 4.1 开发代理 Profile
已落地文件:
- [application-dev-agent.properties](e:/AIcoding/FtpTool/src/main/resources/application-dev-agent.properties)
主要内容:
- `spring.config.activate.on-profile=dev-agent`
- 开发侧端口
- 开发侧三类任务 cron
- 开发侧 FTP 账号示例
- Git 分支覆盖项
当前定时任务:
- `sync.jobs.dev-git-scan.cron`
- `sync.jobs.dev-consume-prod-package.cron`
- `sync.jobs.dev-ack-scan.cron`
### 4.2 生产代理 Profile
已落地文件:
- [application-prod-agent.properties](e:/AIcoding/FtpTool/src/main/resources/application-prod-agent.properties)
主要内容:
- `spring.config.activate.on-profile=prod-agent`
- 生产侧端口
- 生产侧三类任务 cron
- 生产侧 FTP 账号示例
- 生产接口地址和 token 示例
当前定时任务:
- `sync.jobs.prod-consume-dev-package.cron`
- `sync.jobs.prod-pull-config.cron`
- `sync.jobs.prod-ack-scan.cron`
## 5. H2 设计
已落地文件:
- [schema.sql](e:/AIcoding/FtpTool/src/main/resources/schema.sql)
### 5.1 初始化方式
通过以下配置自动初始化:
```properties
spring.sql.init.mode=always
spring.sql.init.schema-locations=classpath:schema.sql
spring.jpa.hibernate.ddl-auto=none
```
说明:
- 表结构由手工 SQL 控制
- 不依赖 Hibernate 自动建表
- 更适合后续环境迁移和版本管理
### 5.2 已定义表
#### `sync_checkpoint`
用途:
- 保存每个同步方向最后一次成功版本
关键字段:
- `direction`
- `last_success_version`
- `last_success_hash`
- `updated_at`
#### `sync_task`
用途:
- 保存每次同步任务实例
关键字段:
- `trace_id`
- `direction`
- `source_version`
- `content_hash`
- `package_name`
- `status`
- `retry_count`
- `error_msg`
关键约束:
- `trace_id` 唯一
- `direction + source_version + content_hash` 唯一
这组唯一键就是当前骨架里默认采用的幂等键。
#### `sync_ack`
用途:
- 保存跨端 ack 回执
关键字段:
- `trace_id`
- `ack_side`
- `ack_status`
- `ack_time`
- `remark`
## 6. 工程骨架
当前已经在仓库中生成了一套最小 Spring Boot 骨架。
### 6.1 构建文件
- [pom.xml](e:/AIcoding/FtpTool/pom.xml)
已引入的核心依赖:
- `spring-boot-starter`
- `spring-boot-starter-web`
- `spring-boot-starter-data-jpa`
- `spring-boot-starter-actuator`
- `spring-retry`
- `commons-net`
- `org.eclipse.jgit`
- `h2`
### 6.2 启动类
- [FtpSyncToolApplication.java](e:/AIcoding/FtpTool/src/main/java/com/ftptool/sync/FtpSyncToolApplication.java)
作用:
- 启用 Spring Boot
- 启用定时任务
- 启用重试机制
- 注册配置属性类
### 6.3 配置属性类
- [SyncProperties.java](e:/AIcoding/FtpTool/src/main/java/com/ftptool/sync/config/SyncProperties.java)
- [FtpProperties.java](e:/AIcoding/FtpTool/src/main/java/com/ftptool/sync/config/FtpProperties.java)
- [GitRepoProperties.java](e:/AIcoding/FtpTool/src/main/java/com/ftptool/sync/config/GitRepoProperties.java)
- [ProdApiProperties.java](e:/AIcoding/FtpTool/src/main/java/com/ftptool/sync/config/ProdApiProperties.java)
作用:
- 将 `properties` 配置映射为强类型对象
- 避免业务代码直接散落读取字符串 key
### 6.4 基础配置
- [AppConfig.java](e:/AIcoding/FtpTool/src/main/java/com/ftptool/sync/config/AppConfig.java)
当前提供:
- `RestTemplate` Bean
- 读取生产接口超时参数
## 7. 领域模型与仓储
### 7.1 枚举
- [SyncDirection.java](e:/AIcoding/FtpTool/src/main/java/com/ftptool/sync/model/SyncDirection.java)
- [SyncRole.java](e:/AIcoding/FtpTool/src/main/java/com/ftptool/sync/model/SyncRole.java)
- [SyncStatus.java](e:/AIcoding/FtpTool/src/main/java/com/ftptool/sync/model/SyncStatus.java)
用途:
- 统一同步方向、角色和状态定义
### 7.2 实体
- [SyncTask.java](e:/AIcoding/FtpTool/src/main/java/com/ftptool/sync/entity/SyncTask.java)
- [SyncCheckpoint.java](e:/AIcoding/FtpTool/src/main/java/com/ftptool/sync/entity/SyncCheckpoint.java)
- [SyncAck.java](e:/AIcoding/FtpTool/src/main/java/com/ftptool/sync/entity/SyncAck.java)
用途:
- 对应 H2 三张核心业务表
- 内置了基础时间戳维护逻辑
### 7.3 Repository
- [SyncTaskRepository.java](e:/AIcoding/FtpTool/src/main/java/com/ftptool/sync/repository/SyncTaskRepository.java)
- [SyncCheckpointRepository.java](e:/AIcoding/FtpTool/src/main/java/com/ftptool/sync/repository/SyncCheckpointRepository.java)
- [SyncAckRepository.java](e:/AIcoding/FtpTool/src/main/java/com/ftptool/sync/repository/SyncAckRepository.java)
用途:
- 提供基础持久化能力
- 已包含按幂等键和 `traceId` 查询的方法
## 8. 当前服务层设计
### 8.1 已实现基础服务
- [SyncTaskService.java](e:/AIcoding/FtpTool/src/main/java/com/ftptool/sync/service/SyncTaskService.java)
- [CheckpointService.java](e:/AIcoding/FtpTool/src/main/java/com/ftptool/sync/service/CheckpointService.java)
- [AckService.java](e:/AIcoding/FtpTool/src/main/java/com/ftptool/sync/service/AckService.java)
当前能力:
- 创建或加载幂等任务
- 更新任务状态
- 增加重试次数
- 更新检查点
- 记录 ack 回执
### 8.2 当前未实现的业务服务
当前骨架还没有把以下真实能力写完:
- FTP 上传、下载、列目录、重命名
- Git clone / pull / checkout / commit / push
- zip 打包与解包
- manifest 生成与校验
- 生产 `push` / `pull` 接口调用
这些是下一步真正要补的业务实现层。
## 9. 当前调度层设计
### 9.1 开发侧调度
- [DevSyncCoordinator.java](e:/AIcoding/FtpTool/src/main/java/com/ftptool/sync/orchestrator/DevSyncCoordinator.java)
- [DevGitScanJob.java](e:/AIcoding/FtpTool/src/main/java/com/ftptool/sync/job/DevGitScanJob.java)
- [DevConsumeProdPackageJob.java](e:/AIcoding/FtpTool/src/main/java/com/ftptool/sync/job/DevConsumeProdPackageJob.java)
- [DevAckScanJob.java](e:/AIcoding/FtpTool/src/main/java/com/ftptool/sync/job/DevAckScanJob.java)
当前状态:
- 已按 `dev-agent` profile 进行隔离
- 已绑定 cron 表达式
- 当前仅输出清晰日志和待办动作
### 9.2 生产侧调度
- [ProdSyncCoordinator.java](e:/AIcoding/FtpTool/src/main/java/com/ftptool/sync/orchestrator/ProdSyncCoordinator.java)
- [ProdConsumeDevPackageJob.java](e:/AIcoding/FtpTool/src/main/java/com/ftptool/sync/job/ProdConsumeDevPackageJob.java)
- [ProdPullConfigJob.java](e:/AIcoding/FtpTool/src/main/java/com/ftptool/sync/job/ProdPullConfigJob.java)
- [ProdAckScanJob.java](e:/AIcoding/FtpTool/src/main/java/com/ftptool/sync/job/ProdAckScanJob.java)
当前状态:
- 已按 `prod-agent` profile 进行隔离
- 已绑定 cron 表达式
- 当前仅输出清晰日志和待办动作
## 10. 当前目录结构
```text
FtpTool
|- docs
|- pom.xml
|- src
|- main
|- java/com/ftptool/sync
| |- config
| |- entity
| |- job
| |- model
| |- orchestrator
| |- repository
| |- service
|- resources
|- application.properties
|- application-dev-agent.properties
|- application-prod-agent.properties
|- schema.sql
```
## 11. 启动方式
### 11.1 启动开发代理
```bash
mvn spring-boot:run -Dspring-boot.run.profiles=dev-agent
```
### 11.2 启动生产代理
```bash
mvn spring-boot:run -Dspring-boot.run.profiles=prod-agent
```
也可以打包后通过 JVM 参数指定:
```bash
java -jar ftp-sync-tool.jar --spring.profiles.active=dev-agent
java -jar ftp-sync-tool.jar --spring.profiles.active=prod-agent
```
## 12. 下一步建议实现顺序
建议按以下顺序继续落代码:
1. 先实现 `FtpClientService`
2. 再实现 `GitClientService`
3. 再实现 `PackageService`
4. 再实现 `ProdConfigApiService`
5. 最后把 `Coordinator` 中的 TODO 串起来
## 13. 当前边界
当前骨架是“可扩展的项目起点”,不是完整业务实现,现阶段还缺:
- 真正的 FTP 交互
- 真正的 Git 操作
- 真正的生产接口调用
- 包文件读写与校验
- ack 文件协议
- 失败重试细节和告警
但好处是结构已经固定住了:
- 配置口径统一为 `properties`
- profile 隔离清晰
- H2 状态表已定义
- 调度入口已分开
- 任务、检查点、ack 的存储模型已落地

81
pom.xml Normal file
View File

@ -0,0 +1,81 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.7.18</version>
<relativePath/>
</parent>
<groupId>com.ftptool</groupId>
<artifactId>ftp-sync-tool</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>ftp-sync-tool</name>
<description>FTP relay based configuration sync tool</description>
<properties>
<java.version>1.8</java.version>
<jgit.version>6.10.0.202406032230-r</jgit.version>
<commons-net.version>3.11.1</commons-net.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.retry</groupId>
<artifactId>spring-retry</artifactId>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-aspects</artifactId>
</dependency>
<dependency>
<groupId>commons-net</groupId>
<artifactId>commons-net</artifactId>
<version>${commons-net.version}</version>
</dependency>
<dependency>
<groupId>org.eclipse.jgit</groupId>
<artifactId>org.eclipse.jgit</artifactId>
<version>${jgit.version}</version>
</dependency>
<dependency>
<groupId>com.h2database</groupId>
<artifactId>h2</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>

View File

@ -0,0 +1,27 @@
package com.ftptool.sync;
import com.ftptool.sync.config.FtpProperties;
import com.ftptool.sync.config.GitRepoProperties;
import com.ftptool.sync.config.ProdApiProperties;
import com.ftptool.sync.config.SyncProperties;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.retry.annotation.EnableRetry;
import org.springframework.scheduling.annotation.EnableScheduling;
@EnableRetry
@EnableScheduling
@SpringBootApplication
@EnableConfigurationProperties({
SyncProperties.class,
FtpProperties.class,
GitRepoProperties.class,
ProdApiProperties.class
})
public class FtpSyncToolApplication {
public static void main(String[] args) {
SpringApplication.run(FtpSyncToolApplication.class, args);
}
}

View File

@ -0,0 +1,20 @@
package com.ftptool.sync.config;
import org.springframework.boot.web.client.RestTemplateBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.web.client.RestTemplate;
import java.time.Duration;
@Configuration
public class AppConfig {
@Bean
public RestTemplate restTemplate(RestTemplateBuilder builder, ProdApiProperties prodApiProperties) {
return builder
.setConnectTimeout(Duration.ofMillis(prodApiProperties.getConnectTimeoutMs()))
.setReadTimeout(Duration.ofMillis(prodApiProperties.getReadTimeoutMs()))
.build();
}
}

View File

@ -0,0 +1,89 @@
package com.ftptool.sync.config;
import org.springframework.boot.context.properties.ConfigurationProperties;
@ConfigurationProperties(prefix = "ftp")
public class FtpProperties {
private String host;
private int port = 21;
private String username;
private String password;
private boolean passiveMode = true;
private String baseDir;
private int connectTimeoutMs = 10000;
private int dataTimeoutMs = 20000;
private int bufferSize = 8192;
public String getHost() {
return host;
}
public void setHost(String host) {
this.host = host;
}
public int getPort() {
return port;
}
public void setPort(int port) {
this.port = port;
}
public String getUsername() {
return username;
}
public void setUsername(String username) {
this.username = username;
}
public String getPassword() {
return password;
}
public void setPassword(String password) {
this.password = password;
}
public boolean isPassiveMode() {
return passiveMode;
}
public void setPassiveMode(boolean passiveMode) {
this.passiveMode = passiveMode;
}
public String getBaseDir() {
return baseDir;
}
public void setBaseDir(String baseDir) {
this.baseDir = baseDir;
}
public int getConnectTimeoutMs() {
return connectTimeoutMs;
}
public void setConnectTimeoutMs(int connectTimeoutMs) {
this.connectTimeoutMs = connectTimeoutMs;
}
public int getDataTimeoutMs() {
return dataTimeoutMs;
}
public void setDataTimeoutMs(int dataTimeoutMs) {
this.dataTimeoutMs = dataTimeoutMs;
}
public int getBufferSize() {
return bufferSize;
}
public void setBufferSize(int bufferSize) {
this.bufferSize = bufferSize;
}
}

View File

@ -0,0 +1,98 @@
package com.ftptool.sync.config;
import org.springframework.boot.context.properties.ConfigurationProperties;
@ConfigurationProperties(prefix = "git.repo")
public class GitRepoProperties {
private String localPath;
private String remoteUri;
private String username;
private String password;
private String scanBranch;
private String snapshotBranch;
private String commitAuthorName;
private String commitAuthorEmail;
private String commitMessagePrefix;
private boolean pullRebase;
public String getLocalPath() {
return localPath;
}
public void setLocalPath(String localPath) {
this.localPath = localPath;
}
public String getRemoteUri() {
return remoteUri;
}
public void setRemoteUri(String remoteUri) {
this.remoteUri = remoteUri;
}
public String getUsername() {
return username;
}
public void setUsername(String username) {
this.username = username;
}
public String getPassword() {
return password;
}
public void setPassword(String password) {
this.password = password;
}
public String getScanBranch() {
return scanBranch;
}
public void setScanBranch(String scanBranch) {
this.scanBranch = scanBranch;
}
public String getSnapshotBranch() {
return snapshotBranch;
}
public void setSnapshotBranch(String snapshotBranch) {
this.snapshotBranch = snapshotBranch;
}
public String getCommitAuthorName() {
return commitAuthorName;
}
public void setCommitAuthorName(String commitAuthorName) {
this.commitAuthorName = commitAuthorName;
}
public String getCommitAuthorEmail() {
return commitAuthorEmail;
}
public void setCommitAuthorEmail(String commitAuthorEmail) {
this.commitAuthorEmail = commitAuthorEmail;
}
public String getCommitMessagePrefix() {
return commitMessagePrefix;
}
public void setCommitMessagePrefix(String commitMessagePrefix) {
this.commitMessagePrefix = commitMessagePrefix;
}
public boolean isPullRebase() {
return pullRebase;
}
public void setPullRebase(boolean pullRebase) {
this.pullRebase = pullRebase;
}
}

View File

@ -0,0 +1,62 @@
package com.ftptool.sync.config;
import org.springframework.boot.context.properties.ConfigurationProperties;
@ConfigurationProperties(prefix = "prod.api")
public class ProdApiProperties {
private String baseUrl;
private String pushPath;
private String pullPath;
private String token;
private int connectTimeoutMs = 10000;
private int readTimeoutMs = 30000;
public String getBaseUrl() {
return baseUrl;
}
public void setBaseUrl(String baseUrl) {
this.baseUrl = baseUrl;
}
public String getPushPath() {
return pushPath;
}
public void setPushPath(String pushPath) {
this.pushPath = pushPath;
}
public String getPullPath() {
return pullPath;
}
public void setPullPath(String pullPath) {
this.pullPath = pullPath;
}
public String getToken() {
return token;
}
public void setToken(String token) {
this.token = token;
}
public int getConnectTimeoutMs() {
return connectTimeoutMs;
}
public void setConnectTimeoutMs(int connectTimeoutMs) {
this.connectTimeoutMs = connectTimeoutMs;
}
public int getReadTimeoutMs() {
return readTimeoutMs;
}
public void setReadTimeoutMs(int readTimeoutMs) {
this.readTimeoutMs = readTimeoutMs;
}
}

View File

@ -0,0 +1,80 @@
package com.ftptool.sync.config;
import org.springframework.boot.context.properties.ConfigurationProperties;
@ConfigurationProperties(prefix = "sync")
public class SyncProperties {
private String nodeId;
private String role;
private String workDir;
private String packageTempDir;
private String devToProdStagingDir;
private String prodToDevStagingDir;
private int maxRetryCount = 5;
private int ackScanBatchSize = 50;
public String getNodeId() {
return nodeId;
}
public void setNodeId(String nodeId) {
this.nodeId = nodeId;
}
public String getRole() {
return role;
}
public void setRole(String role) {
this.role = role;
}
public String getWorkDir() {
return workDir;
}
public void setWorkDir(String workDir) {
this.workDir = workDir;
}
public String getPackageTempDir() {
return packageTempDir;
}
public void setPackageTempDir(String packageTempDir) {
this.packageTempDir = packageTempDir;
}
public String getDevToProdStagingDir() {
return devToProdStagingDir;
}
public void setDevToProdStagingDir(String devToProdStagingDir) {
this.devToProdStagingDir = devToProdStagingDir;
}
public String getProdToDevStagingDir() {
return prodToDevStagingDir;
}
public void setProdToDevStagingDir(String prodToDevStagingDir) {
this.prodToDevStagingDir = prodToDevStagingDir;
}
public int getMaxRetryCount() {
return maxRetryCount;
}
public void setMaxRetryCount(int maxRetryCount) {
this.maxRetryCount = maxRetryCount;
}
public int getAckScanBatchSize() {
return ackScanBatchSize;
}
public void setAckScanBatchSize(int ackScanBatchSize) {
this.ackScanBatchSize = ackScanBatchSize;
}
}

View File

@ -0,0 +1,89 @@
package com.ftptool.sync.entity;
import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.GeneratedValue;
import javax.persistence.GenerationType;
import javax.persistence.Id;
import javax.persistence.PrePersist;
import javax.persistence.Table;
import java.time.LocalDateTime;
@Entity
@Table(name = "sync_ack")
public class SyncAck {
@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
@Column(name = "trace_id", nullable = false, length = 64)
private String traceId;
@Column(name = "ack_side", nullable = false, length = 32)
private String ackSide;
@Column(name = "ack_status", nullable = false, length = 32)
private String ackStatus;
@Column(name = "ack_time", nullable = false)
private LocalDateTime ackTime;
@Column(name = "remark", length = 500)
private String remark;
@PrePersist
public void prePersist() {
if (this.ackTime == null) {
this.ackTime = LocalDateTime.now();
}
}
public Long getId() {
return id;
}
public void setId(Long id) {
this.id = id;
}
public String getTraceId() {
return traceId;
}
public void setTraceId(String traceId) {
this.traceId = traceId;
}
public String getAckSide() {
return ackSide;
}
public void setAckSide(String ackSide) {
this.ackSide = ackSide;
}
public String getAckStatus() {
return ackStatus;
}
public void setAckStatus(String ackStatus) {
this.ackStatus = ackStatus;
}
public LocalDateTime getAckTime() {
return ackTime;
}
public void setAckTime(LocalDateTime ackTime) {
this.ackTime = ackTime;
}
public String getRemark() {
return remark;
}
public void setRemark(String remark) {
this.remark = remark;
}
}

View File

@ -0,0 +1,90 @@
package com.ftptool.sync.entity;
import com.ftptool.sync.model.SyncDirection;
import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.EnumType;
import javax.persistence.Enumerated;
import javax.persistence.GeneratedValue;
import javax.persistence.GenerationType;
import javax.persistence.Id;
import javax.persistence.PrePersist;
import javax.persistence.PreUpdate;
import javax.persistence.Table;
import javax.persistence.UniqueConstraint;
import java.time.LocalDateTime;
@Entity
@Table(name = "sync_checkpoint", uniqueConstraints = {
@UniqueConstraint(name = "uk_sync_checkpoint_direction", columnNames = "direction")
})
public class SyncCheckpoint {
@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
@Enumerated(EnumType.STRING)
@Column(name = "direction", nullable = false, length = 32)
private SyncDirection direction;
@Column(name = "last_success_version", length = 128)
private String lastSuccessVersion;
@Column(name = "last_success_hash", length = 128)
private String lastSuccessHash;
@Column(name = "updated_at", nullable = false)
private LocalDateTime updatedAt;
@PrePersist
public void prePersist() {
this.updatedAt = LocalDateTime.now();
}
@PreUpdate
public void preUpdate() {
this.updatedAt = LocalDateTime.now();
}
public Long getId() {
return id;
}
public void setId(Long id) {
this.id = id;
}
public SyncDirection getDirection() {
return direction;
}
public void setDirection(SyncDirection direction) {
this.direction = direction;
}
public String getLastSuccessVersion() {
return lastSuccessVersion;
}
public void setLastSuccessVersion(String lastSuccessVersion) {
this.lastSuccessVersion = lastSuccessVersion;
}
public String getLastSuccessHash() {
return lastSuccessHash;
}
public void setLastSuccessHash(String lastSuccessHash) {
this.lastSuccessHash = lastSuccessHash;
}
public LocalDateTime getUpdatedAt() {
return updatedAt;
}
public void setUpdatedAt(LocalDateTime updatedAt) {
this.updatedAt = updatedAt;
}
}

View File

@ -0,0 +1,169 @@
package com.ftptool.sync.entity;
import com.ftptool.sync.model.SyncDirection;
import com.ftptool.sync.model.SyncStatus;
import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.EnumType;
import javax.persistence.Enumerated;
import javax.persistence.GeneratedValue;
import javax.persistence.GenerationType;
import javax.persistence.Id;
import javax.persistence.Lob;
import javax.persistence.PrePersist;
import javax.persistence.PreUpdate;
import javax.persistence.Table;
import javax.persistence.UniqueConstraint;
import java.time.LocalDateTime;
@Entity
@Table(name = "sync_task", uniqueConstraints = {
@UniqueConstraint(name = "uk_sync_task_trace", columnNames = "trace_id"),
@UniqueConstraint(name = "uk_sync_task_business", columnNames = {"direction", "source_version", "content_hash"})
})
public class SyncTask {
@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
@Column(name = "trace_id", nullable = false, length = 64)
private String traceId;
@Enumerated(EnumType.STRING)
@Column(name = "direction", nullable = false, length = 32)
private SyncDirection direction;
@Column(name = "source_version", nullable = false, length = 128)
private String sourceVersion;
@Column(name = "content_hash", nullable = false, length = 128)
private String contentHash;
@Column(name = "package_name", length = 255)
private String packageName;
@Enumerated(EnumType.STRING)
@Column(name = "status", nullable = false, length = 32)
private SyncStatus status;
@Column(name = "retry_count", nullable = false)
private Integer retryCount;
@Lob
@Column(name = "error_msg")
private String errorMsg;
@Column(name = "created_at", nullable = false)
private LocalDateTime createdAt;
@Column(name = "updated_at", nullable = false)
private LocalDateTime updatedAt;
@PrePersist
public void prePersist() {
LocalDateTime now = LocalDateTime.now();
this.createdAt = now;
this.updatedAt = now;
if (this.retryCount == null) {
this.retryCount = 0;
}
if (this.status == null) {
this.status = SyncStatus.CREATED;
}
}
@PreUpdate
public void preUpdate() {
this.updatedAt = LocalDateTime.now();
}
public Long getId() {
return id;
}
public void setId(Long id) {
this.id = id;
}
public String getTraceId() {
return traceId;
}
public void setTraceId(String traceId) {
this.traceId = traceId;
}
public SyncDirection getDirection() {
return direction;
}
public void setDirection(SyncDirection direction) {
this.direction = direction;
}
public String getSourceVersion() {
return sourceVersion;
}
public void setSourceVersion(String sourceVersion) {
this.sourceVersion = sourceVersion;
}
public String getContentHash() {
return contentHash;
}
public void setContentHash(String contentHash) {
this.contentHash = contentHash;
}
public String getPackageName() {
return packageName;
}
public void setPackageName(String packageName) {
this.packageName = packageName;
}
public SyncStatus getStatus() {
return status;
}
public void setStatus(SyncStatus status) {
this.status = status;
}
public Integer getRetryCount() {
return retryCount;
}
public void setRetryCount(Integer retryCount) {
this.retryCount = retryCount;
}
public String getErrorMsg() {
return errorMsg;
}
public void setErrorMsg(String errorMsg) {
this.errorMsg = errorMsg;
}
public LocalDateTime getCreatedAt() {
return createdAt;
}
public void setCreatedAt(LocalDateTime createdAt) {
this.createdAt = createdAt;
}
public LocalDateTime getUpdatedAt() {
return updatedAt;
}
public void setUpdatedAt(LocalDateTime updatedAt) {
this.updatedAt = updatedAt;
}
}

View File

@ -0,0 +1,22 @@
package com.ftptool.sync.job;
import com.ftptool.sync.orchestrator.DevSyncCoordinator;
import org.springframework.context.annotation.Profile;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Component;
@Component
@Profile("dev-agent")
public class DevAckScanJob {
private final DevSyncCoordinator devSyncCoordinator;
public DevAckScanJob(DevSyncCoordinator devSyncCoordinator) {
this.devSyncCoordinator = devSyncCoordinator;
}
@Scheduled(cron = "${sync.jobs.dev-ack-scan.cron}")
public void execute() {
devSyncCoordinator.scanProdAcks();
}
}

View File

@ -0,0 +1,22 @@
package com.ftptool.sync.job;
import com.ftptool.sync.orchestrator.DevSyncCoordinator;
import org.springframework.context.annotation.Profile;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Component;
@Component
@Profile("dev-agent")
public class DevConsumeProdPackageJob {
private final DevSyncCoordinator devSyncCoordinator;
public DevConsumeProdPackageJob(DevSyncCoordinator devSyncCoordinator) {
this.devSyncCoordinator = devSyncCoordinator;
}
@Scheduled(cron = "${sync.jobs.dev-consume-prod-package.cron}")
public void execute() {
devSyncCoordinator.consumeProdPackages();
}
}

View File

@ -0,0 +1,22 @@
package com.ftptool.sync.job;
import com.ftptool.sync.orchestrator.DevSyncCoordinator;
import org.springframework.context.annotation.Profile;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Component;
@Component
@Profile("dev-agent")
public class DevGitScanJob {
private final DevSyncCoordinator devSyncCoordinator;
public DevGitScanJob(DevSyncCoordinator devSyncCoordinator) {
this.devSyncCoordinator = devSyncCoordinator;
}
@Scheduled(cron = "${sync.jobs.dev-git-scan.cron}")
public void execute() {
devSyncCoordinator.scanGitAndStagePackage();
}
}

View File

@ -0,0 +1,22 @@
package com.ftptool.sync.job;
import com.ftptool.sync.orchestrator.ProdSyncCoordinator;
import org.springframework.context.annotation.Profile;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Component;
@Component
@Profile("prod-agent")
public class ProdAckScanJob {
private final ProdSyncCoordinator prodSyncCoordinator;
public ProdAckScanJob(ProdSyncCoordinator prodSyncCoordinator) {
this.prodSyncCoordinator = prodSyncCoordinator;
}
@Scheduled(cron = "${sync.jobs.prod-ack-scan.cron}")
public void execute() {
prodSyncCoordinator.scanDevAcks();
}
}

View File

@ -0,0 +1,22 @@
package com.ftptool.sync.job;
import com.ftptool.sync.orchestrator.ProdSyncCoordinator;
import org.springframework.context.annotation.Profile;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Component;
@Component
@Profile("prod-agent")
public class ProdConsumeDevPackageJob {
private final ProdSyncCoordinator prodSyncCoordinator;
public ProdConsumeDevPackageJob(ProdSyncCoordinator prodSyncCoordinator) {
this.prodSyncCoordinator = prodSyncCoordinator;
}
@Scheduled(cron = "${sync.jobs.prod-consume-dev-package.cron}")
public void execute() {
prodSyncCoordinator.consumeDevPackages();
}
}

View File

@ -0,0 +1,22 @@
package com.ftptool.sync.job;
import com.ftptool.sync.orchestrator.ProdSyncCoordinator;
import org.springframework.context.annotation.Profile;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Component;
@Component
@Profile("prod-agent")
public class ProdPullConfigJob {
private final ProdSyncCoordinator prodSyncCoordinator;
public ProdPullConfigJob(ProdSyncCoordinator prodSyncCoordinator) {
this.prodSyncCoordinator = prodSyncCoordinator;
}
@Scheduled(cron = "${sync.jobs.prod-pull-config.cron}")
public void execute() {
prodSyncCoordinator.pullProdConfigAndStagePackage();
}
}

View File

@ -0,0 +1,6 @@
package com.ftptool.sync.model;
public enum SyncDirection {
DEV_TO_PROD,
PROD_TO_DEV
}

View File

@ -0,0 +1,7 @@
package com.ftptool.sync.model;
public enum SyncRole {
DEV,
PROD,
UNSET
}

View File

@ -0,0 +1,10 @@
package com.ftptool.sync.model;
public enum SyncStatus {
CREATED,
STAGED,
UPLOADED,
CONSUMING,
SUCCESS,
FAILED
}

View File

@ -0,0 +1,55 @@
package com.ftptool.sync.orchestrator;
import com.ftptool.sync.config.FtpProperties;
import com.ftptool.sync.config.GitRepoProperties;
import com.ftptool.sync.config.SyncProperties;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.context.annotation.Profile;
import org.springframework.stereotype.Service;
@Service
@Profile("dev-agent")
public class DevSyncCoordinator {
private static final Logger log = LoggerFactory.getLogger(DevSyncCoordinator.class);
private final SyncProperties syncProperties;
private final GitRepoProperties gitRepoProperties;
private final FtpProperties ftpProperties;
public DevSyncCoordinator(
SyncProperties syncProperties,
GitRepoProperties gitRepoProperties,
FtpProperties ftpProperties
) {
this.syncProperties = syncProperties;
this.gitRepoProperties = gitRepoProperties;
this.ftpProperties = ftpProperties;
}
public void scanGitAndStagePackage() {
log.info(
"DEV scan tick. nodeId={}, branch={}, localRepo={}, ftpBaseDir={}",
syncProperties.getNodeId(),
gitRepoProperties.getScanBranch(),
gitRepoProperties.getLocalPath(),
ftpProperties.getBaseDir()
);
log.info("TODO implement: Git pull -> package build -> upload to FTP dev-to-prod/out");
}
public void consumeProdPackages() {
log.info(
"DEV consume tick. snapshotBranch={}, stagingDir={}",
gitRepoProperties.getSnapshotBranch(),
syncProperties.getProdToDevStagingDir()
);
log.info("TODO implement: download prod-to-dev package -> write Git -> commit/push");
}
public void scanProdAcks() {
log.info("DEV ack scan tick. batchSize={}", syncProperties.getAckScanBatchSize());
log.info("TODO implement: read dev-to-prod/ack and update sync_task state");
}
}

View File

@ -0,0 +1,55 @@
package com.ftptool.sync.orchestrator;
import com.ftptool.sync.config.FtpProperties;
import com.ftptool.sync.config.ProdApiProperties;
import com.ftptool.sync.config.SyncProperties;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.context.annotation.Profile;
import org.springframework.stereotype.Service;
@Service
@Profile("prod-agent")
public class ProdSyncCoordinator {
private static final Logger log = LoggerFactory.getLogger(ProdSyncCoordinator.class);
private final SyncProperties syncProperties;
private final FtpProperties ftpProperties;
private final ProdApiProperties prodApiProperties;
public ProdSyncCoordinator(
SyncProperties syncProperties,
FtpProperties ftpProperties,
ProdApiProperties prodApiProperties
) {
this.syncProperties = syncProperties;
this.ftpProperties = ftpProperties;
this.prodApiProperties = prodApiProperties;
}
public void consumeDevPackages() {
log.info(
"PROD consume tick. nodeId={}, ftpBaseDir={}, pushPath={}",
syncProperties.getNodeId(),
ftpProperties.getBaseDir(),
prodApiProperties.getPushPath()
);
log.info("TODO implement: download dev-to-prod package -> validate -> call prod push API");
}
public void pullProdConfigAndStagePackage() {
log.info(
"PROD pull tick. apiBaseUrl={}, pullPath={}, stagingDir={}",
prodApiProperties.getBaseUrl(),
prodApiProperties.getPullPath(),
syncProperties.getProdToDevStagingDir()
);
log.info("TODO implement: call prod pull API -> build package -> upload to FTP prod-to-dev/out");
}
public void scanDevAcks() {
log.info("PROD ack scan tick. batchSize={}", syncProperties.getAckScanBatchSize());
log.info("TODO implement: read prod-to-dev/ack and update sync_task state");
}
}

View File

@ -0,0 +1,11 @@
package com.ftptool.sync.repository;
import com.ftptool.sync.entity.SyncAck;
import org.springframework.data.jpa.repository.JpaRepository;
import java.util.List;
public interface SyncAckRepository extends JpaRepository<SyncAck, Long> {
List<SyncAck> findTop50ByTraceIdOrderByAckTimeDesc(String traceId);
}

View File

@ -0,0 +1,12 @@
package com.ftptool.sync.repository;
import com.ftptool.sync.entity.SyncCheckpoint;
import com.ftptool.sync.model.SyncDirection;
import org.springframework.data.jpa.repository.JpaRepository;
import java.util.Optional;
public interface SyncCheckpointRepository extends JpaRepository<SyncCheckpoint, Long> {
Optional<SyncCheckpoint> findByDirection(SyncDirection direction);
}

View File

@ -0,0 +1,24 @@
package com.ftptool.sync.repository;
import com.ftptool.sync.entity.SyncTask;
import com.ftptool.sync.model.SyncDirection;
import org.springframework.data.jpa.repository.JpaRepository;
import java.util.Optional;
public interface SyncTaskRepository extends JpaRepository<SyncTask, Long> {
Optional<SyncTask> findByTraceId(String traceId);
Optional<SyncTask> findByDirectionAndSourceVersionAndContentHash(
SyncDirection direction,
String sourceVersion,
String contentHash
);
boolean existsByDirectionAndSourceVersionAndContentHash(
SyncDirection direction,
String sourceVersion,
String contentHash
);
}

View File

@ -0,0 +1,33 @@
package com.ftptool.sync.service;
import com.ftptool.sync.entity.SyncAck;
import com.ftptool.sync.repository.SyncAckRepository;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
import java.util.List;
@Service
public class AckService {
private final SyncAckRepository syncAckRepository;
public AckService(SyncAckRepository syncAckRepository) {
this.syncAckRepository = syncAckRepository;
}
@Transactional
public SyncAck recordAck(String traceId, String ackSide, String ackStatus, String remark) {
SyncAck syncAck = new SyncAck();
syncAck.setTraceId(traceId);
syncAck.setAckSide(ackSide);
syncAck.setAckStatus(ackStatus);
syncAck.setRemark(remark);
return syncAckRepository.save(syncAck);
}
@Transactional(readOnly = true)
public List<SyncAck> findLatestByTraceId(String traceId) {
return syncAckRepository.findTop50ByTraceIdOrderByAckTimeDesc(traceId);
}
}

View File

@ -0,0 +1,34 @@
package com.ftptool.sync.service;
import com.ftptool.sync.entity.SyncCheckpoint;
import com.ftptool.sync.model.SyncDirection;
import com.ftptool.sync.repository.SyncCheckpointRepository;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
import java.util.Optional;
@Service
public class CheckpointService {
private final SyncCheckpointRepository syncCheckpointRepository;
public CheckpointService(SyncCheckpointRepository syncCheckpointRepository) {
this.syncCheckpointRepository = syncCheckpointRepository;
}
@Transactional(readOnly = true)
public Optional<SyncCheckpoint> getCheckpoint(SyncDirection direction) {
return syncCheckpointRepository.findByDirection(direction);
}
@Transactional
public SyncCheckpoint saveCheckpoint(SyncDirection direction, String version, String hash) {
SyncCheckpoint checkpoint = syncCheckpointRepository.findByDirection(direction)
.orElseGet(SyncCheckpoint::new);
checkpoint.setDirection(direction);
checkpoint.setLastSuccessVersion(version);
checkpoint.setLastSuccessHash(hash);
return syncCheckpointRepository.save(checkpoint);
}
}

View File

@ -0,0 +1,69 @@
package com.ftptool.sync.service;
import com.ftptool.sync.entity.SyncTask;
import com.ftptool.sync.model.SyncDirection;
import com.ftptool.sync.model.SyncStatus;
import com.ftptool.sync.repository.SyncTaskRepository;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
import java.util.Optional;
import java.util.UUID;
@Service
public class SyncTaskService {
private final SyncTaskRepository syncTaskRepository;
public SyncTaskService(SyncTaskRepository syncTaskRepository) {
this.syncTaskRepository = syncTaskRepository;
}
@Transactional
public SyncTask createOrLoadTask(SyncDirection direction, String sourceVersion, String contentHash, String packageName) {
Optional<SyncTask> existing = syncTaskRepository.findByDirectionAndSourceVersionAndContentHash(
direction, sourceVersion, contentHash
);
if (existing.isPresent()) {
return existing.get();
}
SyncTask task = new SyncTask();
task.setTraceId(UUID.randomUUID().toString().replace("-", ""));
task.setDirection(direction);
task.setSourceVersion(sourceVersion);
task.setContentHash(contentHash);
task.setPackageName(packageName);
task.setStatus(SyncStatus.CREATED);
return syncTaskRepository.save(task);
}
@Transactional(readOnly = true)
public Optional<SyncTask> findByTraceId(String traceId) {
return syncTaskRepository.findByTraceId(traceId);
}
@Transactional
public void markStatus(String traceId, SyncStatus status, String errorMsg) {
syncTaskRepository.findByTraceId(traceId).ifPresent(task -> {
task.setStatus(status);
task.setErrorMsg(errorMsg);
syncTaskRepository.save(task);
});
}
@Transactional
public void increaseRetryCount(String traceId, String errorMsg) {
syncTaskRepository.findByTraceId(traceId).ifPresent(task -> {
int current = task.getRetryCount() == null ? 0 : task.getRetryCount();
task.setRetryCount(current + 1);
task.setErrorMsg(errorMsg);
syncTaskRepository.save(task);
});
}
@Transactional(readOnly = true)
public boolean existsProcessed(SyncDirection direction, String sourceVersion, String contentHash) {
return syncTaskRepository.existsByDirectionAndSourceVersionAndContentHash(direction, sourceVersion, contentHash);
}
}

View File

@ -0,0 +1,19 @@
spring.config.activate.on-profile=dev-agent
server.port=8081
sync.node-id=dev-agent-01
sync.role=DEV
# DEV side pulls Git, stages packages to FTP, and consumes prod snapshots
sync.jobs.dev-git-scan.cron=0 */2 * * * *
sync.jobs.dev-consume-prod-package.cron=30 */1 * * * *
sync.jobs.dev-ack-scan.cron=45 */1 * * * *
# Example overrides
ftp.host=ftp-a.example.com
ftp.port=21
ftp.username=dev_sync_user
ftp.password=change-me
git.repo.remote-uri=https://git.example.com/config.git
git.repo.scan-branch=config-dev-main
git.repo.snapshot-branch=config-prod-snapshot

View File

@ -0,0 +1,20 @@
spring.config.activate.on-profile=prod-agent
server.port=8082
sync.node-id=prod-agent-01
sync.role=PROD
# PROD side consumes dev packages, calls pull/push APIs, and stages snapshots
sync.jobs.prod-consume-dev-package.cron=0 */1 * * * *
sync.jobs.prod-pull-config.cron=20 */2 * * * *
sync.jobs.prod-ack-scan.cron=40 */1 * * * *
# Example overrides
ftp.host=ftp-a.example.com
ftp.port=21
ftp.username=prod_sync_user
ftp.password=change-me
prod.api.base-url=https://prod.example.com
prod.api.push-path=/api/config/push
prod.api.pull-path=/api/config/pull
prod.api.token=change-me

View File

@ -0,0 +1,62 @@
# Common application settings
spring.application.name=ftp-sync-tool
server.port=8080
spring.main.banner-mode=off
# H2 file mode to persist checkpoints and retry state
spring.datasource.url=jdbc:h2:file:./data/ftp-sync-tool-db;AUTO_SERVER=TRUE;MODE=MYSQL
spring.datasource.driver-class-name=org.h2.Driver
spring.datasource.username=sa
spring.datasource.password=
spring.jpa.hibernate.ddl-auto=none
spring.jpa.open-in-view=false
spring.jpa.show-sql=false
spring.sql.init.mode=always
spring.sql.init.schema-locations=classpath:schema.sql
spring.h2.console.enabled=true
spring.h2.console.path=/h2-console
management.endpoints.web.exposure.include=health,info
# Common sync settings
sync.node-id=default-node
sync.role=UNSET
sync.work-dir=./work
sync.package-temp-dir=./work/package
sync.dev-to-prod-staging-dir=./work/staging/dev-to-prod
sync.prod-to-dev-staging-dir=./work/staging/prod-to-dev
sync.max-retry-count=5
sync.ack-scan-batch-size=50
# FTP defaults
ftp.host=127.0.0.1
ftp.port=21
ftp.username=replace-me
ftp.password=replace-me
ftp.passive-mode=true
ftp.base-dir=/sync
ftp.connect-timeout-ms=10000
ftp.data-timeout-ms=20000
ftp.buffer-size=8192
# Git defaults
git.repo.local-path=./work/git/config-repo
git.repo.remote-uri=https://git.example.com/config.git
git.repo.username=replace-me
git.repo.password=replace-me
git.repo.scan-branch=config-dev-main
git.repo.snapshot-branch=config-prod-snapshot
git.repo.commit-author-name=ftp-sync-bot
git.repo.commit-author-email=ftp-sync-bot@example.com
git.repo.commit-message-prefix=sync(prod->git)
git.repo.pull-rebase=false
# Production API defaults
prod.api.base-url=https://prod.example.com
prod.api.push-path=/api/config/push
prod.api.pull-path=/api/config/pull
prod.api.token=replace-me
prod.api.connect-timeout-ms=10000
prod.api.read-timeout-ms=30000
# Keep profile specific cron expressions in application-<profile>.properties

View File

@ -0,0 +1,38 @@
create table if not exists sync_checkpoint (
id bigint generated by default as identity primary key,
direction varchar(32) not null,
last_success_version varchar(128),
last_success_hash varchar(128),
updated_at timestamp not null,
constraint uk_sync_checkpoint_direction unique (direction)
);
create table if not exists sync_task (
id bigint generated by default as identity primary key,
trace_id varchar(64) not null,
direction varchar(32) not null,
source_version varchar(128) not null,
content_hash varchar(128) not null,
package_name varchar(255),
status varchar(32) not null,
retry_count int not null default 0,
error_msg clob,
created_at timestamp not null,
updated_at timestamp not null,
constraint uk_sync_task_trace unique (trace_id),
constraint uk_sync_task_business unique (direction, source_version, content_hash)
);
create index if not exists idx_sync_task_status on sync_task (status);
create index if not exists idx_sync_task_direction on sync_task (direction);
create table if not exists sync_ack (
id bigint generated by default as identity primary key,
trace_id varchar(64) not null,
ack_side varchar(32) not null,
ack_status varchar(32) not null,
ack_time timestamp not null,
remark varchar(500)
);
create index if not exists idx_sync_ack_trace on sync_ack (trace_id);