refactor: 收敛为 Git 直连同步架构并完成收尾
- 将整体方案从 FTP 中转双端代理重构为单 prod-agent 的 Git 直连架构 - 重写主设计文档、详细设计文档和补充方案文档,统一新架构口径 - 新增 GitDirectSyncToolApplication 并调整 Maven 坐标与运行命名 - 清理 application.properties、SyncProperties 和 schema.sql 中的 FTP/ACK 遗留配置与表结构 - 将生产侧主流程收敛为 Git -> PROD 与 PROD -> Git 两条正式链路 - 新增 SyncManagementController,提供最近同步状态与失败任务查询接口 - 补充 ProdSyncCoordinatorIntegrationTest,覆盖两条主链路的最小集成测试与幂等行为 - 为主链路代码补充必要中文注释,提升可读性 - 删除旧架构源码主体并同步修正文档中的过期引用 - 完成编译与测试验证
This commit is contained in:
parent
49c9155533
commit
0162117ae4
@ -378,9 +378,9 @@ sync-tool
|
|||||||
|
|
||||||
已退出主运行面:
|
已退出主运行面:
|
||||||
|
|
||||||
- `FtpClientService`
|
|
||||||
- FTP 包上传下载逻辑
|
- FTP 包上传下载逻辑
|
||||||
- FTP ACK 逻辑
|
- FTP ACK 逻辑
|
||||||
|
- 双端代理运行路径
|
||||||
|
|
||||||
## 16. 定时任务建议
|
## 16. 定时任务建议
|
||||||
|
|
||||||
@ -481,6 +481,6 @@ sync-tool
|
|||||||
|
|
||||||
1. 先确认生产环境对开发 Git 是否具备推送权限
|
1. 先确认生产环境对开发 Git 是否具备推送权限
|
||||||
2. 确认生产 `push/pull` 接口最终协议
|
2. 确认生产 `push/pull` 接口最终协议
|
||||||
3. 在文件系统允许时物理删除退役占位类
|
3. 删除退役标记文件 `application-dev-agent.properties`
|
||||||
4. 将工程命名中残留的 `ftp` 语义继续清理
|
4. 将工程命名中残留的 `ftp` 语义继续清理
|
||||||
5. 补充新的 `application-prod-agent.properties` 配置说明
|
5. 补充新的 `application-prod-agent.properties` 配置说明
|
||||||
|
|||||||
@ -114,9 +114,8 @@ Git 配置:
|
|||||||
|
|
||||||
当前状态:
|
当前状态:
|
||||||
|
|
||||||
- `FtpClientService` 已降为占位类
|
|
||||||
- FTP 调度主路径已退出运行面
|
- FTP 调度主路径已退出运行面
|
||||||
- 旧 FTP 类仍保留在源码树中,但不再作为正式能力
|
- FTP 相关主类已从当前源码树中移除
|
||||||
|
|
||||||
## 6. H2 状态设计
|
## 6. H2 状态设计
|
||||||
|
|
||||||
@ -174,25 +173,19 @@ Git 配置:
|
|||||||
|
|
||||||
### 7.4 当前遗留占位类
|
### 7.4 当前遗留占位类
|
||||||
|
|
||||||
- [ProdConsumeDevPackageJob.java](e:/AIcoding/FtpTool/src/main/java/com/ftptool/sync/job/ProdConsumeDevPackageJob.java)
|
这批旧调度类已经从当前源码树中删除:
|
||||||
- [ProdPullConfigJob.java](e:/AIcoding/FtpTool/src/main/java/com/ftptool/sync/job/ProdPullConfigJob.java)
|
|
||||||
- [DevGitScanJob.java](e:/AIcoding/FtpTool/src/main/java/com/ftptool/sync/job/DevGitScanJob.java)
|
|
||||||
- [DevConsumeProdPackageJob.java](e:/AIcoding/FtpTool/src/main/java/com/ftptool/sync/job/DevConsumeProdPackageJob.java)
|
|
||||||
- [DevAckScanJob.java](e:/AIcoding/FtpTool/src/main/java/com/ftptool/sync/job/DevAckScanJob.java)
|
|
||||||
|
|
||||||
说明:
|
说明:
|
||||||
|
|
||||||
- 这些类目前保留为兼容占位
|
|
||||||
- 已不再作为正式运行入口
|
- 已不再作为正式运行入口
|
||||||
- 由于当前环境删除权限受限,暂时保留为空占位实现
|
- 当前代码树只保留 Git 直连架构需要的正式 job
|
||||||
|
|
||||||
### 7.5 当前遗留代码
|
### 7.5 当前遗留代码
|
||||||
|
|
||||||
以下内容仍然存在于代码库,但属于旧架构遗留:
|
以下内容仍然存在于代码库,但属于旧架构遗留:
|
||||||
|
|
||||||
- `dev-agent` 相关 job/coordinator
|
- `application-dev-agent.properties` 退役标记文件
|
||||||
- `FtpClientService`
|
- 少量文件名或文档中的 `ftp` 语义残留
|
||||||
- ACK 文件相关模型和服务
|
|
||||||
|
|
||||||
这些不是当前推荐运行路径。
|
这些不是当前推荐运行路径。
|
||||||
|
|
||||||
@ -234,6 +227,18 @@ Git 配置:
|
|||||||
|
|
||||||
- [prod-api-v1.md](e:/AIcoding/FtpTool/docs/prod-api-v1.md)
|
- [prod-api-v1.md](e:/AIcoding/FtpTool/docs/prod-api-v1.md)
|
||||||
|
|
||||||
|
## 9.1 管理接口
|
||||||
|
|
||||||
|
当前已新增一组只读管理接口,用于查看最近同步状态和失败任务:
|
||||||
|
|
||||||
|
- `GET /api/admin/sync/overview`
|
||||||
|
- `GET /api/admin/sync/tasks/recent`
|
||||||
|
- `GET /api/admin/sync/tasks/failed`
|
||||||
|
|
||||||
|
对应实现:
|
||||||
|
|
||||||
|
- [SyncManagementController.java](e:/AIcoding/FtpTool/src/main/java/com/ftptool/sync/web/SyncManagementController.java)
|
||||||
|
|
||||||
## 10. 当前主要风险
|
## 10. 当前主要风险
|
||||||
|
|
||||||
### 10.1 Git 写权限
|
### 10.1 Git 写权限
|
||||||
@ -247,7 +252,8 @@ Git 配置:
|
|||||||
当前状态:
|
当前状态:
|
||||||
|
|
||||||
- 文件名和类名仍可能误导维护者
|
- 文件名和类名仍可能误导维护者
|
||||||
- 极少量退役文件仍保留在源码树中
|
- 旧架构源码主体已经删除
|
||||||
|
- 主要剩余问题转为命名和文档口径统一
|
||||||
|
|
||||||
### 10.3 双向同步闭环
|
### 10.3 双向同步闭环
|
||||||
|
|
||||||
@ -258,7 +264,7 @@ Git 配置:
|
|||||||
建议按下面顺序继续:
|
建议按下面顺序继续:
|
||||||
|
|
||||||
1. 删除或隔离 `dev-agent` 运行路径
|
1. 删除或隔离 `dev-agent` 运行路径
|
||||||
2. 在文件系统允许时物理删除退役占位类
|
2. 删除退役标记文件 `application-dev-agent.properties`
|
||||||
3. 统一清理残留的 `ftp` 命名
|
3. 统一清理残留的 `ftp` 命名
|
||||||
4. 补充管理接口和健康检查接口
|
4. 补充管理接口和健康检查接口
|
||||||
5. 增加集成测试
|
5. 增加集成测试
|
||||||
|
|||||||
@ -1,4 +1,4 @@
|
|||||||
# 基于 Git 直连的配置双向同步工具设计方案
|
# Git 直连架构补充方案
|
||||||
|
|
||||||
## 1. 背景变化
|
## 1. 背景变化
|
||||||
|
|
||||||
@ -94,7 +94,7 @@
|
|||||||
|
|
||||||
这里有一个必须先确认的关键点:
|
这里有一个必须先确认的关键点:
|
||||||
|
|
||||||
- **生产环境对开发 Git 是否有写权限**
|
- **生产环境对开发 Git 是否有 push 权限**
|
||||||
|
|
||||||
如果只有读权限,没有 push 权限,那么第二条链路“生产 -> 开发 Git”无法闭环。
|
如果只有读权限,没有 push 权限,那么第二条链路“生产 -> 开发 Git”无法闭环。
|
||||||
|
|
||||||
@ -109,8 +109,8 @@
|
|||||||
|
|
||||||
同步规则:
|
同步规则:
|
||||||
|
|
||||||
- `Git -> PROD` 只读取 `config-dev-main`
|
- `Git -> PROD` 只读 `config-dev-main`
|
||||||
- `PROD -> Git` 只写入 `config-prod-snapshot`
|
- `PROD -> Git` 只写 `config-prod-snapshot`
|
||||||
|
|
||||||
这样做的好处:
|
这样做的好处:
|
||||||
|
|
||||||
@ -127,10 +127,12 @@
|
|||||||
- `sync_checkpoint`
|
- `sync_checkpoint`
|
||||||
- `sync_task`
|
- `sync_task`
|
||||||
|
|
||||||
`sync_ack` 在新架构下不再承担跨节点 ACK 作用,可以:
|
`sync_ack` 在新架构下不再承担跨节点 ACK 作用。
|
||||||
|
|
||||||
- 继续保留为接口调用结果日志表
|
当前建议:
|
||||||
- 或后续简化下线
|
|
||||||
|
- 已退出主 schema
|
||||||
|
- 如后续需要审计,可独立恢复
|
||||||
|
|
||||||
## 8. 幂等设计
|
## 8. 幂等设计
|
||||||
|
|
||||||
@ -183,7 +185,7 @@ direction + sourceVersion + contentHash
|
|||||||
|
|
||||||
生产环境访问 Git 的账号建议最小权限化:
|
生产环境访问 Git 的账号建议最小权限化:
|
||||||
|
|
||||||
- 对 `config-dev-main` 至少有读权限
|
- 对 `config-dev-main` 有读权限
|
||||||
- 对 `config-prod-snapshot` 有写权限
|
- 对 `config-prod-snapshot` 有写权限
|
||||||
|
|
||||||
更理想的做法:
|
更理想的做法:
|
||||||
@ -205,26 +207,30 @@ direction + sourceVersion + contentHash
|
|||||||
- H2 表结构
|
- H2 表结构
|
||||||
- 定时任务框架
|
- 定时任务框架
|
||||||
|
|
||||||
### 11.2 应该逐步下线的部分
|
### 11.2 已退出主运行面的部分
|
||||||
|
|
||||||
- `FtpClientService`
|
|
||||||
- FTP 目录配置
|
- FTP 目录配置
|
||||||
- 包上传/下载流程
|
- 包上传/下载流程
|
||||||
- ACK 文件机制
|
- ACK 文件机制
|
||||||
- 双端部署假设
|
- 双端部署假设
|
||||||
|
|
||||||
### 11.3 推荐重构方向
|
### 11.3 当前代码状态
|
||||||
|
|
||||||
把系统收敛为:
|
当前代码已经收敛为:
|
||||||
|
|
||||||
- 一个 `prod-agent`
|
- 一个正式运行角色 `prod-agent`
|
||||||
- 两个核心任务
|
- 两个正式调度任务
|
||||||
|
|
||||||
即:
|
即:
|
||||||
|
|
||||||
1. `GitToProdSyncJob`
|
1. `GitToProdSyncJob`
|
||||||
2. `ProdToGitSnapshotJob`
|
2. `ProdToGitSnapshotJob`
|
||||||
|
|
||||||
|
旧架构源码主体已经删除,当前剩余问题主要是:
|
||||||
|
|
||||||
|
- 个别资源文件仍保留退役标记
|
||||||
|
- 工程内少量 `ftp` 命名仍待统一
|
||||||
|
|
||||||
## 12. 结论
|
## 12. 结论
|
||||||
|
|
||||||
现在最合理的做法不是“在旧 FTP 方案上修修补补”,而是直接把架构收敛成:
|
现在最合理的做法不是“在旧 FTP 方案上修修补补”,而是直接把架构收敛成:
|
||||||
@ -242,6 +248,6 @@ direction + sourceVersion + contentHash
|
|||||||
|
|
||||||
1. 先确认生产环境是否具备开发 Git 的 push 权限
|
1. 先确认生产环境是否具备开发 Git 的 push 权限
|
||||||
2. 确认生产 `push/pull` 接口最终协议
|
2. 确认生产 `push/pull` 接口最终协议
|
||||||
3. 重写主设计文档和详细设计文档,去掉 FTP 相关内容
|
3. 删除退役标记文件 `application-dev-agent.properties`
|
||||||
4. 收敛代码为单 `prod-agent`
|
4. 继续清理工程中残留的 `ftp` 命名
|
||||||
5. 删除 FTP 相关配置和服务
|
5. 补充健康检查和管理能力
|
||||||
|
|||||||
6
pom.xml
6
pom.xml
@ -11,10 +11,10 @@
|
|||||||
<relativePath/>
|
<relativePath/>
|
||||||
</parent>
|
</parent>
|
||||||
|
|
||||||
<groupId>com.ftptool</groupId>
|
<groupId>com.gitdirect</groupId>
|
||||||
<artifactId>ftp-sync-tool</artifactId>
|
<artifactId>git-direct-sync-tool</artifactId>
|
||||||
<version>0.0.1-SNAPSHOT</version>
|
<version>0.0.1-SNAPSHOT</version>
|
||||||
<name>ftp-sync-tool</name>
|
<name>git-direct-sync-tool</name>
|
||||||
<description>Git direct based configuration sync tool</description>
|
<description>Git direct based configuration sync tool</description>
|
||||||
|
|
||||||
<properties>
|
<properties>
|
||||||
|
|||||||
@ -17,9 +17,9 @@ import org.springframework.scheduling.annotation.EnableScheduling;
|
|||||||
GitRepoProperties.class,
|
GitRepoProperties.class,
|
||||||
ProdApiProperties.class
|
ProdApiProperties.class
|
||||||
})
|
})
|
||||||
public class FtpSyncToolApplication {
|
public class GitDirectSyncToolApplication {
|
||||||
|
|
||||||
public static void main(String[] args) {
|
public static void main(String[] args) {
|
||||||
SpringApplication.run(FtpSyncToolApplication.class, args);
|
SpringApplication.run(GitDirectSyncToolApplication.class, args);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@ -1,5 +0,0 @@
|
|||||||
package com.ftptool.sync.config;
|
|
||||||
|
|
||||||
@Deprecated
|
|
||||||
public final class FtpProperties {
|
|
||||||
}
|
|
||||||
@ -1,5 +0,0 @@
|
|||||||
package com.ftptool.sync.entity;
|
|
||||||
|
|
||||||
@Deprecated
|
|
||||||
public final class SyncAck {
|
|
||||||
}
|
|
||||||
@ -1,5 +0,0 @@
|
|||||||
package com.ftptool.sync.job;
|
|
||||||
|
|
||||||
@Deprecated
|
|
||||||
public final class DevAckScanJob {
|
|
||||||
}
|
|
||||||
@ -1,5 +0,0 @@
|
|||||||
package com.ftptool.sync.job;
|
|
||||||
|
|
||||||
@Deprecated
|
|
||||||
public final class DevConsumeProdPackageJob {
|
|
||||||
}
|
|
||||||
@ -1,5 +0,0 @@
|
|||||||
package com.ftptool.sync.job;
|
|
||||||
|
|
||||||
@Deprecated
|
|
||||||
public final class DevGitScanJob {
|
|
||||||
}
|
|
||||||
@ -1,5 +0,0 @@
|
|||||||
package com.ftptool.sync.job;
|
|
||||||
|
|
||||||
@Deprecated
|
|
||||||
public final class ProdAckScanJob {
|
|
||||||
}
|
|
||||||
@ -1,5 +0,0 @@
|
|||||||
package com.ftptool.sync.job;
|
|
||||||
|
|
||||||
@Deprecated
|
|
||||||
public final class ProdConsumeDevPackageJob {
|
|
||||||
}
|
|
||||||
@ -1,5 +0,0 @@
|
|||||||
package com.ftptool.sync.job;
|
|
||||||
|
|
||||||
@Deprecated
|
|
||||||
public final class ProdPullConfigJob {
|
|
||||||
}
|
|
||||||
@ -1,5 +0,0 @@
|
|||||||
package com.ftptool.sync.model;
|
|
||||||
|
|
||||||
@Deprecated
|
|
||||||
public final class RemoteFileInfo {
|
|
||||||
}
|
|
||||||
@ -1,5 +0,0 @@
|
|||||||
package com.ftptool.sync.model;
|
|
||||||
|
|
||||||
@Deprecated
|
|
||||||
public final class SyncAckFile {
|
|
||||||
}
|
|
||||||
@ -1,5 +0,0 @@
|
|||||||
package com.ftptool.sync.orchestrator;
|
|
||||||
|
|
||||||
@Deprecated
|
|
||||||
public final class DevSyncCoordinator {
|
|
||||||
}
|
|
||||||
@ -1,5 +0,0 @@
|
|||||||
package com.ftptool.sync.repository;
|
|
||||||
|
|
||||||
@Deprecated
|
|
||||||
public final class SyncAckRepository {
|
|
||||||
}
|
|
||||||
@ -3,7 +3,9 @@ package com.ftptool.sync.repository;
|
|||||||
import com.ftptool.sync.entity.SyncTask;
|
import com.ftptool.sync.entity.SyncTask;
|
||||||
import com.ftptool.sync.model.SyncDirection;
|
import com.ftptool.sync.model.SyncDirection;
|
||||||
import org.springframework.data.jpa.repository.JpaRepository;
|
import org.springframework.data.jpa.repository.JpaRepository;
|
||||||
|
import org.springframework.data.domain.Pageable;
|
||||||
|
|
||||||
|
import java.util.List;
|
||||||
import java.util.Optional;
|
import java.util.Optional;
|
||||||
|
|
||||||
public interface SyncTaskRepository extends JpaRepository<SyncTask, Long> {
|
public interface SyncTaskRepository extends JpaRepository<SyncTask, Long> {
|
||||||
@ -21,4 +23,11 @@ public interface SyncTaskRepository extends JpaRepository<SyncTask, Long> {
|
|||||||
String sourceVersion,
|
String sourceVersion,
|
||||||
String contentHash
|
String contentHash
|
||||||
);
|
);
|
||||||
|
|
||||||
|
List<SyncTask> findAllByOrderByUpdatedAtDesc(Pageable pageable);
|
||||||
|
|
||||||
|
List<SyncTask> findByStatusOrderByUpdatedAtDesc(
|
||||||
|
com.ftptool.sync.model.SyncStatus status,
|
||||||
|
Pageable pageable
|
||||||
|
);
|
||||||
}
|
}
|
||||||
|
|||||||
@ -1,5 +0,0 @@
|
|||||||
package com.ftptool.sync.service;
|
|
||||||
|
|
||||||
@Deprecated
|
|
||||||
public final class AckFileService {
|
|
||||||
}
|
|
||||||
@ -1,5 +0,0 @@
|
|||||||
package com.ftptool.sync.service;
|
|
||||||
|
|
||||||
@Deprecated
|
|
||||||
public final class AckService {
|
|
||||||
}
|
|
||||||
@ -6,6 +6,7 @@ import com.ftptool.sync.repository.SyncCheckpointRepository;
|
|||||||
import org.springframework.stereotype.Service;
|
import org.springframework.stereotype.Service;
|
||||||
import org.springframework.transaction.annotation.Transactional;
|
import org.springframework.transaction.annotation.Transactional;
|
||||||
|
|
||||||
|
import java.util.List;
|
||||||
import java.util.Optional;
|
import java.util.Optional;
|
||||||
|
|
||||||
@Service
|
@Service
|
||||||
@ -31,4 +32,9 @@ public class CheckpointService {
|
|||||||
checkpoint.setLastSuccessHash(hash);
|
checkpoint.setLastSuccessHash(hash);
|
||||||
return syncCheckpointRepository.save(checkpoint);
|
return syncCheckpointRepository.save(checkpoint);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@Transactional(readOnly = true)
|
||||||
|
public List<SyncCheckpoint> findAllCheckpoints() {
|
||||||
|
return syncCheckpointRepository.findAll();
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@ -1,5 +0,0 @@
|
|||||||
package com.ftptool.sync.service;
|
|
||||||
|
|
||||||
@Deprecated
|
|
||||||
public final class FtpClientService {
|
|
||||||
}
|
|
||||||
@ -4,9 +4,11 @@ import com.ftptool.sync.entity.SyncTask;
|
|||||||
import com.ftptool.sync.model.SyncDirection;
|
import com.ftptool.sync.model.SyncDirection;
|
||||||
import com.ftptool.sync.model.SyncStatus;
|
import com.ftptool.sync.model.SyncStatus;
|
||||||
import com.ftptool.sync.repository.SyncTaskRepository;
|
import com.ftptool.sync.repository.SyncTaskRepository;
|
||||||
|
import org.springframework.data.domain.PageRequest;
|
||||||
import org.springframework.stereotype.Service;
|
import org.springframework.stereotype.Service;
|
||||||
import org.springframework.transaction.annotation.Transactional;
|
import org.springframework.transaction.annotation.Transactional;
|
||||||
|
|
||||||
|
import java.util.List;
|
||||||
import java.util.Optional;
|
import java.util.Optional;
|
||||||
import java.util.UUID;
|
import java.util.UUID;
|
||||||
|
|
||||||
@ -84,4 +86,14 @@ public class SyncTaskService {
|
|||||||
public boolean existsProcessed(SyncDirection direction, String sourceVersion, String contentHash) {
|
public boolean existsProcessed(SyncDirection direction, String sourceVersion, String contentHash) {
|
||||||
return syncTaskRepository.existsByDirectionAndSourceVersionAndContentHash(direction, sourceVersion, contentHash);
|
return syncTaskRepository.existsByDirectionAndSourceVersionAndContentHash(direction, sourceVersion, contentHash);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@Transactional(readOnly = true)
|
||||||
|
public List<SyncTask> findRecentTasks(int limit) {
|
||||||
|
return syncTaskRepository.findAllByOrderByUpdatedAtDesc(PageRequest.of(0, limit));
|
||||||
|
}
|
||||||
|
|
||||||
|
@Transactional(readOnly = true)
|
||||||
|
public List<SyncTask> findFailedTasks(int limit) {
|
||||||
|
return syncTaskRepository.findByStatusOrderByUpdatedAtDesc(SyncStatus.FAILED, PageRequest.of(0, limit));
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
227
src/main/java/com/ftptool/sync/web/SyncManagementController.java
Normal file
227
src/main/java/com/ftptool/sync/web/SyncManagementController.java
Normal file
@ -0,0 +1,227 @@
|
|||||||
|
package com.ftptool.sync.web;
|
||||||
|
|
||||||
|
import com.ftptool.sync.entity.SyncCheckpoint;
|
||||||
|
import com.ftptool.sync.entity.SyncTask;
|
||||||
|
import com.ftptool.sync.service.CheckpointService;
|
||||||
|
import com.ftptool.sync.service.SyncTaskService;
|
||||||
|
import org.springframework.web.bind.annotation.GetMapping;
|
||||||
|
import org.springframework.web.bind.annotation.RequestMapping;
|
||||||
|
import org.springframework.web.bind.annotation.RequestParam;
|
||||||
|
import org.springframework.web.bind.annotation.RestController;
|
||||||
|
|
||||||
|
import java.time.LocalDateTime;
|
||||||
|
import java.util.ArrayList;
|
||||||
|
import java.util.Comparator;
|
||||||
|
import java.util.List;
|
||||||
|
|
||||||
|
@RestController
|
||||||
|
@RequestMapping("/api/admin/sync")
|
||||||
|
public class SyncManagementController {
|
||||||
|
|
||||||
|
private final SyncTaskService syncTaskService;
|
||||||
|
private final CheckpointService checkpointService;
|
||||||
|
|
||||||
|
public SyncManagementController(SyncTaskService syncTaskService, CheckpointService checkpointService) {
|
||||||
|
this.syncTaskService = syncTaskService;
|
||||||
|
this.checkpointService = checkpointService;
|
||||||
|
}
|
||||||
|
|
||||||
|
@GetMapping("/overview")
|
||||||
|
public SyncOverviewResponse overview(
|
||||||
|
@RequestParam(name = "recentLimit", defaultValue = "10") int recentLimit,
|
||||||
|
@RequestParam(name = "failedLimit", defaultValue = "10") int failedLimit
|
||||||
|
) {
|
||||||
|
int normalizedRecentLimit = normalizeLimit(recentLimit);
|
||||||
|
int normalizedFailedLimit = normalizeLimit(failedLimit);
|
||||||
|
return new SyncOverviewResponse(
|
||||||
|
toCheckpointViews(checkpointService.findAllCheckpoints()),
|
||||||
|
toTaskViews(syncTaskService.findRecentTasks(normalizedRecentLimit)),
|
||||||
|
toTaskViews(syncTaskService.findFailedTasks(normalizedFailedLimit))
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
@GetMapping("/tasks/recent")
|
||||||
|
public List<SyncTaskView> recentTasks(@RequestParam(name = "limit", defaultValue = "20") int limit) {
|
||||||
|
return toTaskViews(syncTaskService.findRecentTasks(normalizeLimit(limit)));
|
||||||
|
}
|
||||||
|
|
||||||
|
@GetMapping("/tasks/failed")
|
||||||
|
public List<SyncTaskView> failedTasks(@RequestParam(name = "limit", defaultValue = "20") int limit) {
|
||||||
|
return toTaskViews(syncTaskService.findFailedTasks(normalizeLimit(limit)));
|
||||||
|
}
|
||||||
|
|
||||||
|
private int normalizeLimit(int limit) {
|
||||||
|
if (limit < 1) {
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
return Math.min(limit, 100);
|
||||||
|
}
|
||||||
|
|
||||||
|
private List<SyncTaskView> toTaskViews(List<SyncTask> tasks) {
|
||||||
|
List<SyncTaskView> result = new ArrayList<SyncTaskView>();
|
||||||
|
for (SyncTask task : tasks) {
|
||||||
|
result.add(new SyncTaskView(
|
||||||
|
task.getTraceId(),
|
||||||
|
task.getDirection().name(),
|
||||||
|
task.getSourceVersion(),
|
||||||
|
task.getContentHash(),
|
||||||
|
task.getStatus().name(),
|
||||||
|
task.getRetryCount(),
|
||||||
|
task.getErrorMsg(),
|
||||||
|
task.getCreatedAt(),
|
||||||
|
task.getUpdatedAt()
|
||||||
|
));
|
||||||
|
}
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
|
||||||
|
private List<SyncCheckpointView> toCheckpointViews(List<SyncCheckpoint> checkpoints) {
|
||||||
|
List<SyncCheckpointView> result = new ArrayList<SyncCheckpointView>();
|
||||||
|
checkpoints.sort(Comparator.comparing(checkpoint -> checkpoint.getDirection().name()));
|
||||||
|
for (SyncCheckpoint checkpoint : checkpoints) {
|
||||||
|
result.add(new SyncCheckpointView(
|
||||||
|
checkpoint.getDirection().name(),
|
||||||
|
checkpoint.getLastSuccessVersion(),
|
||||||
|
checkpoint.getLastSuccessHash(),
|
||||||
|
checkpoint.getUpdatedAt()
|
||||||
|
));
|
||||||
|
}
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
|
||||||
|
public static class SyncOverviewResponse {
|
||||||
|
|
||||||
|
private final List<SyncCheckpointView> checkpoints;
|
||||||
|
private final List<SyncTaskView> recentTasks;
|
||||||
|
private final List<SyncTaskView> failedTasks;
|
||||||
|
|
||||||
|
public SyncOverviewResponse(
|
||||||
|
List<SyncCheckpointView> checkpoints,
|
||||||
|
List<SyncTaskView> recentTasks,
|
||||||
|
List<SyncTaskView> failedTasks
|
||||||
|
) {
|
||||||
|
this.checkpoints = checkpoints;
|
||||||
|
this.recentTasks = recentTasks;
|
||||||
|
this.failedTasks = failedTasks;
|
||||||
|
}
|
||||||
|
|
||||||
|
public List<SyncCheckpointView> getCheckpoints() {
|
||||||
|
return checkpoints;
|
||||||
|
}
|
||||||
|
|
||||||
|
public List<SyncTaskView> getRecentTasks() {
|
||||||
|
return recentTasks;
|
||||||
|
}
|
||||||
|
|
||||||
|
public List<SyncTaskView> getFailedTasks() {
|
||||||
|
return failedTasks;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
public static class SyncCheckpointView {
|
||||||
|
|
||||||
|
private final String direction;
|
||||||
|
private final String lastSuccessVersion;
|
||||||
|
private final String lastSuccessHash;
|
||||||
|
private final LocalDateTime updatedAt;
|
||||||
|
|
||||||
|
public SyncCheckpointView(
|
||||||
|
String direction,
|
||||||
|
String lastSuccessVersion,
|
||||||
|
String lastSuccessHash,
|
||||||
|
LocalDateTime updatedAt
|
||||||
|
) {
|
||||||
|
this.direction = direction;
|
||||||
|
this.lastSuccessVersion = lastSuccessVersion;
|
||||||
|
this.lastSuccessHash = lastSuccessHash;
|
||||||
|
this.updatedAt = updatedAt;
|
||||||
|
}
|
||||||
|
|
||||||
|
public String getDirection() {
|
||||||
|
return direction;
|
||||||
|
}
|
||||||
|
|
||||||
|
public String getLastSuccessVersion() {
|
||||||
|
return lastSuccessVersion;
|
||||||
|
}
|
||||||
|
|
||||||
|
public String getLastSuccessHash() {
|
||||||
|
return lastSuccessHash;
|
||||||
|
}
|
||||||
|
|
||||||
|
public LocalDateTime getUpdatedAt() {
|
||||||
|
return updatedAt;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
public static class SyncTaskView {
|
||||||
|
|
||||||
|
private final String traceId;
|
||||||
|
private final String direction;
|
||||||
|
private final String sourceVersion;
|
||||||
|
private final String contentHash;
|
||||||
|
private final String status;
|
||||||
|
private final Integer retryCount;
|
||||||
|
private final String errorMsg;
|
||||||
|
private final LocalDateTime createdAt;
|
||||||
|
private final LocalDateTime updatedAt;
|
||||||
|
|
||||||
|
public SyncTaskView(
|
||||||
|
String traceId,
|
||||||
|
String direction,
|
||||||
|
String sourceVersion,
|
||||||
|
String contentHash,
|
||||||
|
String status,
|
||||||
|
Integer retryCount,
|
||||||
|
String errorMsg,
|
||||||
|
LocalDateTime createdAt,
|
||||||
|
LocalDateTime updatedAt
|
||||||
|
) {
|
||||||
|
this.traceId = traceId;
|
||||||
|
this.direction = direction;
|
||||||
|
this.sourceVersion = sourceVersion;
|
||||||
|
this.contentHash = contentHash;
|
||||||
|
this.status = status;
|
||||||
|
this.retryCount = retryCount;
|
||||||
|
this.errorMsg = errorMsg;
|
||||||
|
this.createdAt = createdAt;
|
||||||
|
this.updatedAt = updatedAt;
|
||||||
|
}
|
||||||
|
|
||||||
|
public String getTraceId() {
|
||||||
|
return traceId;
|
||||||
|
}
|
||||||
|
|
||||||
|
public String getDirection() {
|
||||||
|
return direction;
|
||||||
|
}
|
||||||
|
|
||||||
|
public String getSourceVersion() {
|
||||||
|
return sourceVersion;
|
||||||
|
}
|
||||||
|
|
||||||
|
public String getContentHash() {
|
||||||
|
return contentHash;
|
||||||
|
}
|
||||||
|
|
||||||
|
public String getStatus() {
|
||||||
|
return status;
|
||||||
|
}
|
||||||
|
|
||||||
|
public Integer getRetryCount() {
|
||||||
|
return retryCount;
|
||||||
|
}
|
||||||
|
|
||||||
|
public String getErrorMsg() {
|
||||||
|
return errorMsg;
|
||||||
|
}
|
||||||
|
|
||||||
|
public LocalDateTime getCreatedAt() {
|
||||||
|
return createdAt;
|
||||||
|
}
|
||||||
|
|
||||||
|
public LocalDateTime getUpdatedAt() {
|
||||||
|
return updatedAt;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@ -1,2 +0,0 @@
|
|||||||
# Retired profile.
|
|
||||||
# The Git direct architecture no longer requires a dev-agent deployment.
|
|
||||||
@ -0,0 +1,149 @@
|
|||||||
|
package com.ftptool.sync.orchestrator;
|
||||||
|
|
||||||
|
import com.ftptool.sync.GitDirectSyncToolApplication;
|
||||||
|
import com.ftptool.sync.entity.SyncCheckpoint;
|
||||||
|
import com.ftptool.sync.entity.SyncTask;
|
||||||
|
import com.ftptool.sync.model.PackageBuildResult;
|
||||||
|
import com.ftptool.sync.model.PackageManifest;
|
||||||
|
import com.ftptool.sync.model.ProdPullResult;
|
||||||
|
import com.ftptool.sync.model.SyncDirection;
|
||||||
|
import com.ftptool.sync.model.SyncStatus;
|
||||||
|
import com.ftptool.sync.repository.SyncCheckpointRepository;
|
||||||
|
import com.ftptool.sync.repository.SyncTaskRepository;
|
||||||
|
import com.ftptool.sync.service.GitClientService;
|
||||||
|
import com.ftptool.sync.service.PackageService;
|
||||||
|
import com.ftptool.sync.service.ProdConfigApiService;
|
||||||
|
import org.junit.jupiter.api.BeforeEach;
|
||||||
|
import org.junit.jupiter.api.Test;
|
||||||
|
import org.springframework.beans.factory.annotation.Autowired;
|
||||||
|
import org.springframework.boot.test.context.SpringBootTest;
|
||||||
|
import org.springframework.boot.test.mock.mockito.MockBean;
|
||||||
|
import org.springframework.test.context.ActiveProfiles;
|
||||||
|
|
||||||
|
import java.nio.file.Files;
|
||||||
|
import java.nio.file.Path;
|
||||||
|
import java.util.List;
|
||||||
|
import java.util.Optional;
|
||||||
|
|
||||||
|
import static org.junit.jupiter.api.Assertions.assertEquals;
|
||||||
|
import static org.junit.jupiter.api.Assertions.assertTrue;
|
||||||
|
import static org.mockito.ArgumentMatchers.any;
|
||||||
|
import static org.mockito.ArgumentMatchers.contains;
|
||||||
|
import static org.mockito.ArgumentMatchers.eq;
|
||||||
|
import static org.mockito.Mockito.doNothing;
|
||||||
|
import static org.mockito.Mockito.times;
|
||||||
|
import static org.mockito.Mockito.verify;
|
||||||
|
import static org.mockito.Mockito.when;
|
||||||
|
|
||||||
|
@SpringBootTest(
|
||||||
|
classes = GitDirectSyncToolApplication.class,
|
||||||
|
properties = {
|
||||||
|
"spring.datasource.url=jdbc:h2:mem:gitdirect-it;DB_CLOSE_DELAY=-1;MODE=MYSQL",
|
||||||
|
"spring.datasource.driver-class-name=org.h2.Driver",
|
||||||
|
"spring.datasource.username=sa",
|
||||||
|
"spring.datasource.password=",
|
||||||
|
"spring.jpa.hibernate.ddl-auto=none",
|
||||||
|
"spring.sql.init.mode=always",
|
||||||
|
"sync.work-dir=./target/test-work",
|
||||||
|
"sync.package-temp-dir=./target/test-work/package",
|
||||||
|
"sync.dev-to-prod-staging-dir=./target/test-work/dev-to-prod",
|
||||||
|
"sync.prod-to-dev-staging-dir=./target/test-work/prod-to-dev",
|
||||||
|
"git.repo.scan-branch=config-dev-main",
|
||||||
|
"git.repo.snapshot-branch=config-prod-snapshot",
|
||||||
|
"git.repo.local-path=./target/test-work/git/config-repo"
|
||||||
|
}
|
||||||
|
)
|
||||||
|
@ActiveProfiles("prod-agent")
|
||||||
|
class ProdSyncCoordinatorIntegrationTest {
|
||||||
|
|
||||||
|
@Autowired
|
||||||
|
private ProdSyncCoordinator prodSyncCoordinator;
|
||||||
|
|
||||||
|
@Autowired
|
||||||
|
private SyncTaskRepository syncTaskRepository;
|
||||||
|
|
||||||
|
@Autowired
|
||||||
|
private SyncCheckpointRepository syncCheckpointRepository;
|
||||||
|
|
||||||
|
@MockBean
|
||||||
|
private GitClientService gitClientService;
|
||||||
|
|
||||||
|
@MockBean
|
||||||
|
private PackageService packageService;
|
||||||
|
|
||||||
|
@MockBean
|
||||||
|
private ProdConfigApiService prodConfigApiService;
|
||||||
|
|
||||||
|
@BeforeEach
|
||||||
|
void setUp() {
|
||||||
|
syncTaskRepository.deleteAll();
|
||||||
|
syncCheckpointRepository.deleteAll();
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void shouldSyncGitToProdAndKeepItIdempotent() throws Exception {
|
||||||
|
Path zipFile = Files.createTempFile("git-to-prod-", ".zip");
|
||||||
|
when(gitClientService.prepareRepositoryAndGetHead("config-dev-main")).thenReturn("commit-a");
|
||||||
|
when(gitClientService.exportBranchSnapshot(eq("config-dev-main"), any(Path.class)))
|
||||||
|
.thenAnswer(invocation -> invocation.getArgument(1));
|
||||||
|
when(packageService.calculateDirectoryHash(any(Path.class))).thenReturn("hash-a");
|
||||||
|
when(packageService.buildPackageFromDirectory(any(Path.class), any(PackageManifest.class)))
|
||||||
|
.thenAnswer(invocation -> {
|
||||||
|
PackageManifest manifest = invocation.getArgument(1);
|
||||||
|
return new PackageBuildResult(zipFile, manifest.getPackageName(), "hash-a");
|
||||||
|
});
|
||||||
|
doNothing().when(prodConfigApiService).pushPackage(any(PackageManifest.class), eq(zipFile));
|
||||||
|
|
||||||
|
prodSyncCoordinator.syncLatestGitToProd();
|
||||||
|
prodSyncCoordinator.syncLatestGitToProd();
|
||||||
|
|
||||||
|
List<SyncTask> tasks = syncTaskRepository.findAll();
|
||||||
|
assertEquals(1, tasks.size());
|
||||||
|
SyncTask task = tasks.get(0);
|
||||||
|
assertEquals(SyncDirection.DEV_TO_PROD, task.getDirection());
|
||||||
|
assertEquals("commit-a", task.getSourceVersion());
|
||||||
|
assertEquals("hash-a", task.getContentHash());
|
||||||
|
assertEquals(SyncStatus.SUCCESS, task.getStatus());
|
||||||
|
|
||||||
|
Optional<SyncCheckpoint> checkpoint = syncCheckpointRepository.findByDirection(SyncDirection.DEV_TO_PROD);
|
||||||
|
assertTrue(checkpoint.isPresent());
|
||||||
|
assertEquals("commit-a", checkpoint.get().getLastSuccessVersion());
|
||||||
|
assertEquals("hash-a", checkpoint.get().getLastSuccessHash());
|
||||||
|
|
||||||
|
verify(prodConfigApiService, times(1)).pushPackage(any(PackageManifest.class), eq(zipFile));
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void shouldSyncProdSnapshotToGitAndKeepItIdempotent() throws Exception {
|
||||||
|
Path contentDirectory = Files.createTempDirectory("prod-to-git-");
|
||||||
|
Files.write(contentDirectory.resolve("prod-config.json"), "{\"version\":\"prod-v1\"}".getBytes("UTF-8"));
|
||||||
|
when(prodConfigApiService.pullConfigSnapshot()).thenReturn(new ProdPullResult(contentDirectory, "prod-v1", "hash-b"));
|
||||||
|
when(gitClientService.syncDirectoryToBranch(
|
||||||
|
eq(contentDirectory),
|
||||||
|
eq("config-prod-snapshot"),
|
||||||
|
contains("prod-v1")
|
||||||
|
)).thenReturn(true);
|
||||||
|
|
||||||
|
prodSyncCoordinator.syncProdSnapshotToGit();
|
||||||
|
prodSyncCoordinator.syncProdSnapshotToGit();
|
||||||
|
|
||||||
|
List<SyncTask> tasks = syncTaskRepository.findAll();
|
||||||
|
assertEquals(1, tasks.size());
|
||||||
|
SyncTask task = tasks.get(0);
|
||||||
|
assertEquals(SyncDirection.PROD_TO_DEV, task.getDirection());
|
||||||
|
assertEquals("prod-v1", task.getSourceVersion());
|
||||||
|
assertEquals("hash-b", task.getContentHash());
|
||||||
|
assertEquals(SyncStatus.SUCCESS, task.getStatus());
|
||||||
|
|
||||||
|
Optional<SyncCheckpoint> checkpoint = syncCheckpointRepository.findByDirection(SyncDirection.PROD_TO_DEV);
|
||||||
|
assertTrue(checkpoint.isPresent());
|
||||||
|
assertEquals("prod-v1", checkpoint.get().getLastSuccessVersion());
|
||||||
|
assertEquals("hash-b", checkpoint.get().getLastSuccessHash());
|
||||||
|
|
||||||
|
verify(gitClientService, times(1)).syncDirectoryToBranch(
|
||||||
|
eq(contentDirectory),
|
||||||
|
eq("config-prod-snapshot"),
|
||||||
|
contains("prod-v1")
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
Loading…
x
Reference in New Issue
Block a user