后台自动核对REVIEW状态的项目
This commit is contained in:
@@ -624,6 +624,8 @@ interface CommitInput {
|
||||
|
||||
版本冲突返回 409 `CONFLICT`。保留未保存的编辑内容,提示用户刷新并比对差异,不能直接换成最新 SHA 自动覆盖。
|
||||
|
||||
普通文件提交结果不明时由后台自动核对,核对成功后自动恢复写入,刷新工作区即可继续。请求与响应协议不变,不需要前端接入新的轮询接口。超时不代表提交失败,暂勿自动重传或自动发布。
|
||||
|
||||
### 6.4 多文件上传(保留目录)
|
||||
|
||||
`POST /admin-api/projects/{projectCode}/uploads/commit`,`multipart/form-data` → `CommitView`。
|
||||
|
||||
@@ -85,3 +85,33 @@ ALTER TABLE project MODIFY COLUMN publish_status VARCHAR(16) NOT NULL DEFAULT 'U
|
||||
```
|
||||
|
||||
旧版七个字段及其索引可以暂留,代码已不再读写;确认备份及来源回填完成后再清理。特别是只有旧 Excel 批次、没有后台任务的历史项目,其来源需保留原批次清单或另行归档,不能直接删除唯一来源信息。本次不对实际数据库执行增删字段操作。
|
||||
|
||||
## 新建普通文件写入操作记录表(本次上线前执行)
|
||||
|
||||
只新增此表,不需要修改项目表。由运维手工执行;应用不自动迁库。时间全部为 UTC。
|
||||
|
||||
```sql
|
||||
CREATE TABLE repository_write_operation (
|
||||
operation_code CHAR(32) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL COMMENT '写入操作业务编码,同时作为幂等键',
|
||||
project_code CHAR(32) NOT NULL COMMENT '项目业务编码',
|
||||
repository_id BIGINT NOT NULL COMMENT '操作发起时Gitea仓库内部标识,仅核验使用',
|
||||
repository_owner VARCHAR(255) NOT NULL COMMENT '操作发起时仓库所属账号',
|
||||
repository_name VARCHAR(255) NOT NULL COMMENT '操作发起时仓库名称',
|
||||
base_commit_sha VARCHAR(64) NOT NULL COMMENT '本次写入的基准提交SHA',
|
||||
request_digest CHAR(64) NOT NULL COMMENT '请求内容SHA256摘要,不保存文件正文',
|
||||
commit_message VARCHAR(1000) NOT NULL COMMENT '原始提交说明,不包含系统凭据尾注',
|
||||
status VARCHAR(16) NOT NULL COMMENT '操作状态 PROCESSING处理中 NEEDS_REVIEW超时待确认 SUCCEEDED已成功 FAILED明确失败',
|
||||
result_commit_sha VARCHAR(64) NULL COMMENT '确认成功的提交SHA',
|
||||
failure_status INT NULL COMMENT '明确失败时原HTTP状态码',
|
||||
last_error_code VARCHAR(64) NULL COMMENT '最近核对错误编码,不保存异常正文或凭证',
|
||||
check_count INT NOT NULL DEFAULT 0 COMMENT '后台已执行核对次数',
|
||||
next_check_at DATETIME(3) NOT NULL COMMENT '下次只读核对时间UTC',
|
||||
created_at DATETIME(3) NOT NULL COMMENT '创建时间UTC',
|
||||
updated_at DATETIME(3) NOT NULL COMMENT '最近更新时间UTC',
|
||||
PRIMARY KEY (operation_code),
|
||||
KEY idx_write_project_status (project_code, status, created_at),
|
||||
KEY idx_write_next_check (status, next_check_at)
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci COMMENT='普通文件写入幂等凭据及自动核对记录';
|
||||
```
|
||||
|
||||
已完成的操作记录仍用于幂等重放,不要在客户端可能重试期间删除。历史 REVIEW 没有操作记录,不会凭最新 SHA 自动判成功;需核对实际结果后按原维护流程处理。迁移、Fork 继续使用原流程。
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
## 部署与初始化
|
||||
|
||||
1. 备份数据库、暂停写入、排空旧进程。已有数据库手工执行 [增量 SQL](admin-database.md),新数据库先执行 [基础四表 SQL](database.md),再创建迁移任务表。应用不自动执行 SQL。社区端也会读取迁移任务,因此暂不启用管理端时仍需创建该表。
|
||||
1. 备份数据库、暂停写入、排空旧进程。已有数据库手工执行 [增量 SQL](admin-database.md),新数据库先执行 [基础四表 SQL](database.md),再创建迁移任务表和普通文件写入操作记录表。应用不自动执行 SQL。社区端也会读取这两张表,因此暂不启用管理端时仍需创建。
|
||||
2. 更新社区端、管理端和 Excel 导入工具,确保全部写入端使用数据库共享锁。
|
||||
3. 在 `.env` 或部署环境中提供现有 `DB_*`、`GITEA_*`、`OSS_*`;后台默认关闭 AI,OSS 也需显式启用。另需提供 `REDIS_*` 连接变量,默认环境为 `test`,正式部署显式设置 `SPRING_PROFILES_ACTIVE=prod`。真实 `.env` 不提交,不用示例覆盖已有凭据。
|
||||
4. 选择一个已有、正常的普通用户,执行初始化命令,交互输入两次管理员密码。密码至少 12 字符、UTF-8 最多 72 字节,不会写入命令参数或日志。该命令只初始化 USER,不用于重置已有管理员密码;成功后进程退出。
|
||||
@@ -49,7 +49,7 @@ project 表只新增 moderation_reason 和 repo_write_status,来源标识、
|
||||
|
||||
全部写入端按项目编码取得同一 MySQL 命名锁,使用固定连接取得、持有和释放,支持内部重入。锁连接池每应用最多 8 个连接,另有普通业务连接池,需要预留数据库容量。远端请求期间不持有数据库长事务;写入前持久化状态,结果不明时重启后仍阻断写入。
|
||||
|
||||
普通文件上传的写入状态依次为 `IDLE → WRITING → IDLE`,最终恢复空闲后记录“仓库写入完成”日志。Gitea 已提交但响应丢失、状态回写失败等未知结果进入 `REVIEW`;即使 main 已有最新提交,发布仍会被阻断。上传提交前或发布前的 Gitea 读取失败应恢复 `IDLE`,异常传递保留远端结果确定性,避免把只读失败误标为未知写入。历史 `REVIEW` 仍需按实际操作结果人工核对恢复。
|
||||
普通文件上传和编辑使用 repository_write_operation 持久化幂等凭据。正常为 `IDLE → WRITING → IDLE`;响应丢失先尝试只读匹配凭据,匹配成功仍返回 200。未知结果持久化 REVIEW,保持原有错误响应,由后台每 15 秒核对,10 分钟后改为每分钟继续核对。核对成功自动恢复 IDLE,不自动发布。提交前或发布前的 Gitea 读取失败保持 IDLE,不能误标为未知写入。操作编码由后端自动生成,前端不需要改造。
|
||||
|
||||
上线前确认目标 Gitea 已开启迁移、HTTP Git、LFS,技术账号具有组织仓库写权限。GitHub 预检优先读取 API;受限时严格核对公开网页身份、可见性和默认分支,无法确认时失败。网页回退拿不到的简介、许可证可能为空,需要管理员检查原仓库文件。
|
||||
|
||||
@@ -74,13 +74,17 @@ project 表只新增 moderation_reason 和 repo_write_status,来源标识、
|
||||
|
||||
逻辑删除、撤回和下架不会物理删除 Gitea 仓库或 OSS 对象;已公开 OSS 链接沿用现有生命周期。
|
||||
|
||||
### 普通项目人工核对后的写入恢复
|
||||
### 普通项目后台自动核对
|
||||
|
||||
普通项目同时使用数据库 `repo_write_status` 和各写入进程内的待核对集合。只把数据库改为 `IDLE`,已有内存标记会在下一次提交或发布时重新写回 `REVIEW`;仅重启进程也无法解除数据库中保留的 `REVIEW` 或 `WRITING`。启动后首次请求若发现数据库未空闲,又会建立本进程的待核对标记。
|
||||
上线顺序:手工执行 [操作表 SQL](admin-database.md#新建普通文件写入操作记录表本次上线前执行) → 排空旧写入进程 → 同时升级社区 API 与管理 API。接口沿用原有请求及成功/错误响应,不新增 202、前端结果查询或在线恢复接口。前端不需要接入新协议。
|
||||
|
||||
先按项目编码核对 Gitea 仓库身份、预期文件和提交历史,确认上次远端写入已结束。普通项目当前没有在线恢复接口;核对完成后停止社区 API、管理 API 及仍在运行的导入写入进程,再仅针对已核实的项目恢复数据库为 `IDLE`,最后启动上述服务。恢复后刷新目录并由管理员明确发布,恢复本身不更改发布状态或远端文件。多个进程需全部处理,避免仍持有旧标记的进程再次写回 `REVIEW`。
|
||||
后台任务在两个 API 应用中默认启动,由数据库项目命名锁避免重复核对;在独立线程执行,避免阻塞迁移任务调度。`vibemake.repository-write.worker-enabled=false` 可停用本实例的核对消费,但必须至少保留一个启用的实例。操作记录在远端写入前落盘,进程重启后按 next_check_at 继续;结果未知只查不重发。每次扫描 main 可达的最近至多 1000 个提交,精确校验操作尾注、请求摘要、唯一父提交及仓库身份;超范围、读取失败或不匹配均保留未知,不自动判失败。
|
||||
|
||||
写入拦截日志同时记录 `persistedWriteStatus` 和 `memoryUncertain`。`persistedWriteStatus=IDLE memoryUncertain=true` 表示数据库已恢复、本进程仍保留待核对标记;`memoryUncertain=false` 且数据库非 `IDLE` 表示本次由持久化状态阻断。这两种拦截均发生在本次业务写入和发布 SHA 校验之前,不表示本次请求已调用 Gitea 写入。
|
||||
前 10 分钟每 15 秒核对,超过 10 分钟改为每分钟继续核对,不强制解除保护。使用 projectCode、operationCode、baseCommitSha、commitSha、checkCount、errorCode 关联日志;原始请求正文和认证信息不进入操作表。操作状态 PROCESSING 为处理中,NEEDS_REVIEW 为超过 10 分钟仍未确认,SUCCEEDED 为确认成功,FAILED 为明确拒绝。操作终态与 IDLE 在同一事务确认;其他实例下次取得共享锁时按数据库结果清除旧内存标记,无需重启。
|
||||
|
||||
上传或编辑超时后如果立即查到凭据,原接口直接返回 200;暂时不能确认时仍返回原错误,不会把本次上传显示为成功。后台确认后自动恢复写入,管理员刷新工作区后即可继续。未确认期间禁止新的远端写入,重试不自动重发 Gitea POST。恢复不会自动发布。
|
||||
|
||||
历史 REVIEW/遗留 WRITING 没有操作记录时不做自动猜测,仍需核对仓库身份、文件、历史和上次操作是否已结束,再按原维护流程处理。迁移和 Fork 继续使用原专门核验流程。本阶段暂不提供管理员在线恢复入口。
|
||||
|
||||
## 验证
|
||||
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
# 数据库初始化
|
||||
|
||||
业务数据库名为 `vibemake`,只包含以下四张表。由人工执行;应用不运行迁移或自动建表。以下 SQL 面向 MySQL 8.0.13 及以上(URDF JSON 默认值使用表达式)。
|
||||
业务数据库名为 `vibemake`,基础业务包含以下四张表;完整部署还需执行 [管理端增量 SQL](admin-database.md),新增迁移任务与仓库写入操作表。由人工执行;应用不运行迁移或自动建表。以下 SQL 面向 MySQL 8.0.13 及以上(URDF JSON 默认值使用表达式)。
|
||||
|
||||
```sql
|
||||
CREATE DATABASE IF NOT EXISTS vibemake CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
|
||||
|
||||
@@ -841,6 +841,8 @@ entries 仅包含当前目录的直接子项,目录在前,名称升序。展
|
||||
|
||||
如果 expectedCommitSha 已不是最新 main,返回 409 CONFLICT。前端保留当前未提交输入,重新读取工作区版本,让用户处理差异后再提交;不要只替换 SHA 后自动覆盖内容。浏览历史版本时应使用只读模式,回到最新版本后再编辑。
|
||||
|
||||
普通文件提交结果不明时由后台自动核对,核对成功后自动恢复写入,刷新工作区即可继续。请求与响应协议不变,不需要前端接入新的轮询接口。超时不代表提交失败,暂勿自动重传或自动发布。
|
||||
|
||||
### 上传并提交
|
||||
|
||||
`POST /api/workspace/projects/{projectCode}/uploads/commit`,需要登录,使用 FormData。
|
||||
|
||||
+127
-31
@@ -42,6 +42,8 @@ import java.util.regex.*;
|
||||
"spring.datasource.driver-class-name=org.h2.Driver",
|
||||
"vibemake.admin.worker-enabled=false",
|
||||
"vibemake.oss.enabled=false",
|
||||
"vibemake.gitea.owner=vibemake",
|
||||
"vibemake.repository-write.worker-enabled=false",
|
||||
"logging.level.root=WARN"
|
||||
})
|
||||
@ActiveProfiles("test")
|
||||
@@ -95,8 +97,13 @@ class AdminWorkflowTest {
|
||||
@Autowired ObjectMapper json;
|
||||
@Autowired FakeGiteaClient gitea;
|
||||
@Autowired ProjectService projectService;
|
||||
@Autowired FileService fileService;
|
||||
@Autowired UserService userService;
|
||||
@Autowired ProjectLocks locks;
|
||||
@Autowired com.linkerbot.vibemake.service.write.RepositoryWrites writes;
|
||||
@Autowired RepositoryWriteOperationMapper writeOperations;
|
||||
@Autowired javax.sql.DataSource dataSource;
|
||||
@Autowired org.springframework.transaction.support.TransactionTemplate transactions;
|
||||
@Autowired OssProperties oss;
|
||||
@Autowired FakeOssObjectStore ossObjects;
|
||||
private User admin;
|
||||
@@ -209,47 +216,136 @@ class AdminWorkflowTest {
|
||||
assertThat(projectService.published(code).getPublishedCommitSha()).isEqualTo(current);
|
||||
}
|
||||
|
||||
private String initialSha(String code) throws Exception {
|
||||
return request(signed(get("/admin-api/projects/" + code + "/tree")), 200)
|
||||
.path("data").path("commitSha").asText();
|
||||
}
|
||||
|
||||
private void due(String operationCode, boolean overdue) {
|
||||
java.time.LocalDateTime now = ProjectSupport.now();
|
||||
jdbc.update("UPDATE repository_write_operation SET next_check_at=?, created_at=? WHERE operation_code=?",
|
||||
now.minusSeconds(1), now.minusMinutes(overdue ? 11 : 1), operationCode);
|
||||
}
|
||||
|
||||
private String operationState(String operationCode) {
|
||||
return writeOperations.selectById(operationCode).getStatus();
|
||||
}
|
||||
|
||||
private String operationCode(String projectCode) {
|
||||
return jdbc.queryForObject("SELECT operation_code FROM repository_write_operation WHERE project_code=?",
|
||||
String.class, projectCode);
|
||||
}
|
||||
|
||||
private com.linkerbot.vibemake.bean.Contracts.CommitView replayUpload(String code, String base, String key,
|
||||
String directory) {
|
||||
try (ProjectActors.Scope scope = ProjectActors.enter(new ProjectActors.AdminActor(admin.getUserCode(), admin.getNickname()))) {
|
||||
return fileService.upload(0, code, base, "测试上传", directory,
|
||||
List.of(new MockMultipartFile("files", "sample.txt", "text/plain", "hello".getBytes())),
|
||||
List.of("models/sample.txt"), key);
|
||||
}
|
||||
}
|
||||
|
||||
@Test
|
||||
void uploadedCommitWithLostResponseStillBlocksPublicationAtLatestSha() throws Exception {
|
||||
void lostUploadResponseIsConfirmedAndSameOperationNeverRepeatsWrite() throws Exception {
|
||||
String code = createProject();
|
||||
String initial = request(signed(get("/admin-api/projects/" + code + "/tree")), 200)
|
||||
.path("data").path("commitSha").asText();
|
||||
String initial = initialSha(code);
|
||||
gitea.timeoutNextCommit = true;
|
||||
JsonNode failure = request(uploadRequest(code, initial), 504);
|
||||
assertThat(failure.path("code").asText()).isEqualTo("REMOTE_RESULT_UNKNOWN");
|
||||
assertThat(failure.path("data").path("projectCode").asText()).isEqualTo(code);
|
||||
assertThat(writeState(code)).isEqualTo("REVIEW");
|
||||
String current = request(signed(get("/admin-api/projects/" + code + "/tree")), 200)
|
||||
.path("data").path("commitSha").asText();
|
||||
JsonNode result = request(uploadRequest(code, initial), 200);
|
||||
String key = operationCode(code);
|
||||
String current = result.path("data").path("commitSha").asText();
|
||||
assertThat(current).isNotEqualTo(initial);
|
||||
assertThat(request(signed(get("/admin-api/projects/" + code + "/files")
|
||||
.param("path", "models/sample.txt")), 200)
|
||||
.path("data").path("content").asText()).isEqualTo("hello");
|
||||
JsonNode blocked = request(signed(body(
|
||||
put("/admin-api/projects/" + code + "/publication"), Map.of("commitSha", current))), 409);
|
||||
assertThat(blocked.path("code").asText()).isEqualTo("REMOTE_RESULT_UNKNOWN");
|
||||
assertThat(writeState(code)).isEqualTo("REVIEW");
|
||||
assertThat(projects.selectOne(new com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper<Project>()
|
||||
.eq(Project::getProjectCode, code)).getPublishedCommitSha()).isNull();
|
||||
|
||||
// 只改数据库不能清除当前进程的待核对标记,下一次写入会重新持久化 REVIEW。
|
||||
jdbc.update("UPDATE project SET repo_write_status='IDLE' WHERE project_code=?", code);
|
||||
assertThat(result.path("data").path("message").asText()).isEqualTo("测试上传");
|
||||
assertThat(writeState(code)).isEqualTo("IDLE");
|
||||
JsonNode blockedAgain = request(signed(body(
|
||||
put("/admin-api/projects/" + code + "/publication"), Map.of("commitSha", current))), 409);
|
||||
assertThat(blockedAgain.path("code").asText()).isEqualTo("REMOTE_RESULT_UNKNOWN");
|
||||
assertThat(writeState(code)).isEqualTo("REVIEW");
|
||||
assertThat(operationState(key)).isEqualTo("SUCCEEDED");
|
||||
int calls = gitea.commitCalls;
|
||||
assertThat(replayUpload(code, initial, key, "").commitSha()).isEqualTo(current);
|
||||
assertThat(gitea.commitCalls).isEqualTo(calls);
|
||||
assertThatThrownBy(() -> replayUpload(code, initial, key, "other"))
|
||||
.isInstanceOfSatisfying(BusinessException.class,
|
||||
error -> assertThat(error.code()).isEqualTo("IDEMPOTENCY_KEY_REUSED"));
|
||||
assertThat(request(signed(get("/admin-api/projects/" + code + "/commits")), 200)
|
||||
.path("data").path("items").get(0).path("message").asText()).isEqualTo("测试上传");
|
||||
assertThat(jdbc.queryForObject("SELECT published_commit_sha FROM project WHERE project_code=?",
|
||||
String.class, code)).isNull();
|
||||
}
|
||||
|
||||
// 模拟核对远端结果后持锁恢复,同时清除数据库和本进程标记。
|
||||
locks.serial(code, () -> {
|
||||
locks.confirmKnownResult(code);
|
||||
return null;
|
||||
});
|
||||
@Test
|
||||
void pendingWriteKeepsOriginalErrorAndRecoversAfterRestartWithoutFrontendChanges() throws Exception {
|
||||
String code = createProject();
|
||||
String initial = initialSha(code);
|
||||
gitea.timeoutNextCommit = true;
|
||||
gitea.hideReceipts = true;
|
||||
JsonNode error = request(uploadRequest(code, initial), 504);
|
||||
assertThat(error.path("code").asText()).isEqualTo("REMOTE_RESULT_UNKNOWN");
|
||||
assertThat(error.path("data").path("projectCode").asText()).isEqualTo(code);
|
||||
String key = operationCode(code);
|
||||
assertThat(operationState(key)).isEqualTo("PROCESSING");
|
||||
assertThat(writeState(code)).isEqualTo("REVIEW");
|
||||
int calls = gitea.commitCalls;
|
||||
assertThat(request(uploadRequest(code, initial), 409).path("code").asText())
|
||||
.isEqualTo("REMOTE_RESULT_UNKNOWN");
|
||||
assertThatThrownBy(() -> replayUpload(code, initial, key, ""))
|
||||
.isInstanceOfSatisfying(BusinessException.class, business -> assertThat(business.status()).isEqualTo(409));
|
||||
assertThat(gitea.commitCalls).isEqualTo(calls);
|
||||
String current = initialSha(code);
|
||||
request(signed(body(put("/admin-api/projects/" + code + "/publication"),
|
||||
Map.of("commitSha", current))), 409);
|
||||
due(key, false);
|
||||
gitea.hideReceipts = false;
|
||||
try (ProjectLocks restarted = new ProjectLocks(dataSource, projects)) {
|
||||
com.linkerbot.vibemake.service.write.RepositoryWrites worker =
|
||||
new com.linkerbot.vibemake.service.write.RepositoryWrites(writeOperations, projects,
|
||||
projectService, restarted, gitea, transactions);
|
||||
worker.reconcileDue();
|
||||
}
|
||||
assertThat(operationState(key)).isEqualTo("SUCCEEDED");
|
||||
assertThat(writeState(code)).isEqualTo("IDLE");
|
||||
assertThat(writeOperations.selectById(key).getResultCommitSha()).isEqualTo(current);
|
||||
assertThat(gitea.commitCalls).isEqualTo(calls);
|
||||
assertThat(jdbc.queryForObject("SELECT published_commit_sha FROM project WHERE project_code=?",
|
||||
String.class, code)).isNull();
|
||||
// 原实例仍保留 uncertain,下一次正常写入必须按数据库 IDLE 清理,无需重启。
|
||||
request(signed(body(put("/admin-api/projects/" + code + "/publication"),
|
||||
Map.of("commitSha", current))), 200);
|
||||
assertThat(writeState(code)).isEqualTo("IDLE");
|
||||
assertThat(projectService.published(code).getPublishedCommitSha()).isEqualTo(current);
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@ValueSource(strings = {"missing", "parent", "identity", "read"})
|
||||
void missingOrInvalidReceiptNeverBecomesSuccessOrResendsWrite(String fault) throws Exception {
|
||||
String code = createProject();
|
||||
String initial = initialSha(code);
|
||||
gitea.hideReceipts = fault.equals("missing");
|
||||
gitea.wrongReceiptParent = fault.equals("parent");
|
||||
gitea.wrongRepositoryIdentity = fault.equals("identity");
|
||||
gitea.failReceiptReads = fault.equals("read");
|
||||
gitea.timeoutNextCommit = true;
|
||||
request(uploadRequest(code, initial), 504);
|
||||
String key = operationCode(code);
|
||||
int calls = gitea.commitCalls;
|
||||
due(key, false);
|
||||
writes.reconcileDue();
|
||||
assertThat(operationState(key)).isEqualTo("PROCESSING");
|
||||
assertThat(writeState(code)).isEqualTo("REVIEW");
|
||||
assertThat(writeOperations.selectById(key).getNextCheckAt())
|
||||
.isAfter(ProjectSupport.now().plusSeconds(10));
|
||||
due(key, true);
|
||||
writes.reconcileDue();
|
||||
assertThat(operationState(key)).isEqualTo("NEEDS_REVIEW");
|
||||
assertThat(writeOperations.selectById(key).getNextCheckAt())
|
||||
.isAfter(ProjectSupport.now().plusSeconds(50));
|
||||
assertThat(gitea.commitCalls).isEqualTo(calls);
|
||||
assertThat(writeState(code)).isEqualTo("REVIEW");
|
||||
}
|
||||
|
||||
@Test
|
||||
void legacyReviewWithoutOperationIsNeverGuessedSuccessful() throws Exception {
|
||||
String code = createProject();
|
||||
locks.serial(code, () -> { locks.markUncertain(code); return null; });
|
||||
int calls = gitea.commitCalls;
|
||||
writes.reconcileDue();
|
||||
assertThat(writeState(code)).isEqualTo("REVIEW");
|
||||
assertThat(gitea.commitCalls).isEqualTo(calls);
|
||||
}
|
||||
|
||||
private JsonNode createImport() throws Exception {
|
||||
|
||||
+46
-2
@@ -12,6 +12,7 @@ final class FakeGiteaClient implements GiteaClient {
|
||||
private static final class RepoData {
|
||||
Repository repository;
|
||||
String head;
|
||||
Map<String, String> parents = new HashMap<>();
|
||||
Map<String, Map<String, byte[]>> snapshots = new HashMap<>();
|
||||
List<HistoryEntry> history = new ArrayList<>();
|
||||
}
|
||||
@@ -27,8 +28,14 @@ final class FakeGiteaClient implements GiteaClient {
|
||||
int deletions;
|
||||
boolean mainConflict;
|
||||
private final Map<String, RemoteRepository> migrated = new HashMap<>();
|
||||
private final Set<String> migrationNames = new HashSet<>();
|
||||
final Map<String, String> migrationDescriptions = new HashMap<>();
|
||||
private final Set<String> mainReady = new HashSet<>();
|
||||
boolean hideReceipts;
|
||||
boolean failReceiptReads;
|
||||
boolean wrongReceiptParent;
|
||||
boolean wrongRepositoryIdentity;
|
||||
int commitCalls;
|
||||
boolean timeoutNextCommit;
|
||||
boolean timeoutNextHead;
|
||||
boolean timeoutNextCreate;
|
||||
@@ -36,7 +43,13 @@ final class FakeGiteaClient implements GiteaClient {
|
||||
|
||||
void reset() {
|
||||
repositories.clear();
|
||||
hideReceipts = false;
|
||||
failReceiptReads = false;
|
||||
wrongReceiptParent = false;
|
||||
wrongRepositoryIdentity = false;
|
||||
commitCalls = 0;
|
||||
migrated.clear();
|
||||
migrationNames.clear();
|
||||
mainReady.clear();
|
||||
migrations = 0;
|
||||
deletions = 0;
|
||||
@@ -151,6 +164,7 @@ final class FakeGiteaClient implements GiteaClient {
|
||||
String authorName,
|
||||
String authorCode) {
|
||||
lastAuthorCode = authorCode;
|
||||
commitCalls++;
|
||||
RepoData repo = repo(repository);
|
||||
Map<String, byte[]> snapshot =
|
||||
repo.head == null ? new TreeMap<>() : new TreeMap<>(repo.snapshots.get(repo.head));
|
||||
@@ -164,7 +178,9 @@ final class FakeGiteaClient implements GiteaClient {
|
||||
if (change.operation().equals("delete")) snapshot.remove(change.path());
|
||||
else snapshot.put(change.path(), change.bytes().clone());
|
||||
}
|
||||
String parent = repo.head;
|
||||
repo.head = sha();
|
||||
repo.parents.put(repo.head, parent);
|
||||
repo.snapshots.put(repo.head, snapshot);
|
||||
repo.history.addFirst(
|
||||
new HistoryEntry(repo.head, message, authorName, "2026-10-13T00:00:00Z"));
|
||||
@@ -181,7 +197,9 @@ final class FakeGiteaClient implements GiteaClient {
|
||||
List<HistoryEntry> history = repo(repository).history;
|
||||
int from = (int) Math.min(history.size(), (page - 1) * size);
|
||||
int to = (int) Math.min(history.size(), from + size);
|
||||
return new PageResponse<>(history.subList(from, to), page, size, history.size());
|
||||
return new PageResponse<>(history.subList(from, to).stream().map(item ->
|
||||
new HistoryEntry(item.commitSha(), WriteReceipt.displayMessage(item.message()),
|
||||
item.authorName(), item.authorDate())).toList(), page, size, history.size());
|
||||
}
|
||||
|
||||
@Override
|
||||
@@ -197,6 +215,7 @@ final class FakeGiteaClient implements GiteaClient {
|
||||
public synchronized RemoteRepository migrate(
|
||||
String source, String name, java.time.Duration timeout, String description) {
|
||||
migrations++;
|
||||
migrationNames.add(name);
|
||||
Repository repository = createRepository(name, true);
|
||||
RemoteRepository remote =
|
||||
new RemoteRepository(
|
||||
@@ -232,7 +251,16 @@ final class FakeGiteaClient implements GiteaClient {
|
||||
@Override
|
||||
public synchronized RemoteRepository inspect(String name) {
|
||||
RemoteRepository remote = migrated.get(name);
|
||||
if (remote == null) throw new GiteaFailure(404, "NOT_FOUND", "仓库不存在", false);
|
||||
if (remote == null && migrationNames.contains(name))
|
||||
throw new GiteaFailure(404, "NOT_FOUND", "仓库不存在", false);
|
||||
if (remote == null) {
|
||||
for (RepoData data : repositories.values()) {
|
||||
if (data.repository.name().equals(name))
|
||||
return new RemoteRepository(data.repository.id() + (wrongRepositoryIdentity ? 1 : 0),
|
||||
data.repository.owner(), name, true, false, false, null, "main");
|
||||
}
|
||||
throw new GiteaFailure(404, "NOT_FOUND", "仓库不存在", false);
|
||||
}
|
||||
return remote;
|
||||
}
|
||||
|
||||
@@ -264,4 +292,20 @@ final class FakeGiteaClient implements GiteaClient {
|
||||
remote.originalUrl(),
|
||||
branch));
|
||||
}
|
||||
|
||||
@Override
|
||||
public synchronized String findWriteReceipt(Repository repository, String baseSha,
|
||||
String operationCode, String digest) {
|
||||
if (failReceiptReads) throw new GiteaFailure(504, "READ_TIMEOUT", "核对读取超时", false);
|
||||
if (hideReceipts) return null;
|
||||
RepoData data = repo(repository);
|
||||
for (HistoryEntry entry : data.history) {
|
||||
if (entry.message().endsWith(WriteReceipt.suffix(operationCode, digest))) {
|
||||
if (wrongReceiptParent || !baseSha.equals(data.parents.get(entry.commitSha())))
|
||||
throw new GiteaFailure(409, "WRITE_RECEIPT_PARENT_MISMATCH", "父提交不匹配", false);
|
||||
return entry.commitSha();
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -12,18 +12,29 @@ final class FakeGiteaClient implements GiteaClient {
|
||||
private static final class RepoData {
|
||||
Repository repository;
|
||||
String head;
|
||||
Map<String, String> parents = new HashMap<>();
|
||||
Map<String, Map<String, byte[]>> snapshots = new HashMap<>();
|
||||
List<HistoryEntry> history = new ArrayList<>();
|
||||
}
|
||||
|
||||
private final Map<Long, RepoData> repositories = new HashMap<>();
|
||||
private final AtomicLong sequence = new AtomicLong();
|
||||
boolean hideReceipts;
|
||||
boolean failReceiptReads;
|
||||
boolean wrongReceiptParent;
|
||||
boolean wrongRepositoryIdentity;
|
||||
int commitCalls;
|
||||
boolean timeoutNextCommit;
|
||||
boolean timeoutNextCreate;
|
||||
boolean failNextCreate;
|
||||
|
||||
void reset() {
|
||||
repositories.clear();
|
||||
hideReceipts = false;
|
||||
failReceiptReads = false;
|
||||
wrongReceiptParent = false;
|
||||
wrongRepositoryIdentity = false;
|
||||
commitCalls = 0;
|
||||
timeoutNextCommit = false;
|
||||
timeoutNextCreate = false;
|
||||
failNextCreate = false;
|
||||
@@ -123,6 +134,7 @@ final class FakeGiteaClient implements GiteaClient {
|
||||
String message,
|
||||
String authorName,
|
||||
String authorCode) {
|
||||
commitCalls++;
|
||||
RepoData repo = repo(repository);
|
||||
Map<String, byte[]> snapshot =
|
||||
repo.head == null ? new TreeMap<>() : new TreeMap<>(repo.snapshots.get(repo.head));
|
||||
@@ -136,7 +148,9 @@ final class FakeGiteaClient implements GiteaClient {
|
||||
if (change.operation().equals("delete")) snapshot.remove(change.path());
|
||||
else snapshot.put(change.path(), change.bytes().clone());
|
||||
}
|
||||
String parent = repo.head;
|
||||
repo.head = sha();
|
||||
repo.parents.put(repo.head, parent);
|
||||
repo.snapshots.put(repo.head, snapshot);
|
||||
repo.history.addFirst(
|
||||
new HistoryEntry(repo.head, message, authorName, "2026-10-13T00:00:00Z"));
|
||||
@@ -153,6 +167,34 @@ final class FakeGiteaClient implements GiteaClient {
|
||||
List<HistoryEntry> history = repo(repository).history;
|
||||
int from = (int) Math.min(history.size(), (page - 1) * size);
|
||||
int to = (int) Math.min(history.size(), from + size);
|
||||
return new PageResponse<>(history.subList(from, to), page, size, history.size());
|
||||
return new PageResponse<>(history.subList(from, to).stream().map(item ->
|
||||
new HistoryEntry(item.commitSha(), WriteReceipt.displayMessage(item.message()),
|
||||
item.authorName(), item.authorDate())).toList(), page, size, history.size());
|
||||
}
|
||||
|
||||
@Override
|
||||
public synchronized String findWriteReceipt(Repository repository, String baseSha,
|
||||
String operationCode, String digest) {
|
||||
if (failReceiptReads) throw new GiteaFailure(504, "READ_TIMEOUT", "核对读取超时", false);
|
||||
if (hideReceipts) return null;
|
||||
RepoData data = repo(repository);
|
||||
for (HistoryEntry entry : data.history) {
|
||||
if (entry.message().endsWith(WriteReceipt.suffix(operationCode, digest))) {
|
||||
if (wrongReceiptParent || !baseSha.equals(data.parents.get(entry.commitSha())))
|
||||
throw new GiteaFailure(409, "WRITE_RECEIPT_PARENT_MISMATCH", "父提交不匹配", false);
|
||||
return entry.commitSha();
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
@Override
|
||||
public synchronized RemoteRepository inspect(String name) {
|
||||
for (RepoData data : repositories.values()) {
|
||||
if (data.repository.name().equals(name))
|
||||
return new RemoteRepository(data.repository.id() + (wrongRepositoryIdentity ? 1 : 0),
|
||||
data.repository.owner(), name, true, false, false, null, "main");
|
||||
}
|
||||
throw new GiteaFailure(404, "NOT_FOUND", "仓库不存在", false);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -98,6 +98,7 @@ class QueryEfficiencyTest {
|
||||
}
|
||||
for (String table :
|
||||
List.of(
|
||||
"repository_write_operation",
|
||||
"project_import_task",
|
||||
"project_like",
|
||||
"project_comment",
|
||||
|
||||
@@ -31,6 +31,6 @@ final class SchemaSupport {
|
||||
jdbc.execute(ddl);
|
||||
tables++;
|
||||
}
|
||||
if (tables != 5) throw new IllegalStateException("必须包含四张基础表和迁移任务表");
|
||||
if (tables != 6) throw new IllegalStateException("必须包含四张基础表、迁移任务表和仓库写入操作表");
|
||||
}
|
||||
}
|
||||
|
||||
@@ -38,6 +38,8 @@ import java.util.concurrent.*;
|
||||
"spring.datasource.password=",
|
||||
"spring.datasource.driver-class-name=org.h2.Driver",
|
||||
"logging.level.root=INFO",
|
||||
"vibemake.gitea.owner=vibemake",
|
||||
"vibemake.repository-write.worker-enabled=false",
|
||||
"logging.level.org.springframework=WARN"
|
||||
})
|
||||
@ActiveProfiles("test")
|
||||
@@ -82,6 +84,8 @@ class WorkflowTest {
|
||||
@Autowired ObjectMapper json;
|
||||
@Autowired JdbcTemplate jdbc;
|
||||
@Autowired FakeGiteaClient gitea;
|
||||
@Autowired com.linkerbot.vibemake.service.FileService fileService;
|
||||
@Autowired com.linkerbot.vibemake.service.write.RepositoryWrites writes;
|
||||
@Autowired OssProperties oss;
|
||||
@Autowired FakeOssObjectStore ossObjects;
|
||||
@Autowired UrdfProperties urdfLimits;
|
||||
@@ -1107,7 +1111,7 @@ class WorkflowTest {
|
||||
"PROJECT",
|
||||
"PROJECT_COMMENT",
|
||||
"PROJECT_LIKE",
|
||||
"PROJECT_IMPORT_TASK");
|
||||
"PROJECT_IMPORT_TASK", "REPOSITORY_WRITE_OPERATION");
|
||||
}
|
||||
|
||||
@Test
|
||||
@@ -1487,6 +1491,58 @@ class WorkflowTest {
|
||||
413);
|
||||
}
|
||||
|
||||
@Test
|
||||
void editorWriteRecoversInBackgroundWithoutChangingHttpContract() throws Exception {
|
||||
String email = "pending-editor@example.invalid";
|
||||
String owner = login(email);
|
||||
String code = create(owner, "编辑处理中");
|
||||
String sha = head(owner, code);
|
||||
Map<String, Object> input = Map.of("expectedCommitSha", sha, "message", "编辑说明",
|
||||
"changes", List.of(Map.of("action", "UPDATE_FILE", "path", "README.md", "content", "新说明")));
|
||||
gitea.hideReceipts = true;
|
||||
gitea.timeoutNextCommit = true;
|
||||
JsonNode failure = perform(body(auth(post("/api/workspace/projects/" + code + "/commits"), owner), input), 504);
|
||||
assertThat(failure.path("code").asText()).isEqualTo("REMOTE_RESULT_UNKNOWN");
|
||||
String operation = jdbc.queryForObject("SELECT operation_code FROM repository_write_operation WHERE project_code=?",
|
||||
String.class, code);
|
||||
int calls = gitea.commitCalls;
|
||||
perform(body(auth(post("/api/workspace/projects/" + code + "/commits"), owner), input), 409);
|
||||
assertThat(gitea.commitCalls).isEqualTo(calls);
|
||||
gitea.hideReceipts = false;
|
||||
jdbc.update("UPDATE repository_write_operation SET next_check_at=? WHERE operation_code=?",
|
||||
com.linkerbot.vibemake.service.support.ProjectSupport.now().minusSeconds(1), operation);
|
||||
writes.reconcileDue();
|
||||
String confirmed = head(owner, code);
|
||||
assertThat(confirmed).isNotEqualTo(sha);
|
||||
long userId = jdbc.queryForObject("SELECT id FROM sys_user WHERE email=?", Long.class, email);
|
||||
assertThat(fileService.commit(userId, code,
|
||||
json.convertValue(input, com.linkerbot.vibemake.bean.Contracts.CommitInput.class), operation).commitSha())
|
||||
.isEqualTo(confirmed);
|
||||
assertThat(gitea.commitCalls).isEqualTo(calls);
|
||||
}
|
||||
|
||||
@Test
|
||||
void rejectedEditorWriteReplaysFailureWithoutCallingGitea() throws Exception {
|
||||
String email = "rejected-editor@example.invalid";
|
||||
String owner = login(email);
|
||||
String code = create(owner, "拒绝重复创建");
|
||||
Map<String, Object> input = Map.of("expectedCommitSha", head(owner, code), "message", "不能重复创建",
|
||||
"changes", List.of(Map.of("action", "CREATE_FILE", "path", "README.md", "content", "重复")));
|
||||
perform(body(auth(post("/api/workspace/projects/" + code + "/commits"), owner), input), 409);
|
||||
String key = jdbc.queryForObject("SELECT operation_code FROM repository_write_operation WHERE project_code=?",
|
||||
String.class, code);
|
||||
int calls = gitea.commitCalls;
|
||||
long userId = jdbc.queryForObject("SELECT id FROM sys_user WHERE email=?", Long.class, email);
|
||||
assertThatThrownBy(() -> fileService.commit(userId, code,
|
||||
json.convertValue(input, com.linkerbot.vibemake.bean.Contracts.CommitInput.class), key))
|
||||
.isInstanceOfSatisfying(BusinessException.class, error -> assertThat(error.status()).isEqualTo(409));
|
||||
assertThat(gitea.commitCalls).isEqualTo(calls);
|
||||
assertThat(jdbc.queryForObject("SELECT status FROM repository_write_operation WHERE operation_code=?",
|
||||
String.class, key)).isEqualTo("FAILED");
|
||||
assertThat(jdbc.queryForObject("SELECT repo_write_status FROM project WHERE project_code=?",
|
||||
String.class, code)).isEqualTo("IDLE");
|
||||
}
|
||||
|
||||
@Test
|
||||
void unknownRemoteWritesAreNotRepeatedAndFailedCreatesRemainVisible() throws Exception {
|
||||
String owner = login("failure@example.com");
|
||||
@@ -1545,7 +1601,7 @@ class WorkflowTest {
|
||||
"实际成功")));
|
||||
perform(
|
||||
body(auth(post("/api/workspace/projects/" + code + "/commits"), owner), request),
|
||||
504);
|
||||
200);
|
||||
perform(
|
||||
body(auth(post("/api/workspace/projects/" + code + "/commits"), owner), request),
|
||||
409);
|
||||
|
||||
+30
@@ -0,0 +1,30 @@
|
||||
package com.linkerbot.vibemake.bean.entity;
|
||||
|
||||
import com.baomidou.mybatisplus.annotation.IdType;
|
||||
import com.baomidou.mybatisplus.annotation.TableId;
|
||||
import lombok.Getter;
|
||||
import lombok.Setter;
|
||||
import java.time.LocalDateTime;
|
||||
|
||||
/** 仓库写入凭据。不保存文件正文或认证信息。 */
|
||||
@Getter
|
||||
@Setter
|
||||
public class RepositoryWriteOperation {
|
||||
@TableId(type = IdType.INPUT)
|
||||
private String operationCode;
|
||||
private String projectCode;
|
||||
private Long repositoryId;
|
||||
private String repositoryOwner;
|
||||
private String repositoryName;
|
||||
private String baseCommitSha;
|
||||
private String requestDigest;
|
||||
private String commitMessage;
|
||||
private String status;
|
||||
private String resultCommitSha;
|
||||
private Integer failureStatus;
|
||||
private String lastErrorCode;
|
||||
private Integer checkCount;
|
||||
private LocalDateTime nextCheckAt;
|
||||
private LocalDateTime createdAt;
|
||||
private LocalDateTime updatedAt;
|
||||
}
|
||||
@@ -28,6 +28,11 @@ public interface FileService {
|
||||
|
||||
CommitView commit(long userId, String code, CommitInput input);
|
||||
|
||||
CommitView commit(long userId, String code, CommitInput input, String operationCode);
|
||||
|
||||
CommitView upload(long userId, String code, String expectedSha, String message, String directory,
|
||||
List<MultipartFile> files, List<String> paths, String operationCode);
|
||||
|
||||
CommitView upload(
|
||||
long userId,
|
||||
String code,
|
||||
|
||||
@@ -15,6 +15,12 @@ public interface GiteaClient {
|
||||
|
||||
record Change(String operation, String path, String blobSha, byte[] bytes) {}
|
||||
|
||||
/** 只从 main 可达的提交中查找,并校验完整尾注及唯一父提交;空结果不是失败证明。 */
|
||||
default String findWriteReceipt(Repository repository, String baseSha, String operationCode,
|
||||
String requestDigest) {
|
||||
throw new UnsupportedOperationException("未实现仓库写入凭据核对");
|
||||
}
|
||||
|
||||
record RemoteRepository(long id, String owner, String name, boolean privateRepo,
|
||||
boolean mirror, boolean empty, String originalUrl, String defaultBranch) {
|
||||
public Repository repository() { return new Repository(id, owner, name); }
|
||||
|
||||
+32
-1
@@ -425,7 +425,7 @@ public class RestGiteaClient implements GiteaClient {
|
||||
items.add(
|
||||
new HistoryEntry(
|
||||
node.path("sha").asText(),
|
||||
commit.path("message").asText(),
|
||||
WriteReceipt.displayMessage(commit.path("message").asText()),
|
||||
commit.path("author").path("name").asText(),
|
||||
commit.path("author").path("date").asText()));
|
||||
}
|
||||
@@ -436,4 +436,35 @@ public class RestGiteaClient implements GiteaClient {
|
||||
.orElse((long) items.size());
|
||||
return new PageResponse<>(items, page, size, total);
|
||||
}
|
||||
|
||||
@Override
|
||||
public String findWriteReceipt(Repository repo, String baseSha, String operationCode,
|
||||
String requestDigest) {
|
||||
// 普通文件写入被项目锁串行化,未知期间禁止下一次写入。限制扫描量防止异常仓库拖垮任务。
|
||||
// 超出扫描范围不推断失败,留在处理中并交给管理员兜底。
|
||||
String suffix = WriteReceipt.suffix(operationCode, requestDigest);
|
||||
String head = head(repo);
|
||||
String found = null;
|
||||
for (int page = 1; page <= 20; page++) {
|
||||
JsonNode entries = parse(request("GET", repoPath(repo)
|
||||
+ "/commits?sha=" + head + "&page=" + page + "&limit=50", null).body(), false);
|
||||
if (!entries.isArray())
|
||||
throw new GiteaFailure(502, "GITEA_INVALID_RESPONSE", "提交记录格式异常", false);
|
||||
for (JsonNode entry : entries) {
|
||||
if (entry.path("commit").path("message").asText().stripTrailing().endsWith(suffix)) {
|
||||
JsonNode parents = entry.path("parents");
|
||||
if (parents.size() != 1 || !baseSha.equals(parents.path(0).path("sha").asText()))
|
||||
throw new GiteaFailure(409, "WRITE_RECEIPT_PARENT_MISMATCH", "提交父版本不匹配", false);
|
||||
String sha = FilePaths.sha(entry.path("sha").asText());
|
||||
if (found != null && !found.equals(sha))
|
||||
throw new GiteaFailure(409, "WRITE_RECEIPT_AMBIGUOUS", "存在多个操作凭据", false);
|
||||
found = sha;
|
||||
}
|
||||
}
|
||||
if (entries.size() < 50) break;
|
||||
}
|
||||
if (!head.equals(head(repo)))
|
||||
throw new GiteaFailure(409, "WRITE_RECEIPT_BRANCH_CHANGED", "核对期间分支版本已变化", false);
|
||||
return found;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,14 @@
|
||||
package com.linkerbot.vibemake.service.gitea;
|
||||
|
||||
/** 固定尾注同时绑定操作编码和请求摘要,普通提交信息不会被当作凭据。 */
|
||||
public final class WriteReceipt {
|
||||
private WriteReceipt() {}
|
||||
|
||||
public static String suffix(String code, String digest) {
|
||||
return "\n\nVibemake-Operation: " + code + "\nVibemake-Request: " + digest;
|
||||
}
|
||||
|
||||
public static String displayMessage(String message) {
|
||||
return message.replaceFirst("\\n\\nVibemake-Operation: [a-f0-9]{32}\\nVibemake-Request: [a-f0-9]{64}\\s*$", "");
|
||||
}
|
||||
}
|
||||
+27
-9
@@ -11,6 +11,8 @@ import com.linkerbot.vibemake.service.gitea.*;
|
||||
import com.linkerbot.vibemake.service.state.RateLimits;
|
||||
import com.linkerbot.vibemake.service.support.ProjectActors;
|
||||
import com.linkerbot.vibemake.service.support.ProjectSupport;
|
||||
import com.linkerbot.vibemake.service.write.RepositoryWrites;
|
||||
import com.linkerbot.vibemake.service.write.WriteRequestDigest;
|
||||
|
||||
import lombok.RequiredArgsConstructor;
|
||||
|
||||
@@ -33,6 +35,8 @@ public class FileServiceImpl implements FileService {
|
||||
private final UserService users;
|
||||
private final GiteaClient gitea;
|
||||
private final VibemakeProperties config;
|
||||
private final RepositoryWrites writes;
|
||||
private final WriteRequestDigest digests;
|
||||
|
||||
@Override
|
||||
public TreeView ownerTree(long userId, String code, String path, String commitSha) {
|
||||
@@ -137,8 +141,14 @@ public class FileServiceImpl implements FileService {
|
||||
|
||||
@Override
|
||||
public CommitView commit(long userId, String code, CommitInput input) {
|
||||
return commit(userId, code, input, null);
|
||||
}
|
||||
|
||||
@Override
|
||||
public CommitView commit(long userId, String code, CommitInput input, String operationCode) {
|
||||
projects.readyOwner(userId, code);
|
||||
rateLimits.operation("file-write", userId);
|
||||
return projects.write(
|
||||
return writes.execute(userId, code, operationCode, digests.commit(input), request -> projects.write(
|
||||
userId,
|
||||
code,
|
||||
project -> {
|
||||
@@ -191,19 +201,18 @@ public class FileServiceImpl implements FileService {
|
||||
size(bytes, config.getMaxRequestBytes());
|
||||
ProjectActors.Author author = ProjectActors.author(users, userId);
|
||||
CommitView result =
|
||||
gitea.commit(
|
||||
repo,
|
||||
writes.commit(
|
||||
request, project, sha,
|
||||
changes,
|
||||
input.message(),
|
||||
author.nickname(),
|
||||
author.code());
|
||||
author);
|
||||
log.info(
|
||||
"文件提交 projectCode={} commitSha={} fileCount={}",
|
||||
code,
|
||||
result.commitSha(),
|
||||
changes.size());
|
||||
return result;
|
||||
});
|
||||
}));
|
||||
}
|
||||
|
||||
@Override
|
||||
@@ -215,6 +224,14 @@ public class FileServiceImpl implements FileService {
|
||||
String directory,
|
||||
List<MultipartFile> files,
|
||||
List<String> paths) {
|
||||
return upload(userId, code, expectedSha, message, directory, files, paths, null);
|
||||
}
|
||||
|
||||
@Override
|
||||
public CommitView upload(long userId, String code, String expectedSha, String message,
|
||||
String directory, List<MultipartFile> files, List<String> paths,
|
||||
String operationCode) {
|
||||
projects.readyOwner(userId, code);
|
||||
rateLimits.operation("file-write", userId);
|
||||
if (message == null || message.isBlank() || message.length() > 1000)
|
||||
throw BusinessException.badRequest("提交说明不合法");
|
||||
@@ -224,7 +241,8 @@ public class FileServiceImpl implements FileService {
|
||||
|| (paths != null && paths.size() != files.size()))
|
||||
throw BusinessException.badRequest("文件和路径数量不合法");
|
||||
String prefix = FilePaths.directory(directory);
|
||||
return projects.write(
|
||||
return writes.execute(userId, code, operationCode,
|
||||
digests.upload(expectedSha, message, directory, files, paths), request -> projects.write(
|
||||
userId,
|
||||
code,
|
||||
project -> {
|
||||
@@ -264,14 +282,14 @@ public class FileServiceImpl implements FileService {
|
||||
}
|
||||
ProjectActors.Author author = ProjectActors.author(users, userId);
|
||||
CommitView result =
|
||||
gitea.commit(repo, changes, message, author.nickname(), author.code());
|
||||
writes.commit(request, project, sha, changes, message, author);
|
||||
log.info(
|
||||
"文件上传 projectCode={} commitSha={} fileCount={}",
|
||||
code,
|
||||
result.commitSha(),
|
||||
changes.size());
|
||||
return result;
|
||||
});
|
||||
}));
|
||||
}
|
||||
|
||||
@Override
|
||||
|
||||
+8
@@ -0,0 +1,8 @@
|
||||
package com.linkerbot.vibemake.service.mapper;
|
||||
|
||||
import com.baomidou.mybatisplus.core.mapper.BaseMapper;
|
||||
import com.linkerbot.vibemake.bean.entity.RepositoryWriteOperation;
|
||||
import org.apache.ibatis.annotations.Mapper;
|
||||
|
||||
@Mapper
|
||||
public interface RepositoryWriteOperationMapper extends BaseMapper<RepositoryWriteOperation> {}
|
||||
+6
-2
@@ -28,7 +28,7 @@ import javax.sql.DataSource;
|
||||
|
||||
/** 数据库命名锁不持有业务事务;专用连接在整个项目操作期间保持占用。 */
|
||||
@Component
|
||||
public class ProjectLocks {
|
||||
public class ProjectLocks implements AutoCloseable {
|
||||
private static final org.slf4j.Logger log =
|
||||
org.slf4j.LoggerFactory.getLogger(ProjectLocks.class);
|
||||
private final ReentrantLock[] locks = new ReentrantLock[128];
|
||||
@@ -83,6 +83,10 @@ public class ProjectLocks {
|
||||
() -> {
|
||||
if (writing.get().contains(code)) return action.get();
|
||||
Project guard = guard(code);
|
||||
// 已持有共享项目锁,以数据库为准同步其他进程确认的恢复结果。
|
||||
if (guard != null && "IDLE".equals(guard.getRepoWriteStatus())
|
||||
&& uncertain.remove(code))
|
||||
log.info("同步已恢复的仓库写入状态 projectCode={} writeStatus=IDLE", code);
|
||||
boolean memoryUncertain = uncertain.contains(code);
|
||||
if (memoryUncertain
|
||||
|| guard != null && !"IDLE".equals(guard.getRepoWriteStatus())) {
|
||||
@@ -212,7 +216,7 @@ public class ProjectLocks {
|
||||
}
|
||||
|
||||
private BusinessException blocked() {
|
||||
return new BusinessException(409, "REMOTE_RESULT_UNKNOWN", "上次仓库写入结果待人工核对,暂不能继续写入");
|
||||
return new BusinessException(409, "REMOTE_RESULT_UNKNOWN", "仓库写入结果正在核对,请稍后查询处理结果");
|
||||
}
|
||||
|
||||
@PreDestroy
|
||||
|
||||
+45
@@ -0,0 +1,45 @@
|
||||
package com.linkerbot.vibemake.service.write;
|
||||
|
||||
import lombok.RequiredArgsConstructor;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
|
||||
import org.springframework.context.annotation.Configuration;
|
||||
import org.springframework.scheduling.annotation.EnableScheduling;
|
||||
import org.springframework.scheduling.annotation.Scheduled;
|
||||
|
||||
@Configuration
|
||||
@EnableScheduling
|
||||
@RequiredArgsConstructor
|
||||
@ConditionalOnProperty(name = "vibemake.repository-write.worker-enabled", matchIfMissing = true)
|
||||
public class RepositoryWriteWorker {
|
||||
private static final Logger log = LoggerFactory.getLogger(RepositoryWriteWorker.class);
|
||||
private final RepositoryWrites writes;
|
||||
private final java.util.concurrent.ExecutorService executor = java.util.concurrent.Executors
|
||||
.newSingleThreadExecutor(Thread.ofPlatform().name("repository-write-check").factory());
|
||||
private final java.util.concurrent.atomic.AtomicBoolean running = new java.util.concurrent.atomic.AtomicBoolean();
|
||||
|
||||
@Scheduled(fixedDelay = 1000, initialDelay = 15000)
|
||||
public void reconcile() {
|
||||
if (executor.isShutdown() || !running.compareAndSet(false, true)) return;
|
||||
try {
|
||||
executor.execute(() -> {
|
||||
try {
|
||||
writes.reconcileDue();
|
||||
} catch (RuntimeException error) {
|
||||
com.linkerbot.vibemake.common.SafeExceptionLog.write(log, "仓库写入核对任务失败", error);
|
||||
} finally {
|
||||
running.set(false);
|
||||
}
|
||||
});
|
||||
} catch (java.util.concurrent.RejectedExecutionException shutdown) {
|
||||
running.set(false);
|
||||
}
|
||||
}
|
||||
|
||||
@jakarta.annotation.PreDestroy
|
||||
public void close() throws InterruptedException {
|
||||
executor.shutdown();
|
||||
if (!executor.awaitTermination(5, java.util.concurrent.TimeUnit.SECONDS)) executor.shutdownNow();
|
||||
}
|
||||
}
|
||||
+239
@@ -0,0 +1,239 @@
|
||||
package com.linkerbot.vibemake.service.write;
|
||||
|
||||
import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper;
|
||||
import com.linkerbot.vibemake.bean.Contracts.CommitView;
|
||||
import com.linkerbot.vibemake.bean.entity.Project;
|
||||
import com.linkerbot.vibemake.bean.entity.RepositoryWriteOperation;
|
||||
import com.linkerbot.vibemake.common.*;
|
||||
import com.linkerbot.vibemake.service.ProjectService;
|
||||
import com.linkerbot.vibemake.service.gitea.*;
|
||||
import com.linkerbot.vibemake.service.mapper.ProjectMapper;
|
||||
import com.linkerbot.vibemake.service.mapper.RepositoryWriteOperationMapper;
|
||||
import com.linkerbot.vibemake.service.support.ProjectActors;
|
||||
import com.linkerbot.vibemake.service.support.ProjectLocks;
|
||||
import com.linkerbot.vibemake.service.support.ProjectSupport;
|
||||
import lombok.RequiredArgsConstructor;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
import org.springframework.stereotype.Service;
|
||||
import org.springframework.transaction.support.TransactionTemplate;
|
||||
import java.time.LocalDateTime;
|
||||
import java.util.List;
|
||||
import java.util.Objects;
|
||||
import java.util.function.Function;
|
||||
|
||||
/** 数据库凭据先于远端写入落盘。未知结果只读取核对,绝不自动重发写请求。 */
|
||||
@Service
|
||||
@RequiredArgsConstructor
|
||||
public class RepositoryWrites {
|
||||
private static final Logger log = LoggerFactory.getLogger(RepositoryWrites.class);
|
||||
private static final List<String> ACTIVE = List.of("PROCESSING", "NEEDS_REVIEW");
|
||||
private final RepositoryWriteOperationMapper operations;
|
||||
private final ProjectMapper projectMapper;
|
||||
private final ProjectService projects;
|
||||
private final ProjectLocks locks;
|
||||
private final GiteaClient gitea;
|
||||
private final TransactionTemplate transactions;
|
||||
|
||||
public record Request(String code, String digest) {}
|
||||
|
||||
public CommitView execute(long userId, String projectCode, String key, String digest,
|
||||
Function<Request, CommitView> action) {
|
||||
String operationCode = key == null ? Codes.next() : key.replace("-", "").toLowerCase(java.util.Locale.ROOT);
|
||||
if (!operationCode.matches("[a-f0-9]{32}"))
|
||||
throw BusinessException.badRequest("操作编码必须为 UUID 或 32 位十六进制编码");
|
||||
return locks.serial(projectCode, () -> {
|
||||
projects.readyOwner(userId, projectCode); // 重放也必须重新鉴权。
|
||||
RepositoryWriteOperation previous = operations.selectById(operationCode);
|
||||
if (previous != null) {
|
||||
if (!projectCode.equals(previous.getProjectCode()) || !digest.equals(previous.getRequestDigest()))
|
||||
throw new BusinessException(409, "IDEMPOTENCY_KEY_REUSED", "操作编码已用于不同请求,请勿复用");
|
||||
return replay(previous);
|
||||
}
|
||||
RepositoryWriteOperation pending = active(projectCode);
|
||||
if (pending != null)
|
||||
throw pending(pending);
|
||||
try {
|
||||
CommitView result = action.apply(new Request(operationCode, digest));
|
||||
RepositoryWriteOperation operation = operations.selectById(operationCode);
|
||||
if (operation == null) throw new IllegalStateException("仓库写入缺少持久化凭据");
|
||||
complete(operation, result.commitSha(), "SUCCEEDED");
|
||||
return result;
|
||||
} catch (RuntimeException error) {
|
||||
RepositoryWriteOperation operation = operations.selectById(operationCode);
|
||||
if (operation == null) throw error; // 参数校验、读取失败发生在任何远端写入之前。
|
||||
if (!projectCode.equals(operation.getProjectCode()) || !digest.equals(operation.getRequestDigest()))
|
||||
throw new BusinessException(409, "IDEMPOTENCY_KEY_REUSED", "操作编码已用于不同请求,请勿复用");
|
||||
boolean definite = error instanceof GiteaFailure remote && !remote.uncertain();
|
||||
if (definite) {
|
||||
BusinessException business = (BusinessException) error;
|
||||
operation.setStatus("FAILED");
|
||||
operation.setFailureStatus(business.status());
|
||||
operation.setLastErrorCode(business.code());
|
||||
save(operation);
|
||||
throw error;
|
||||
}
|
||||
locks.markUncertain(projectCode);
|
||||
operation.setLastErrorCode(error instanceof BusinessException business
|
||||
? business.code() : "WRITE_RESULT_UNKNOWN");
|
||||
save(operation);
|
||||
log.warn("仓库写入转后台核对 projectCode={} operationCode={} errorCode={} errorType={}",
|
||||
projectCode, operationCode, operation.getLastErrorCode(), error.getClass().getSimpleName());
|
||||
if (!(error instanceof BusinessException)) SafeExceptionLog.write(log, "仓库提交结果未知", error);
|
||||
throw error;
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
/** 必须在 ProjectLocks.write 内调用;此处是唯一一次发起远端写请求的位置。 */
|
||||
public CommitView commit(Request request, Project project, String baseSha,
|
||||
List<GiteaClient.Change> changes, String message,
|
||||
ProjectActors.Author author) {
|
||||
GiteaClient.Repository repository = projects.repository(project);
|
||||
RepositoryWriteOperation operation = new RepositoryWriteOperation();
|
||||
operation.setOperationCode(request.code());
|
||||
operation.setProjectCode(project.getProjectCode());
|
||||
operation.setRepositoryId(repository.id());
|
||||
operation.setRepositoryOwner(repository.owner());
|
||||
operation.setRepositoryName(repository.name());
|
||||
operation.setBaseCommitSha(baseSha);
|
||||
operation.setRequestDigest(request.digest());
|
||||
operation.setCommitMessage(message);
|
||||
operation.setStatus("PROCESSING");
|
||||
operation.setCheckCount(0);
|
||||
operation.setCreatedAt(ProjectSupport.now());
|
||||
operation.setUpdatedAt(operation.getCreatedAt());
|
||||
operation.setNextCheckAt(operation.getCreatedAt().plusSeconds(15));
|
||||
try {
|
||||
operations.insert(operation);
|
||||
} catch (org.springframework.dao.DuplicateKeyException collision) {
|
||||
throw new BusinessException(409, "IDEMPOTENCY_KEY_REUSED", "操作编码已用于其他请求,请勿复用");
|
||||
}
|
||||
log.info("仓库写入开始 projectCode={} operationCode={} baseCommitSha={} fileCount={}",
|
||||
project.getProjectCode(), request.code(), baseSha, changes.size());
|
||||
try {
|
||||
CommitView result = gitea.commit(repository, changes,
|
||||
message + WriteReceipt.suffix(request.code(), request.digest()), author.nickname(), author.code());
|
||||
return new CommitView(result.commitSha(), message);
|
||||
} catch (GiteaFailure error) {
|
||||
if (!error.uncertain()) throw error;
|
||||
// 尝试立即消除响应丢失的影响,查询失败依旧保留未知,不覆盖原始错误确定性。
|
||||
String receipt = receipt(operation, project);
|
||||
if (receipt != null) return new CommitView(receipt, message);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
private CommitView replay(RepositoryWriteOperation operation) {
|
||||
return switch (operation.getStatus()) {
|
||||
case "SUCCEEDED" -> new CommitView(operation.getResultCommitSha(), operation.getCommitMessage());
|
||||
case "FAILED" -> throw new BusinessException(operation.getFailureStatus(),
|
||||
operation.getLastErrorCode(), "该操作已被仓库拒绝,请刷新后使用新的操作编码提交");
|
||||
default -> throw pending(operation);
|
||||
};
|
||||
}
|
||||
|
||||
private BusinessException pending(RepositoryWriteOperation operation) {
|
||||
return new BusinessException(409, "REMOTE_RESULT_UNKNOWN", "仓库提交结果正在后台核对,请稍后刷新工作区,暂勿重复提交",
|
||||
java.util.Map.of("projectCode", operation.getProjectCode()));
|
||||
}
|
||||
|
||||
public RepositoryWriteOperation active(String code) {
|
||||
return operations.selectOne(new LambdaQueryWrapper<RepositoryWriteOperation>()
|
||||
.eq(RepositoryWriteOperation::getProjectCode, code)
|
||||
.in(RepositoryWriteOperation::getStatus, ACTIVE)
|
||||
.orderByDesc(RepositoryWriteOperation::getCreatedAt).last("LIMIT 1"));
|
||||
}
|
||||
|
||||
private boolean slowCheckDue(RepositoryWriteOperation operation) {
|
||||
return !ProjectSupport.now().isBefore(operation.getCreatedAt().plusMinutes(10));
|
||||
}
|
||||
|
||||
/** 重启后直接扫描数据库,无内存任务队列;多实例由相同项目命名锁串行核对。 */
|
||||
public void reconcileDue() {
|
||||
List<RepositoryWriteOperation> due = operations.selectList(new LambdaQueryWrapper<RepositoryWriteOperation>()
|
||||
.in(RepositoryWriteOperation::getStatus, ACTIVE)
|
||||
.le(RepositoryWriteOperation::getNextCheckAt, ProjectSupport.now())
|
||||
.orderByAsc(RepositoryWriteOperation::getNextCheckAt).last("LIMIT 20"));
|
||||
for (RepositoryWriteOperation candidate : due) {
|
||||
try {
|
||||
locks.serial(candidate.getProjectCode(), () -> {
|
||||
reconcile(candidate.getOperationCode());
|
||||
return null;
|
||||
});
|
||||
} catch (RuntimeException error) {
|
||||
log.warn("仓库后台核对未完成 projectCode={} operationCode={} errorCode={} errorType={}",
|
||||
candidate.getProjectCode(), candidate.getOperationCode(),
|
||||
error instanceof BusinessException business ? business.code() : "CHECK_FAILED",
|
||||
error.getClass().getSimpleName());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private void reconcile(String operationCode) {
|
||||
RepositoryWriteOperation operation = operations.selectById(operationCode);
|
||||
if (operation == null || !ACTIVE.contains(operation.getStatus())
|
||||
|| operation.getNextCheckAt().isAfter(ProjectSupport.now())) return;
|
||||
Project project = projectMapper.selectOne(new LambdaQueryWrapper<Project>()
|
||||
.eq(Project::getProjectCode, operation.getProjectCode()));
|
||||
String receipt = project == null ? null : receipt(operation, project);
|
||||
operation.setCheckCount(operation.getCheckCount() + 1);
|
||||
if (receipt != null) {
|
||||
complete(operation, receipt, "SUCCEEDED");
|
||||
return;
|
||||
}
|
||||
boolean overdue = slowCheckDue(operation);
|
||||
operation.setStatus(overdue ? "NEEDS_REVIEW" : "PROCESSING");
|
||||
operation.setNextCheckAt(ProjectSupport.now().plusSeconds(overdue ? 60 : 15));
|
||||
save(operation);
|
||||
log.info("仓库写入继续核对 projectCode={} operationCode={} status={} checkCount={} errorCode={}",
|
||||
operation.getProjectCode(), operationCode, operation.getStatus(),
|
||||
operation.getCheckCount(), operation.getLastErrorCode());
|
||||
}
|
||||
|
||||
private String receipt(RepositoryWriteOperation operation, Project project) {
|
||||
try {
|
||||
if (!"READY".equals(project.getRepoStatus())
|
||||
|| !Objects.equals(project.getGiteaRepoId(), operation.getRepositoryId())
|
||||
|| !Objects.equals(project.getGiteaOwner(), operation.getRepositoryOwner())
|
||||
|| !Objects.equals(project.getGiteaRepoName(), operation.getRepositoryName()))
|
||||
throw new GiteaFailure(409, "WRITE_REPOSITORY_MISMATCH", "项目仓库身份发生变化", false);
|
||||
GiteaClient.RemoteRepository remote = gitea.inspect(operation.getRepositoryName());
|
||||
if (remote.id() != operation.getRepositoryId()
|
||||
|| !remote.owner().equals(operation.getRepositoryOwner())
|
||||
|| !remote.name().equals(operation.getRepositoryName())
|
||||
|| !remote.privateRepo() || remote.mirror() || remote.empty()
|
||||
|| !"main".equals(remote.defaultBranch()))
|
||||
throw new GiteaFailure(409, "WRITE_REPOSITORY_MISMATCH", "远端仓库身份发生变化", false);
|
||||
String sha = gitea.findWriteReceipt(remote.repository(), operation.getBaseCommitSha(),
|
||||
operation.getOperationCode(), operation.getRequestDigest());
|
||||
if (sha == null) operation.setLastErrorCode("WRITE_RECEIPT_NOT_FOUND");
|
||||
return sha;
|
||||
} catch (RuntimeException error) {
|
||||
operation.setLastErrorCode(error instanceof BusinessException business
|
||||
? business.code() : "WRITE_RECEIPT_READ_FAILED");
|
||||
log.warn("仓库写入凭据读取未确认 projectCode={} operationCode={} errorCode={} errorType={}",
|
||||
operation.getProjectCode(), operation.getOperationCode(), operation.getLastErrorCode(),
|
||||
error.getClass().getSimpleName());
|
||||
if (!(error instanceof BusinessException)) SafeExceptionLog.write(log, "仓库凭据核对异常", error);
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
private void complete(RepositoryWriteOperation operation, String sha, String status) {
|
||||
// 项目恢复与操作终态在同一事务提交;任务获得锁后重读终态,旧任务不会清除新操作的保护。
|
||||
transactions.executeWithoutResult(transaction -> {
|
||||
operation.setStatus(status);
|
||||
operation.setResultCommitSha(sha);
|
||||
save(operation);
|
||||
locks.confirmKnownResult(operation.getProjectCode());
|
||||
});
|
||||
log.info("仓库写入结果已落盘 projectCode={} operationCode={} status={} commitSha={}",
|
||||
operation.getProjectCode(), operation.getOperationCode(), status, sha);
|
||||
}
|
||||
|
||||
private void save(RepositoryWriteOperation operation) {
|
||||
operation.setUpdatedAt(ProjectSupport.now());
|
||||
if (operations.updateById(operation) != 1) throw new IllegalStateException("仓库写入记录更新失败");
|
||||
}
|
||||
}
|
||||
+78
@@ -0,0 +1,78 @@
|
||||
package com.linkerbot.vibemake.service.write;
|
||||
|
||||
import com.fasterxml.jackson.core.JsonProcessingException;
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
import com.linkerbot.vibemake.bean.Contracts.CommitInput;
|
||||
import com.linkerbot.vibemake.common.BusinessException;
|
||||
import com.linkerbot.vibemake.common.FilePaths;
|
||||
import com.linkerbot.vibemake.service.config.VibemakeProperties;
|
||||
import lombok.RequiredArgsConstructor;
|
||||
import org.springframework.stereotype.Component;
|
||||
import org.springframework.web.multipart.MultipartFile;
|
||||
import java.io.IOException;
|
||||
import java.io.InputStream;
|
||||
import java.security.MessageDigest;
|
||||
import java.security.NoSuchAlgorithmException;
|
||||
import java.util.ArrayList;
|
||||
import java.util.HexFormat;
|
||||
import java.util.List;
|
||||
|
||||
@Component
|
||||
@RequiredArgsConstructor
|
||||
public class WriteRequestDigest {
|
||||
private final ObjectMapper json;
|
||||
private final VibemakeProperties config;
|
||||
|
||||
public String commit(CommitInput input) {
|
||||
return digest(List.of("commit-v1", input));
|
||||
}
|
||||
|
||||
public String upload(String expected, String message, String directory,
|
||||
List<MultipartFile> files, List<String> paths) {
|
||||
if (message == null || files == null || files.isEmpty() || files.size() > 500
|
||||
|| paths != null && paths.size() != files.size())
|
||||
throw BusinessException.badRequest("文件和路径数量不合法");
|
||||
List<Object> items = new ArrayList<>();
|
||||
long total = 0;
|
||||
for (int i = 0; i < files.size(); i++) {
|
||||
MultipartFile file = files.get(i);
|
||||
total += file.getSize();
|
||||
if (file.getSize() > config.getMaxFileBytes() || total > config.getMaxRequestBytes())
|
||||
throw new BusinessException(413, "FILE_TOO_LARGE", "文件或请求超过大小限制");
|
||||
MessageDigest hash = sha256();
|
||||
long bytes = 0;
|
||||
try (InputStream stream = file.getInputStream()) {
|
||||
byte[] buffer = new byte[8192];
|
||||
int count;
|
||||
while ((count = stream.read(buffer)) != -1) {
|
||||
bytes += count;
|
||||
if (bytes > config.getMaxFileBytes())
|
||||
throw new BusinessException(413, "FILE_TOO_LARGE", "文件超过大小限制");
|
||||
hash.update(buffer, 0, count);
|
||||
}
|
||||
} catch (IOException error) {
|
||||
throw BusinessException.badRequest("读取上传文件失败");
|
||||
}
|
||||
items.add(List.of(FilePaths.file(paths == null ? file.getOriginalFilename() : paths.get(i)),
|
||||
bytes, HexFormat.of().formatHex(hash.digest())));
|
||||
}
|
||||
return digest(List.of("upload-v1", FilePaths.sha(expected), message,
|
||||
FilePaths.directory(directory), items));
|
||||
}
|
||||
|
||||
private String digest(Object value) {
|
||||
try {
|
||||
return HexFormat.of().formatHex(sha256().digest(json.writeValueAsBytes(value)));
|
||||
} catch (JsonProcessingException error) {
|
||||
throw new IllegalStateException("无法生成仓库请求摘要", error);
|
||||
}
|
||||
}
|
||||
|
||||
private MessageDigest sha256() {
|
||||
try {
|
||||
return MessageDigest.getInstance("SHA-256");
|
||||
} catch (NoSuchAlgorithmException error) {
|
||||
throw new IllegalStateException(error);
|
||||
}
|
||||
}
|
||||
}
|
||||
+78
@@ -0,0 +1,78 @@
|
||||
package com.linkerbot.vibemake.service.gitea;
|
||||
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
import com.linkerbot.vibemake.service.config.VibemakeProperties;
|
||||
import com.sun.net.httpserver.HttpServer;
|
||||
import org.junit.jupiter.params.ParameterizedTest;
|
||||
import org.junit.jupiter.params.provider.ValueSource;
|
||||
import java.net.InetSocketAddress;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.concurrent.atomic.AtomicInteger;
|
||||
import static org.assertj.core.api.Assertions.*;
|
||||
|
||||
class GiteaWriteReceiptTest {
|
||||
@ParameterizedTest
|
||||
@ValueSource(strings = {"valid", "wrongOperation", "wrongDigest", "wrongParent", "duplicate", "branchChanged", "readFailure"})
|
||||
void receiptRequiresExactIdentityAndParentOnStableMainHistory(String mode) throws Exception {
|
||||
String code = "a".repeat(32);
|
||||
String digest = "b".repeat(64);
|
||||
String base = "1".repeat(40);
|
||||
String sha = "2".repeat(40);
|
||||
String head = "3".repeat(40);
|
||||
ObjectMapper json = new ObjectMapper();
|
||||
HttpServer server = HttpServer.create(new InetSocketAddress("127.0.0.1", 0), 0);
|
||||
AtomicInteger heads = new AtomicInteger();
|
||||
server.createContext("/api/v1/repos/vibemake/repo/branches/main", exchange -> {
|
||||
String value = mode.equals("branchChanged") && heads.incrementAndGet() > 1 ? "4".repeat(40) : head;
|
||||
byte[] body = json.writeValueAsBytes(Map.of("commit", Map.of("id", value)));
|
||||
exchange.sendResponseHeaders(200, body.length);
|
||||
exchange.getResponseBody().write(body);
|
||||
exchange.close();
|
||||
});
|
||||
server.createContext("/api/v1/repos/vibemake/repo/commits", exchange -> {
|
||||
assertThat(exchange.getRequestMethod()).isEqualTo("GET");
|
||||
assertThat(exchange.getRequestURI().getQuery()).contains("sha=" + head);
|
||||
String message = "用户提交" + WriteReceipt.suffix(mode.equals("wrongOperation") ? "c".repeat(32) : code,
|
||||
mode.equals("wrongDigest") ? "c".repeat(64) : digest);
|
||||
Map<String, Object> match = Map.of("sha", sha, "commit", Map.of("message", message),
|
||||
"parents", List.of(Map.of("sha", mode.equals("wrongParent") ? head : base)));
|
||||
Map<String, Object> unrelated = Map.of("sha", head, "commit", Map.of("message", "另一次提交"),
|
||||
"parents", List.of(Map.of("sha", sha)));
|
||||
List<Object> entries = new java.util.ArrayList<>(List.of(unrelated, match));
|
||||
if (mode.equals("duplicate")) entries.add(Map.of("sha", "5".repeat(40), "commit", Map.of("message", message),
|
||||
"parents", List.of(Map.of("sha", base))));
|
||||
byte[] body = mode.equals("readFailure") ? "failure".getBytes(StandardCharsets.UTF_8)
|
||||
: json.writeValueAsBytes(entries);
|
||||
exchange.sendResponseHeaders(mode.equals("readFailure") ? 503 : 200, body.length);
|
||||
exchange.getResponseBody().write(body);
|
||||
exchange.close();
|
||||
});
|
||||
server.start();
|
||||
try {
|
||||
VibemakeProperties config = new VibemakeProperties();
|
||||
config.getGitea().setBaseUrl("http://127.0.0.1:" + server.getAddress().getPort());
|
||||
config.getGitea().setToken("unit-test-placeholder");
|
||||
RestGiteaClient client = new RestGiteaClient(config, json);
|
||||
GiteaClient.Repository repo = new GiteaClient.Repository(1, "vibemake", "repo");
|
||||
if (List.of("wrongParent", "duplicate", "branchChanged", "readFailure").contains(mode)) {
|
||||
assertThatThrownBy(() -> client.findWriteReceipt(repo, base, code, digest))
|
||||
.isInstanceOfSatisfying(GiteaFailure.class, error -> {
|
||||
assertThat(error.uncertain()).isFalse();
|
||||
assertThat(error.code()).isEqualTo(switch (mode) {
|
||||
case "wrongParent" -> "WRITE_RECEIPT_PARENT_MISMATCH";
|
||||
case "duplicate" -> "WRITE_RECEIPT_AMBIGUOUS";
|
||||
case "branchChanged" -> "WRITE_RECEIPT_BRANCH_CHANGED";
|
||||
default -> "GITEA_ERROR";
|
||||
});
|
||||
});
|
||||
} else {
|
||||
assertThat(client.findWriteReceipt(repo, base, code, digest))
|
||||
.isEqualTo(mode.equals("valid") ? sha : null);
|
||||
}
|
||||
} finally {
|
||||
server.stop(0);
|
||||
}
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user