git clone功能

网站登录 → 创建项目 Git Token → clone → 本地修改、commit → push main → 网站工作区刷新后显示最新内容 → 网站手动发布到社区
This commit is contained in:
lxp
2026-11-03 10:42:08 +08:00
parent b02d070c39
commit 1ae21eef5b
37 changed files with 4204 additions and 96 deletions
+24
View File
@@ -304,3 +304,27 @@ bash scripts/test-admin-mysql.sh
项目响应采用数据库批量查询,不缓存业务列表。列表和 AI 检索不读取体积较大的 URDF 文件清单,Fork 来源仅批量读取标识和编码。按业务服务层统计(不包含安全过滤器独立账号查询),20 条带 Fork 来源、登录用户点赞和导入来源的项目列表最多 8 次 SQL,评论列表最多 4 次,后台用户列表最多 3 次;避免记录数增加造成逐条查询。昵称、互动和下架可见性仍读取当前数据。
普通 `AI_ENABLED=false OSS_ENABLED=false mvn verify` 使用只存在于 tests 分类制品中的状态存储,不依赖 Redis,生产 jar 不包含该实现。真实 Redis 联调不自动读取 `.env`,通过 `REDIS_TEST_HOST`、`REDIS_TEST_PORT`、`REDIS_TEST_USERNAME`、`REDIS_TEST_PASSWORD`、`REDIS_TEST_DATABASE`、`REDIS_TEST_SSL_ENABLED` 连接独立测试实例,执行 `bash scripts/test-redis.sh`;只使用随机测试 Key 前缀,不执行 FLUSHDB。真实联调验证 TTL、消费竞争、计数、断开后重建连接、会话撤销与账号禁用。
## HTTPS Git 网关
网站身份可通过项目独立凭据 clone、fetch、pull、push main,支持 LFS Basic Transfer;社区仍手动发布。默认关闭,普通用户无需 Gitea 账号。先执行 [Git 凭据建表 SQL](docs/database.md#项目-git-凭据) 和 [写入记录扩展 SQL](docs/admin-database.md#https-git-写入记录扩展),同时部署社区端和管理端兼容版本,再设置 `GIT_ENABLED=true`。应用不自动迁库,本次不修改 `.sql` 文件。
| 环境变量 | 默认值/含义 |
| --- | --- |
| GIT_ENABLED | false |
| GIT_PUBLIC_BASE_URL | 必填的固定公网 HTTPS 地址,例如 `https://git.example.com/git`,不根据 Host 构造 |
| GIT_UPSTREAM_USERNAME / GIT_UPSTREAM_TOKEN | 独立非管理员、非组织所有者技术账号及其 PAT;需要 read:user、write:repository 范围,与仓库管理 GITEA_TOKEN 分离 |
| GIT_REQUEST_TIMEOUT | 10m,覆盖请求及响应流 |
| GIT_MAX_PACK_BYTES / GIT_MAX_LFS_OBJECT_BYTES | 5368709120 / 1073741824 |
| GIT_MAX_METADATA_BYTES / GIT_MAX_BATCH_OBJECTS | 8388608 / 500 |
| GIT_READ_CONCURRENT / GIT_PUSH_CONCURRENT / GIT_LFS_CONCURRENT | 4 / 2 / 8,单实例 |
| GIT_REQUESTS_PER_MINUTE / GIT_LFS_REQUESTS_PER_MINUTE | 120 / 600,每用户 |
| GIT_AUTH_FAILURES_PER_MINUTE | 20,每IP;仅信任已配置代理 |
技术账号不能是仓库管理员;首次签发项目 Token 时后端管理凭据会给它授予该仓库 write 权限。首个读写 Token 初始化 main 保护,已有规则必须兼容;每次 push 重新核验,非快进和删除由 Gitea 接收钩子拒绝。仓库主存储、对象完整性和 LFS 保留在 Gitea。部署必须确保 Git/LFS 原生端点直接响应,无重定向、且上游 LFS action 使用配置的 Gitea origin 和本项目路径;不支持外部对象存储 action 直链。
代理 `/git/` 关闭请求/响应缓冲、缓存,透传 Authorization、Git-Protocol、Content-Type、Accept、Content-Encoding;请求大小允许 pack/LFS 额度,代理与 Gitea 超时至少覆盖10分钟预算,默认读取超时不低于660秒。服务端不得记录认证头或 action Bearer。使用 HTTPS 对外开放;本机真实联调才允许回环 HTTP。保持当前社区端、管理端各单实例部署范围。
前端对接见 [HTTPS Git 与本地开发](docs/vibemake-api.md#https-git-与本地开发),运维恢复与回滚见 [Git 网关运维](docs/admin-deployment.md#https-git-网关运维)。Git Token 继承现有网页登录身份边界:邮箱加图片验证码目前不验证邮箱归属。
验证:`AI_ENABLED=false VIBEMAKE_AI_ENABLED=false OSS_ENABLED=false mvn verify`。`python3 scripts/test-git-gateway.py` 读取 `.env`,在随机测试仓库及临时技术账号上运行真实 Git/Gitea/LFS(含大于50 MiB对象)闭环,测试完成后清理;需要 GITEA_TOKEN 具备创建/删除测试用户权限,以及可连接的 Redis(只使用随机 git-live 前缀)。配置 DB_URL 时还会创建随机 `vibemake_git_verify_*` 临时 MySQL 库,需建库权限;`GIT_GATEWAY_MYSQL=false` 可单独使用 H2 验证 Git/Redis。MySQL 锁测试只创建随机临时库,Redis 测试使用独立随机前缀,均不修改已有业务数据。
+17
View File
@@ -163,6 +163,10 @@ ALTER TABLE project_import_task
```sql
CREATE TABLE repository_write_operation (
operation_code CHAR(32) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL COMMENT '写入操作业务编码,同时作为幂等键',
operation_kind VARCHAR(16) NOT NULL DEFAULT 'WEB_COMMIT' COMMENT '操作类型 WEB_COMMIT网页提交 GIT_PUSH本地推送',
target_commit_sha VARCHAR(64) NULL COMMENT 'Git推送请求的目标提交SHA,网页提交可为空',
actor_user_code VARCHAR(32) NULL COMMENT '实际发起推送的网站用户业务编码',
git_token_code CHAR(32) CHARACTER SET ascii COLLATE ascii_bin NULL COMMENT '本次推送使用的Git凭据业务编码,不保存明文',
project_code CHAR(32) NOT NULL COMMENT '项目业务编码',
repository_id BIGINT NOT NULL COMMENT '操作发起时Gitea仓库内部标识,仅核验使用',
repository_owner VARCHAR(255) NOT NULL COMMENT '操作发起时仓库所属账号',
@@ -260,3 +264,16 @@ ALTER TABLE repository_write_operation
```
旧导入状态首次由新版读取时,将未声明时区的 `reservedAt` 从 UTC 转为上海时间,原子保存并标记 `timeZone=Asia/Shanghai`,后续不会再次转换。应在业务库完成转换后再续跑;未迁移的数据库时间与清单不一致时,现有归属校验会阻止接管项目。回滚应用前需要同时恢复数据库和导入状态备份,并恢复前端的旧时间解析约定。
## HTTPS Git 写入记录扩展
已有 `repository_write_operation` 表手工执行下列 SQL;新库使用上方完整建表语句,不重复执行 ALTER。先升级社区端和管理端兼容代码,再开启 Git 网关。既有记录默认归类为 WEB_COMMIT;Git 推送不使用网页提交尾注核对。
```sql
ALTER TABLE repository_write_operation
ADD COLUMN operation_kind VARCHAR(16) NOT NULL DEFAULT 'WEB_COMMIT' COMMENT '操作类型 WEB_COMMIT网页提交 GIT_PUSH本地推送',
ADD COLUMN target_commit_sha VARCHAR(64) NULL COMMENT 'Git推送请求的目标提交SHA,网页提交可为空',
ADD COLUMN actor_user_code VARCHAR(32) NULL COMMENT '实际发起推送的网站用户业务编码',
ADD COLUMN git_token_code CHAR(32) CHARACTER SET ascii COLLATE ascii_bin NULL COMMENT '本次推送使用的Git凭据业务编码,不保存明文';
```
+20
View File
@@ -203,3 +203,23 @@ Gitea 原始文件有界读取,contents JSON 上限按单文件 Base64 膨胀
项目 SQL 从 122 降到 8,评论列表从 23 降到 4,20 用户后台列表从 22 降到 3。MySQL 四并发 P95 从 2414.935ms 降到 221.998ms,约降低 91%。本地微基准受运行波动影响,不能将 SQL 数变化等同于所有环境的固定延迟收益。堆池峰值是各 heap memory pool 的峰值之和,并非进程 RSS;普通小请求的内存收益不稳定,本期内存保护主要针对超大正文、无界上游响应和大文件并发。
复测:`AI_ENABLED=false OSS_ENABLED=false mvn -pl vibemake-api -am -Dtest=QueryEfficiencyTest -Dsurefire.failIfNoSpecifiedTests=false '-DargLine=-Xms128m -Xmx512m' test`。默认使用 H2;显式设置 BENCHMARK_MYSQL_URL、BENCHMARK_MYSQL_USERNAME、BENCHMARK_MYSQL_PASSWORD 时,自动创建并最终删除 vibemake_benchmark_* 随机临时库,执行 DDL 前验证连接 catalog 必须等于本次随机库。账号需要创建和删除临时库权限,禁止将基准改为直接重置业务库。该小样本不代表正式环境容量,部署仍需结合真实项目体量验证。
## HTTPS Git 网关运维
仅社区 API 暴露 `/git/**`;管理端继续使用后台认证。按 [README 配置](../README.md#https-git-网关) 部署独立 Git 技术账号、HTTPS 地址与无缓冲代理。功能默认关闭;先人工执行 [Git Token 表](database.md#项目-git-凭据) 与 [operation_kind 等字段扩展](admin-database.md#https-git-写入记录扩展),再排空旧进程并升级两个 API,最后对测试项目验证后开放。不能让旧版后台核对器消费 GIT_PUSH 操作。
Git 技术账号须非 site admin、非组织所有者、非仓库管理员,PAT 只给 read:user、write:repository。首次 Token 签发会按项目给技术账号授予 write,不提升其已有 admin/owner 权限;权限不兼容必须先人工处理。已有 main 保护不覆盖,通配/多规则配置会保守拒绝开通;每次 push 复核。运维直接修改仓库引用或 main 保护前应停写,不能绕过网关并发控制。LFS action 以短期 Redis 状态承接,上游地址与认证头只保留服务端。Redis 丢失 action 后需重新请求 batch;原 Git 凭据存 MySQL,可撤销。
网页与本地写入使用同一 ProjectLocks 和 repo_write_status,跨进程由 MySQL 命名锁串行。操作记录必须在转发 receive-pack 前提交,operation_kind=GIT_PUSH 记录 base_commit_sha、target_commit_sha、actor_user_code、git_token_code,摘要不含 pack。HTTP200 不是成功证明;仅接受完整 report-status,成功后再次核对 main,并原子保存终态与 IDLE。推送作者来自本地提交,审计 actor_user_code 是网站发起者,两者可能不同。
后台核对按类型区分:WEB_COMMIT 继续查找原有尾注;GIT_PUSH 先校验项目/远端仓库身份、私有属性及 main,再仅在 main 精确等于 target_commit_sha 时确认成功。main 仍等于 base_commit_sha、为其他SHA、上游不可用或读取失败均保持 REVIEW,不能自动判失败。10分钟后状态为 NEEDS_REVIEW,每分钟继续只读核对,不重发 push。
人工恢复顺序:
1. 关闭 Git 入口、停止并排空社区/后台仓库写入和发布,暂停核对 worker,保留数据库和日志备份。
2. 根据 requestId、projectCode、operationCode、actor_user_code、git_token_code、旧新 SHA 关联网关与 Gitea 接收日志。先确认对应 receive-pack/钩子任务已经结束;单次旧 SHA 查询不能作为结束证明。
3. 核对仓库ID、owner/name、私有、非镜像、main 及对象可达性。精确目标SHA且验证完成可确认 SUCCEEDED;确认任务停止且 main 仍为旧 SHA、未发生引用变更时才可人工标记 FAILED。main 为其他 SHA 或身份变化继续待核对,不改历史、不重推。
4. 在维护事务中同步更新操作 status/result_commit_sha/last_error_code/updated_at 和项目 repo_write_status=IDLE;必须校验项目只有这一条待核对操作,不能解除其他操作的保护。留存审批/排查依据和恢复审计,不能只执行孤立的 IDLE 更新。
5. 重启恢复进程、恢复 worker,验证网页及本地正常写入后再开放 Git。不自动发布社区。
回滚先关闭 Git 功能并排空接收请求,核对处理中操作后再回退应用。旧版本不能消费未解决的 GIT_PUSH;未解决前保持维护停写。保留新增表/列/凭据/审计、仓库内容及 main 保护,不删对象或撤销保护。功能关闭后仍可在网站查询、撤销 Git Token。
+29 -1
View File
@@ -1,6 +1,6 @@
# 数据库初始化
业务数据库名为 `vibemake`,基础业务包含以下四张表;完整部署还需执行 [管理端增量 SQL](admin-database.md),新增迁移任务与仓库写入操作表。由人工执行;应用不运行迁移或自动建表。以下 SQL 面向 MySQL 8.0.13 及以上(URDF JSON 默认值使用表达式)。
业务数据库名为 `vibemake`,基础业务包含以下四张表及项目 Git 凭据表;完整部署还需执行 [管理端增量 SQL](admin-database.md),新增迁移任务与仓库写入操作表。由人工执行;应用不运行迁移或自动建表。以下 SQL 面向 MySQL 8.0.13 及以上(URDF JSON 默认值使用表达式)。
```sql
CREATE DATABASE IF NOT EXISTS vibemake CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
@@ -310,3 +310,31 @@ WHERE LEFT(cover_url, 16) = '/project-covers/'
```
最后查询必须为空,随后验证旧格式地址并回退应用,再恢复写入。旧版不识别相对路径,禁止只回退代码而跳过数据转换。
## 项目 Git 凭据
HTTPS Git 功能默认关闭。已有数据库在启用前手工执行下列建表,以及 [仓库写入记录扩展](admin-database.md#https-git-写入记录扩展)。不自动迁库;回滚保留表和审计记录。所有时间为上海时间。
```sql
CREATE TABLE project_git_token (
id BIGINT NOT NULL AUTO_INCREMENT COMMENT '内部主键,不对用户端暴露',
token_code CHAR(32) CHARACTER SET ascii COLLATE ascii_bin NOT NULL COMMENT 'Git凭据唯一业务编码',
project_code CHAR(32) NOT NULL COMMENT '绑定的项目业务编码',
owner_user_id BIGINT NOT NULL COMMENT '所属用户内部标识,仅内部关联',
token_name VARCHAR(64) NOT NULL COMMENT '用户填写的凭据名称',
token_hash CHAR(64) CHARACTER SET ascii COLLATE ascii_bin NOT NULL COMMENT '凭据明文的SHA256摘要,不保存明文',
permission VARCHAR(16) NOT NULL COMMENT '权限 READ_ONLY只读 READ_WRITE读写',
session_version BIGINT NOT NULL COMMENT '签发时账号会话版本,用于账号变更后失效',
expires_at DATETIME(3) NOT NULL COMMENT '过期时间,上海时间',
revoked_at DATETIME(3) NULL COMMENT '撤销时间,未撤销为空',
last_used_at DATETIME(3) NULL COMMENT '最近成功认证时间,上海时间',
created_at DATETIME(3) NOT NULL COMMENT '创建时间,上海时间',
updated_at DATETIME(3) NOT NULL COMMENT '最近更新时间,上海时间',
PRIMARY KEY (id),
UNIQUE KEY uk_git_token_code (token_code),
UNIQUE KEY uk_git_token_hash (token_hash),
KEY idx_git_token_project_owner (project_code, owner_user_id, created_at)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4
COLLATE=utf8mb4_unicode_ci COMMENT='项目级Git访问凭据';
```
+108
View File
@@ -18,6 +18,7 @@
- [项目 URDF 文件夹](#项目-urdf-文件夹)
- [目录和文件读取](#目录和文件读取)
- [文件编辑与上传提交](#文件编辑与上传提交)
- [HTTPS Git 与本地开发](#https-git-与本地开发)
- [社区发布和撤回](#社区发布和撤回)
- [社区详情和 Fork](#社区详情和-fork)
- [评论和点赞](#评论和点赞)
@@ -128,6 +129,10 @@ Git SHA 使用完整的 40 或 64 位十六进制字符串,不使用缩写。
| GET | `/api/workspace/projects/{projectCode}/tree` | 是 | Tree |
| GET | `/api/workspace/projects/{projectCode}/files` | 是 | ProjectFile 或下载二进制 |
| GET | `/api/workspace/projects/{projectCode}/archive` | 是 | 当前或指定版本的 ZIP、TAR.GZ 仓库归档 |
| GET | `/api/workspace/projects/{projectCode}/git` | 是 | GitConnection |
| GET | `/api/workspace/projects/{projectCode}/git/tokens` | 是 | PageResult<GitTokenView> |
| POST | `/api/workspace/projects/{projectCode}/git/tokens` | 是 | credential 和仅返回一次的 token |
| DELETE | `/api/workspace/projects/{projectCode}/git/tokens/{tokenCode}` | 是 | null |
| POST | `/api/workspace/projects/{projectCode}/commits` | 是 | Commit |
| GET | `/api/workspace/projects/{projectCode}/commits` | 是 | PageResult<HistoryEntry> |
| POST | `/api/workspace/projects/{projectCode}/uploads/commit` | 是 | Commit |
@@ -1625,3 +1630,106 @@ export default {
Fork 每用户每分钟 10 次、URDF 批次签名 20 次、封面签名 30 次,均为独立请求尝试次数;一个 URDF 文件夹的一次批次签名算一次,不按文件数计。Fork 同时执行仍最多 2 个,文件读取/目录/历史/提交/上传合计最多 4 个。普通 JSON 256 KiB、URDF 清单 2 MiB、提交 JSON 64 MiB,原有单文件和总内容约束继续保留。全部规则和可配置变量见 [部署说明](admin-deployment.md#redis-安全与资源保护)。
登录和签名响应显式禁止缓存。CORS 新增暴露 Retry-After,不启用跨域 Cookie;正文、Token、邮箱及验证码答案不得写入前端日志。
## HTTPS Git 与本地开发
Git 网关默认关闭,开启后仅本人工作区可 clone/fetch/pull/push;社区项目先 Fork。网站身份沿用邮箱加图片验证码,当前不验证邮箱归属,Git Token 继承此身份校验边界。普通用户不需要 Gitea 账号。
### 接口清单与数据
以下接口均携带网站 `Authorization: Bearer <平台 Token>`,仅所有者可访问。所有成功响应仍为平台 ApiResponse,均设置 `Cache-Control: no-store`;Git 协议请求使用独立认证。
| 接口 | 作用 |
| --- | --- |
| `GET /api/workspace/projects/{projectCode}/git` | 获取克隆信息及写入状态 |
| `GET /api/workspace/projects/{projectCode}/git/tokens?page=1&size=20` | 分页查询 Git 凭据元数据 |
| `POST /api/workspace/projects/{projectCode}/git/tokens` | 创建凭据,明文只返回一次 |
| `DELETE /api/workspace/projects/{projectCode}/git/tokens/{tokenCode}` | 撤销凭据,重复撤销成功 |
克隆信息 data:
```json
{
"cloneUrl": "https://git.example.com/git/projects/7m4k9x2p8q6r.git",
"username": "111111111111",
"defaultBranch": "main",
"headCommitSha": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
"publishedCommitSha": null,
"writeStatus": "IDLE",
"canPush": true
}
```
`canPush` 表示当前项目状态、main 保护及技术账号配置允许写入,不保证后续请求一定成功;仍需读写权限的 Git Token。`writeStatus=WRITING` 表示正在写入,`REVIEW` 表示结果待核对。仅刷新页面即可看到已推送内容,不使用项目 updatedAt 判断版本。
创建请求:
```json
{"name":"我的笔记本","scope":"READ_WRITE","expiresInDays":90}
```
name 必填、非空、最多64字符;scope 为 READ_ONLY 或 READ_WRITE;expiresInDays 省略默认90,允许1~365。每项目最多10个有效凭据。读写凭据创建前会核验或初始化 main 保护;配置不兼容返回错误,不覆盖已有规则。首次签发会由后端管理账号给独立技术账号授予该仓库 write 权限,用户侧权限仍由项目 Token 决定。
创建成功 data 为 `{ "credential": GitTokenView, "token": "vbg_..." }`;列表 data 为 `PageResult<GitTokenView>`。GitTokenView 只有 tokenCode、name、scope、status、createdAt、expiresAt、revokedAt、lastUsedAt。status 为 ACTIVE、EXPIRED、REVOKED(包括账号会话版本变化失效)。时间沿用本文上海时间约定。最近使用时间最多每5分钟写回一次。
明文仅在创建时展示一次,不能再次查询;遗失后撤销并新建。列表、日志、数据库均不保存或返回明文。普通退出网站不撤销 Git Token;撤销、过期、禁用账号或账号 sessionVersion 改变后,新 Git/LFS 请求会被拒绝。已经受理的传输不保证立即中止或回滚。
### 网站交互与本地使用
工作区提供“克隆与本地开发”入口,显示克隆 URL、用户名、main SHA、发布 SHA 和写入状态;提供 Token 创建、列表、撤销和一次性明文展示。明文只保存在当前展示状态,不写 localStorage、不放克隆 URL、查询参数或日志。用户可用操作系统凭据管理器保存。
Git 使用 HTTP Basic:用户名为网站 userCode,密码为项目 Git Token;网站 Bearer Token 不能用于 Git,Git Token 不能用于普通 `/api`。同域名不同项目必须配置 HTTP 路径级凭据:
```bash
git lfs install
git -c credential.useHttpPath=true clone --single-branch --branch main <cloneUrl>
cd <本地仓库目录>
git config credential.useHttpPath true
git add .
git commit -m "更新项目"
git push origin main
```
支持标准 Git v0/v2 读取、main 正常快进更新和合并提交,保留本地提交作者、邮箱、时间、签名及 SHA。禁止强推、删除 main、创建其他分支或标签、多引用推送、push-options、signed-push 扩展;普通签名提交支持。仓库已有其他分支和标签可以读取,本期不清理历史数据。LFS Basic Transfer 支持实体上传、下载和 verify;不支持 LFS 文件锁,locks/verify 返回空集合,锁创建/删除返回501。不递归下载子模块,独立绑定的 OSS 封面、URDF、STEP、模型等资源不会自动进入仓库。
本地 push 后重新请求工作区 tree、files、commits 显示最新内容;社区仍固定 publishedCommitSha,需用户明确点击发布。网页编辑基准落后继续返回409,保留用户输入;本地 main 落后时先 pull、合并再 push。REVIEW 期间阻止仓库写入及发布,后台只读核对,不自动重推。网络断开或超时不能视为失败,先刷新核对。
前端创建示例(复用本文 request):
```ts
interface GitTokenView {
tokenCode: string;
name: string;
scope: 'READ_ONLY' | 'READ_WRITE';
status: 'ACTIVE' | 'EXPIRED' | 'REVOKED';
createdAt: string;
expiresAt: string;
revokedAt: string | null;
lastUsedAt: string | null;
}
async function createGitToken(projectCode: string, token: string) {
return request<{credential: GitTokenView; token: string}>(
`/workspace/projects/${encodeURIComponent(projectCode)}/git/tokens`, {
method: 'POST', token,
body: {name: '我的笔记本', scope: 'READ_WRITE', expiresInDays: 90},
});
}
```
协议路径 `/git/**` 成功响应为 Git pkt-line/pack 或 LFS 标准结构;错误使用 Git 文本或 LFS `{message}`,不使用平台包装。401 含 Basic challenge,429 含 Retry-After。不要用浏览器平台 JSON 请求封装访问协议路径。
| 错误码/状态 | 处理 |
| --- | --- |
| GIT_UNAVAILABLE / 503 | 功能关闭或依赖不可用,保留网站编辑入口;仍可查询、撤销已有凭据 |
| GIT_POLICY_UNAVAILABLE / 503 | 技术账号或 main 保护不兼容,联系运维,不放宽规则 |
| CONFLICT / 409 | 有效凭据已达上限,先撤销旧凭据 |
| GIT_UNAUTHORIZED / 401 | 用户名或项目 Git Token 无效、过期或撤销 |
| GIT_READ_ONLY / 403 | 使用读写 Token 后再推送或上传 LFS |
| GIT_COMMAND_REJECTED / 400 | 仅推送 main,移除不支持的引用/扩展 |
| GIT_STALE_MAIN / 409 | 先拉取、合并,再推送 |
| GIT_BUSY / 429 | 用户限流、IP认证失败限流或并发已满,稍后手动重试 |
| GIT_TRANSFER_TOO_LARGE、GIT_METADATA_TOO_LARGE、LFS_TOO_LARGE / 413 | 缩小请求或联系运维核对配置 |
| GIT_TIMEOUT / 504、GIT_UPSTREAM_ERROR / 502、REMOTE_RESULT_UNKNOWN / 409或503 | 涉及 push 时先刷新工作区及历史,结果未确认前不能重复写入 |
默认 pack 5 GiB、LFS 单对象1 GiB、元数据8 MiB、batch最多500个对象;每次 Git/LFS 请求预算10分钟,含请求及响应流。读取/push/LFS 每实例并发4/2/8;每用户每分钟 Git120次、LFS600次,每IP每分钟最多20次认证失败。均可配置,独立于 JSON、multipart、在线50 MiB文件读取和归档下载限制。中途超限、超时或断开会终止传输;不提供任务、进度查询或断点续传接口。
+26
View File
@@ -0,0 +1,26 @@
#!/usr/bin/env python3
"""使用随机测试用户/仓库验证真实 HTTPS Git 接入;不修改已有项目。"""
import os
from pathlib import Path
import shlex
import subprocess
root = Path(__file__).resolve().parent.parent
environment = os.environ.copy()
config = root / '.env'
if config.exists():
for line in config.read_text().splitlines():
if not line or line.startswith('#') or '=' not in line:
continue
key, value = line.split('=', 1)
parsed = shlex.split(value)
if parsed:
environment[key] = parsed[0]
if not environment.get('GITEA_TOKEN'):
raise SystemExit('请配置 GITEA_BASE_URL、GITEA_ORGANIZATION 与可创建测试用户的 GITEA_TOKEN')
if environment.get('DB_URL', '').startswith('jdbc:mysql:'):
environment.setdefault('GIT_GATEWAY_MYSQL', 'true')
environment.update(VIBEMAKE_LIVE_GITEA='true', AI_ENABLED='false', VIBEMAKE_AI_ENABLED='false', OSS_ENABLED='false')
raise SystemExit(subprocess.call([
'mvn', '-q', '-Dtest=GitGatewayLiveTest', '-Dsurefire.failIfNoSpecifiedTests=false', 'test'
], cwd=root, env=environment))
@@ -126,6 +126,7 @@ class AdminWorkflowTest {
"JSON NOT NULL DEFAULT (JSON_ARRAY())",
"CLOB NOT NULL DEFAULT '[]'")
.replace(" CHARACTER SET utf8mb4 COLLATE utf8mb4_bin", "")
.replace(" CHARACTER SET ascii COLLATE ascii_bin", "")
.replaceAll(" COMMENT '[^']*'", "")
.replaceAll("\\) ENGINE=InnoDB[^;]*;", ");"));
}
@@ -0,0 +1,74 @@
package com.linkerbot.vibemake.api.controller;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.linkerbot.vibemake.common.BusinessException;
import com.linkerbot.vibemake.security.GitAuthenticationFilter;
import com.linkerbot.vibemake.service.git.*;
import jakarta.servlet.http.*;
import lombok.RequiredArgsConstructor;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.web.bind.annotation.*;
import java.io.IOException;
@io.swagger.v3.oas.annotations.Hidden
@RestController
@RequiredArgsConstructor
public class GitProtocolController {
private static final Logger log = LoggerFactory.getLogger(GitProtocolController.class);
private final GitGateway git;
private final LfsGateway lfs;
private final ObjectMapper json;
@RequestMapping("/git/projects/{projectCode}.git/{*endpoint}")
public void protocol(
@PathVariable String projectCode,
@PathVariable String endpoint,
HttpServletRequest request,
HttpServletResponse response)
throws IOException {
GitAccess access = (GitAccess) request.getAttribute(GitAuthenticationFilter.ACCESS);
try {
if (access == null || !projectCode.equals(access.projectCode()))
throw GitTokens.unauthorized();
String path = endpoint.startsWith("/") ? endpoint.substring(1) : endpoint;
if (path.startsWith("info/lfs/actions/")) {
LfsGateway.Action action =
(LfsGateway.Action) request.getAttribute(GitAuthenticationFilter.ACTION);
if (action == null) throw GitTokens.unauthorized();
lfs.action(action, request, response);
} else if (path.startsWith("info/lfs/"))
lfs.serve(access, path.substring(9), request, response);
else git.serve(access, path, request, response);
} catch (BusinessException error) {
log.warn(
"Git 协议请求拒绝 projectCode={} code={} status={}",
projectCode,
error.code(),
error.status());
fail(request, response, error);
} catch (Exception error) {
log.warn(
"Git 协议请求中断 projectCode={} errorType={}",
projectCode,
error.getClass().getSimpleName());
fail(
request,
response,
new BusinessException(502, "GIT_UPSTREAM_ERROR", "Git 传输中断,请刷新工作区核对结果"));
}
}
private void fail(
HttpServletRequest request, HttpServletResponse response, BusinessException error) {
try {
GitAuthenticationFilter.failure(request, response, json, error);
} catch (IOException ignored) {
/* 已断开连接不再进入平台 JSON 异常处理器。 */
}
}
}
@@ -0,0 +1,55 @@
package com.linkerbot.vibemake.api.controller;
import com.linkerbot.vibemake.bean.GitContracts.*;
import com.linkerbot.vibemake.common.*;
import com.linkerbot.vibemake.security.CurrentUser;
import com.linkerbot.vibemake.service.git.*;
import jakarta.servlet.http.HttpServletResponse;
import jakarta.validation.Valid;
import lombok.RequiredArgsConstructor;
import org.springframework.web.bind.annotation.*;
@RestController
@RequiredArgsConstructor
@RequestMapping("/api/workspace/projects/{projectCode}/git")
public class ProjectGitController {
private final GitGateway gateway;
private final GitTokens tokens;
@ModelAttribute
public void noStore(HttpServletResponse response) {
response.setHeader("Cache-Control", "no-store");
}
@GetMapping
public ApiResponse<ConnectionView> connection(@PathVariable String projectCode) {
return ApiResponse.success(gateway.connection(CurrentUser.required(), projectCode));
}
@GetMapping("/tokens")
public ApiResponse<PageResponse<TokenView>> list(
@PathVariable String projectCode,
@RequestParam(defaultValue = "1") long page,
@RequestParam(defaultValue = "20") long size) {
return ApiResponse.success(tokens.list(CurrentUser.required(), projectCode, page, size));
}
@PostMapping("/tokens")
public ApiResponse<IssuedToken> issue(
@PathVariable String projectCode,
@Valid @RequestBody TokenInput input,
HttpServletResponse response) {
response.setHeader("Cache-Control", "no-store");
return ApiResponse.success(tokens.issue(CurrentUser.required(), projectCode, input));
}
@DeleteMapping("/tokens/{tokenCode}")
public ApiResponse<Void> revoke(
@PathVariable String projectCode, @PathVariable String tokenCode) {
tokens.revoke(CurrentUser.required(), projectCode, tokenCode);
return ApiResponse.success(null);
}
}
@@ -98,6 +98,22 @@ vibemake:
gitea:
connect-timeout: 5s
request-timeout: 30s
git:
enabled: ${GIT_ENABLED:true}
public-base-url: ${GIT_PUBLIC_BASE_URL:}
upstream-username: ${GIT_UPSTREAM_USERNAME:}
upstream-token: ${GIT_UPSTREAM_TOKEN:}
timeout: ${GIT_REQUEST_TIMEOUT:10m}
max-pack-bytes: ${GIT_MAX_PACK_BYTES:5368709120}
max-lfs-object-bytes: ${GIT_MAX_LFS_OBJECT_BYTES:1073741824}
max-metadata-bytes: ${GIT_MAX_METADATA_BYTES:8388608}
max-batch-objects: ${GIT_MAX_BATCH_OBJECTS:500}
read-concurrent: ${GIT_READ_CONCURRENT:4}
push-concurrent: ${GIT_PUSH_CONCURRENT:2}
lfs-concurrent: ${GIT_LFS_CONCURRENT:8}
git-requests-per-minute: ${GIT_REQUESTS_PER_MINUTE:120}
lfs-requests-per-minute: ${GIT_LFS_REQUESTS_PER_MINUTE:600}
auth-failures-per-minute: ${GIT_AUTH_FAILURES_PER_MINUTE:20}
archive:
max-bytes: ${ARCHIVE_MAX_BYTES:1073741824}
timeout: ${ARCHIVE_TIMEOUT:5m}
@@ -0,0 +1,728 @@
package com.linkerbot.vibemake.api;
import static org.assertj.core.api.Assertions.*;
import static org.mockito.ArgumentMatchers.*;
import static org.mockito.Mockito.*;
import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.*;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.*;
import com.fasterxml.jackson.databind.*;
import com.linkerbot.vibemake.common.SecurityHashes;
import com.linkerbot.vibemake.service.git.*;
import com.linkerbot.vibemake.service.write.RepositoryWrites;
import com.sun.net.httpserver.HttpServer;
import org.junit.jupiter.api.*;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.autoconfigure.web.servlet.*;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.context.annotation.Import;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.test.context.*;
import org.springframework.test.context.bean.override.mockito.MockitoBean;
import org.springframework.test.context.bean.override.mockito.MockitoSpyBean;
import org.springframework.test.web.servlet.*;
import org.springframework.test.web.servlet.request.MockHttpServletRequestBuilder;
import java.io.*;
import java.net.*;
import java.nio.charset.StandardCharsets;
import java.util.*;
import java.util.concurrent.atomic.AtomicReference;
@SpringBootTest(
properties = {
"spring.datasource.url=jdbc:h2:mem:gitapi;MODE=MySQL;DB_CLOSE_DELAY=-1",
"spring.datasource.username=sa",
"spring.datasource.password=",
"spring.datasource.driver-class-name=org.h2.Driver",
"vibemake.ai.enabled=false",
"vibemake.oss.enabled=false",
"vibemake.limits.enabled=false",
"vibemake.gitea.owner=vibemake",
"vibemake.repository-write.worker-enabled=false",
"vibemake.git.enabled=true",
"vibemake.git.public-base-url=https://git.example/git",
"vibemake.git.upstream-username=fixture-bot",
"vibemake.git.upstream-token=fixture-only-secret",
"vibemake.git.auth-failures-per-minute=1000",
"logging.level.root=WARN"
})
@ActiveProfiles("test")
@AutoConfigureMockMvc(print = MockMvcPrint.NONE)
@Import(WorkflowTest.Config.class)
class GitApiTest {
static HttpServer server;
static AtomicReference<String> mode = new AtomicReference<>("ok");
static AtomicReference<byte[]> received = new AtomicReference<>();
static java.util.concurrent.CountDownLatch pushEntered;
static java.util.concurrent.CountDownLatch pushRelease;
static String upstream;
static final byte[] OBJECT = "LFS binary 中文\0fixture".getBytes(StandardCharsets.UTF_8);
static final String OID = java.util.HexFormat.of().formatHex(hash(OBJECT));
private static byte[] hash(byte[] bytes) {
try {
return java.security.MessageDigest.getInstance("SHA-256").digest(bytes);
} catch (Exception error) {
throw new IllegalStateException(error);
}
}
@DynamicPropertySource
static void upstream(DynamicPropertyRegistry registry) throws Exception {
server = HttpServer.create(new InetSocketAddress("127.0.0.1", 0), 0);
upstream = "http://127.0.0.1:" + server.getAddress().getPort();
server.createContext(
"/",
exchange -> {
String path = exchange.getRequestURI().getPath();
byte[] result;
String type = "application/x-git-upload-pack-advertisement";
if (path.endsWith("objects/batch")) {
type = LfsGateway.MEDIA;
JsonNode input = new ObjectMapper().readTree(exchange.getRequestBody());
String op = input.path("operation").asText();
String prefix = path.substring(0, path.length() - "objects/batch".length());
String href =
upstream
+ prefix
+ "objects/"
+ OID
+ (op.equals("upload") ? "/" + OBJECT.length : "");
if (mode.get().equals("evil")) href = "https://untrusted.invalid/" + OID;
result =
("{\"transfer\":\"basic\",\"objects\":[{\"oid\":\""
+ OID
+ "\",\"size\":"
+ OBJECT.length
+ ",\"actions\":{\""
+ op
+ "\":{\"href\":\""
+ href
+ "\",\"header\":{\"Authorization\":\"Bearer"
+ " upstream-secret\"}}}}]}")
.getBytes();
} else if (path.contains("/info/lfs/objects/")) {
type = "application/octet-stream";
if (exchange.getRequestMethod().equals("PUT")) {
received.set(exchange.getRequestBody().readAllBytes());
result = new byte[0];
} else result = OBJECT;
} else if (path.endsWith("git-receive-pack")) {
received.set(exchange.getRequestBody().readAllBytes());
if (mode.get().equals("block")) {
pushEntered.countDown();
try {
pushRelease.await(15, java.util.concurrent.TimeUnit.SECONDS);
} catch (InterruptedException error) {
Thread.currentThread().interrupt();
}
}
type = "application/x-git-receive-pack-result";
result =
mode.get().equals("broken")
? "not status".getBytes()
: packets(
"unpack ok\n",
"ng refs/heads/main non-fast-forward\n",
null);
} else
result =
packets(
"# service=git-upload-pack\n",
null,
"a".repeat(40) + " refs/heads/main\0multi_ack\n",
null);
exchange.getResponseHeaders().set("Content-Type", type);
exchange.sendResponseHeaders(200, result.length == 0 ? -1 : result.length);
exchange.getResponseBody().write(result);
exchange.close();
});
server.start();
registry.add("vibemake.gitea.base-url", () -> upstream);
}
@AfterAll
static void closeUpstream() {
server.stop(0);
}
@Autowired MockMvc mvc;
@Autowired ObjectMapper json;
@Autowired JdbcTemplate jdbc;
@MockitoSpyBean FakeGiteaClient gitea;
@Autowired RepositoryWrites writes;
@Autowired com.linkerbot.vibemake.common.state.TransientStateStore state;
@MockitoBean GitPolicy policy;
@BeforeEach
void schema() throws Exception {
SchemaSupport.reset(jdbc);
reset(gitea);
gitea.reset();
mode.set("ok");
received.set(null);
pushEntered = new java.util.concurrent.CountDownLatch(1);
pushRelease = new java.util.concurrent.CountDownLatch(1);
}
static byte[] packets(String... lines) throws IOException {
ByteArrayOutputStream bytes = new ByteArrayOutputStream();
for (String line : lines)
if (line == null) bytes.write("0000".getBytes());
else GitIo.packet(bytes, line.getBytes(StandardCharsets.UTF_8));
return bytes.toByteArray();
}
JsonNode ok(MockHttpServletRequestBuilder request) throws Exception {
return json.readTree(
mvc.perform(request)
.andExpect(status().isOk())
.andReturn()
.getResponse()
.getContentAsByteArray())
.path("data");
}
MockHttpServletRequestBuilder body(MockHttpServletRequestBuilder request, Object value)
throws Exception {
return request.contentType("application/json").content(json.writeValueAsBytes(value));
}
MockHttpServletRequestBuilder auth(MockHttpServletRequestBuilder request, String token) {
return request.header("Authorization", "Bearer " + token);
}
record Account(String token, String username) {}
Account login(String suffix) throws Exception {
String captcha = ok(get("/api/auth/captcha")).path("captchaCode").asText();
JsonNode response =
ok(
body(
post("/api/auth/login"),
Map.of(
"email",
suffix + "@example.invalid",
"captchaCode",
captcha,
"captchaAnswer",
"ABCDE")));
return new Account(
response.path("token").asText(), response.path("user").path("userCode").asText());
}
String create(Account account) throws Exception {
return ok(body(
auth(post("/api/workspace/projects"), account.token()),
Map.of("name", "Git 项目", "projectType", "GENERAL")))
.path("projectCode")
.asText();
}
JsonNode issue(Account account, String code, String scope) throws Exception {
return ok(
body(
auth(post(api(code) + "/tokens"), account.token()),
Map.of("name", "笔记本", "scope", scope)));
}
String api(String code) {
return "/api/workspace/projects/" + code + "/git";
}
String protocol(String code) {
return "/git/projects/" + code + ".git";
}
MockHttpServletRequestBuilder basic(
MockHttpServletRequestBuilder request, Account account, String secret) {
return request.header(
"Authorization",
"Basic "
+ Base64.getEncoder()
.encodeToString(
(account.username() + ":" + secret)
.getBytes(StandardCharsets.UTF_8)));
}
@Test
void credentialsAreProjectBoundHashedAndSeparateFromWebsiteSessions() throws Exception {
Account owner = login("git-owner"), other = login("git-other");
String code = create(owner), second = create(owner);
JsonNode issued = issue(owner, code, "READ_ONLY");
String secret = issued.path("token").asText();
assertThat(secret).startsWith("vbg_");
assertThat(jdbc.queryForObject("SELECT token_hash FROM project_git_token", String.class))
.isEqualTo(SecurityHashes.sha256(secret));
String list = ok(auth(get(api(code) + "/tokens"), owner.token())).toString();
assertThat(list).doesNotContain(secret, "tokenHash", "ownerUserId", "\"id\"");
mvc.perform(
basic(
get(protocol(code) + "/info/refs")
.param("service", "git-upload-pack"),
owner,
secret))
.andExpect(status().isOk());
mvc.perform(
basic(
get(protocol(second) + "/info/refs")
.param("service", "git-upload-pack"),
owner,
secret))
.andExpect(status().isNotFound());
mvc.perform(
basic(
get(protocol(code) + "/info/refs")
.param("service", "git-upload-pack"),
other,
secret))
.andExpect(status().isUnauthorized());
mvc.perform(auth(get(api(code)), secret)).andExpect(status().isUnauthorized());
mvc.perform(
auth(
get(protocol(code) + "/info/refs")
.param("service", "git-upload-pack"),
owner.token()))
.andExpect(status().isUnauthorized())
.andExpect(
header().string(
"WWW-Authenticate",
org.hamcrest.Matchers.containsString("Basic")));
mvc.perform(
basic(
get(protocol(code) + "/info/refs")
.param("service", "git-receive-pack"),
owner,
secret))
.andExpect(status().isForbidden());
mvc.perform(auth(get(api(code) + "/tokens"), other.token()))
.andExpect(status().isNotFound());
// 普通退出只清网页会话。
ok(auth(post("/api/auth/logout"), owner.token()));
mvc.perform(
basic(
get(protocol(code) + "/info/refs")
.param("service", "git-upload-pack"),
owner,
secret))
.andExpect(status().isOk());
jdbc.update(
"UPDATE sys_user SET session_version=session_version+1 WHERE user_code=?",
owner.username());
mvc.perform(
basic(
get(protocol(code) + "/info/refs")
.param("service", "git-upload-pack"),
owner,
secret))
.andExpect(status().isUnauthorized());
}
@Test
void expirationRevocationAndTokenQuota() throws Exception {
Account owner = login("git-token-lifecycle");
String code = create(owner);
JsonNode first = issue(owner, code, "READ_WRITE");
String tokenCode = first.path("credential").path("tokenCode").asText();
for (int index = 1; index < 10; index++) issue(owner, code, "READ_ONLY");
mvc.perform(
body(
auth(post(api(code) + "/tokens"), owner.token()),
Map.of("name", "超过", "scope", "READ_ONLY")))
.andExpect(status().isConflict());
ok(auth(delete(api(code) + "/tokens/" + tokenCode), owner.token()));
ok(auth(delete(api(code) + "/tokens/" + tokenCode), owner.token()));
mvc.perform(
basic(
get(protocol(code) + "/info/refs")
.param("service", "git-upload-pack"),
owner,
first.path("token").asText()))
.andExpect(status().isUnauthorized());
String replacement = issue(owner, code, "READ_ONLY").path("token").asText();
jdbc.update("UPDATE project_git_token SET expires_at=DATEADD('DAY',-1,CURRENT_TIMESTAMP)");
mvc.perform(
basic(
get(protocol(code) + "/info/refs")
.param("service", "git-upload-pack"),
owner,
replacement))
.andExpect(status().isUnauthorized());
for (Map<String, Object> invalid :
List.of(
Map.<String, Object>of("name", " ", "scope", "READ_ONLY"),
Map.<String, Object>of(
"name", "x", "scope", "READ_ONLY", "expiresInDays", 366)))
mvc.perform(body(auth(post(api(code) + "/tokens"), owner.token()), invalid))
.andExpect(status().isBadRequest());
}
@Test
void lfsActionsHideUpstreamBindProjectAndRecheckRevocation() throws Exception {
Account owner = login("git-lfs");
String code = create(owner);
JsonNode issued = issue(owner, code, "READ_WRITE");
String secret = issued.path("token").asText();
Map<String, Object> batch =
Map.of(
"operation",
"upload",
"objects",
List.of(Map.of("oid", OID, "size", OBJECT.length)));
MvcResult result =
mvc.perform(
basic(
post(protocol(code) + "/info/lfs/objects/batch"),
owner,
secret)
.contentType(LfsGateway.MEDIA)
.content(json.writeValueAsBytes(batch)))
.andExpect(status().isOk())
.andReturn();
String raw = result.getResponse().getContentAsString();
assertThat(raw).doesNotContain(upstream, "upstream-secret", "fixture-only-secret");
JsonNode action = json.readTree(raw).path("objects").get(0).path("actions").path("upload");
String path = URI.create(action.path("href").asText()).getPath();
String auth = action.path("header").path("Authorization").asText();
mvc.perform(
put(path)
.header("Authorization", auth)
.contentType("application/octet-stream")
.content(OBJECT))
.andExpect(status().isOk());
assertThat(received.get()).isEqualTo(OBJECT);
String readonly = issue(owner, code, "READ_ONLY").path("token").asText();
mvc.perform(
basic(post(protocol(code) + "/info/lfs/objects/batch"), owner, readonly)
.contentType(LfsGateway.MEDIA)
.content(json.writeValueAsBytes(batch)))
.andExpect(status().isForbidden());
mode.set("evil");
mvc.perform(
basic(post(protocol(code) + "/info/lfs/objects/batch"), owner, secret)
.contentType(LfsGateway.MEDIA)
.content(json.writeValueAsBytes(batch)))
.andExpect(status().isBadGateway());
ok(
auth(
delete(
api(code)
+ "/tokens/"
+ issued.path("credential").path("tokenCode").asText()),
owner.token()));
mvc.perform(put(path).header("Authorization", auth).content(OBJECT))
.andExpect(status().isUnauthorized());
}
@Test
void explicitGitRejectionUnlocksButUnknownStatusRequiresReview() throws Exception {
Account owner = login("git-push");
String code = create(owner);
String secret = issue(owner, code, "READ_WRITE").path("token").asText();
String old =
ok(auth(get("/api/workspace/projects/" + code + "/tree"), owner.token()))
.path("commitSha")
.asText();
byte[] body =
packets(
old + " " + "b".repeat(40) + " refs/heads/main\0report-status ofs-delta",
null);
mvc.perform(
basic(post(protocol(code) + "/git-receive-pack"), owner, secret)
.contentType("application/x-git-receive-pack-request")
.content(body))
.andExpect(status().isOk());
assertThat(
jdbc.queryForObject(
"SELECT status FROM repository_write_operation", String.class))
.isEqualTo("FAILED");
assertThat(jdbc.queryForObject("SELECT repo_write_status FROM project", String.class))
.isEqualTo("IDLE");
mode.set("broken");
mvc.perform(
basic(post(protocol(code) + "/git-receive-pack"), owner, secret)
.contentType("application/x-git-receive-pack-request")
.content(body))
.andExpect(status().isBadGateway());
assertThat(jdbc.queryForObject("SELECT repo_write_status FROM project", String.class))
.isEqualTo("REVIEW");
jdbc.update(
"UPDATE repository_write_operation SET"
+ " next_check_at=DATEADD('SECOND',-1,CURRENT_TIMESTAMP),created_at=DATEADD('MINUTE',-11,CURRENT_TIMESTAMP)"
+ " WHERE status='PROCESSING'");
writes.reconcileDue();
assertThat(
jdbc.queryForObject(
"SELECT status FROM repository_write_operation WHERE"
+ " status<>'FAILED'",
String.class))
.isEqualTo("NEEDS_REVIEW");
assertThat(jdbc.queryForObject("SELECT repo_write_status FROM project", String.class))
.isEqualTo("REVIEW");
doReturn("b".repeat(40)).when(gitea).head(any());
jdbc.update(
"UPDATE repository_write_operation SET"
+ " next_check_at=DATEADD('SECOND',-1,CURRENT_TIMESTAMP) WHERE"
+ " status='NEEDS_REVIEW'");
writes.reconcileDue();
assertThat(
jdbc.queryForObject(
"SELECT status FROM repository_write_operation WHERE"
+ " status<>'FAILED'",
String.class))
.isEqualTo("SUCCEEDED");
assertThat(jdbc.queryForObject("SELECT repo_write_status FROM project", String.class))
.isEqualTo("IDLE");
}
@Test
void nativePushHoldsSameLockAsWebsiteWritesAndPersistsBeforeForwarding() throws Exception {
Account owner = login("git-shared-lock");
String code = create(owner);
String secret = issue(owner, code, "READ_WRITE").path("token").asText();
String old =
ok(auth(get("/api/workspace/projects/" + code + "/tree"), owner.token()))
.path("commitSha")
.asText();
byte[] payload =
packets(
old + " " + "b".repeat(40) + " refs/heads/main\0 report-status ofs-delta",
null);
mode.set("block");
try (java.util.concurrent.ExecutorService executor =
java.util.concurrent.Executors.newSingleThreadExecutor()) {
java.util.concurrent.Future<MvcResult> pushing =
executor.submit(
() ->
mvc.perform(
basic(
post(
protocol(code)
+ "/git-receive-pack"),
owner,
secret)
.contentType(
"application/x-git-receive-pack-request")
.content(payload))
.andReturn());
try {
assertThat(pushEntered.await(5, java.util.concurrent.TimeUnit.SECONDS)).isTrue();
assertThat(
jdbc.queryForObject(
"SELECT repo_write_status FROM project", String.class))
.isEqualTo("WRITING");
assertThat(
jdbc.queryForObject(
"SELECT operation_kind FROM repository_write_operation",
String.class))
.isEqualTo("GIT_PUSH");
ok(auth(get("/api/workspace/projects/" + code + "/tree"), owner.token()));
mvc.perform(
body(
auth(
post(
"/api/workspace/projects/"
+ code
+ "/commits"),
owner.token()),
Map.of(
"expectedCommitSha",
old,
"message",
"竞争网页提交",
"changes",
List.of(
Map.of(
"action",
"UPDATE_FILE",
"path",
"README.md",
"content",
"保留输入")))))
.andExpect(status().isConflict());
assertThat(
jdbc.queryForObject(
"SELECT COUNT(*) FROM repository_write_operation",
Integer.class))
.isEqualTo(1);
} finally {
pushRelease.countDown();
}
assertThat(
pushing.get(5, java.util.concurrent.TimeUnit.SECONDS)
.getResponse()
.getStatus())
.isEqualTo(200);
assertThat(jdbc.queryForObject("SELECT repo_write_status FROM project", String.class))
.isEqualTo("IDLE");
}
}
@Test
void acceptsGzipAndFlushProbeButRejectsDisabledDeletedAndNonMainUpdates() throws Exception {
Account owner = login("git-wire");
String code = create(owner);
String secret = issue(owner, code, "READ_WRITE").path("token").asText();
mvc.perform(
basic(post(protocol(code) + "/git-receive-pack"), owner, secret)
.contentType("application/x-git-receive-pack-request")
.content("0000"))
.andExpect(status().isOk());
assertThat(
jdbc.queryForObject(
"SELECT COUNT(*) FROM repository_write_operation", Integer.class))
.isZero();
String old =
ok(auth(get("/api/workspace/projects/" + code + "/tree"), owner.token()))
.path("commitSha")
.asText();
byte[] payload =
packets(
old + " " + "b".repeat(40) + " refs/heads/main\0 report-status ofs-delta",
null);
ByteArrayOutputStream compressed = new ByteArrayOutputStream();
try (java.util.zip.GZIPOutputStream gzip = new java.util.zip.GZIPOutputStream(compressed)) {
gzip.write(payload);
}
mvc.perform(
basic(post(protocol(code) + "/git-receive-pack"), owner, secret)
.contentType("application/x-git-receive-pack-request")
.header("Content-Encoding", "gzip")
.content(compressed.toByteArray()))
.andExpect(status().isOk());
assertThat(received.get()).isEqualTo(payload);
jdbc.update("UPDATE sys_user SET status='DISABLED' WHERE user_code=?", owner.username());
assertThat(
mvc.perform(
basic(
get(protocol(code) + "/info/refs")
.param("service", "git-upload-pack"),
owner,
secret))
.andReturn()
.getResponse()
.getStatus())
.isIn(401, 403);
jdbc.update("UPDATE sys_user SET status='ACTIVE' WHERE user_code=?", owner.username());
jdbc.update("UPDATE project SET repo_status='CREATING'");
mvc.perform(
basic(
get(protocol(code) + "/info/refs")
.param("service", "git-upload-pack"),
owner,
secret))
.andExpect(status().isConflict());
jdbc.update("UPDATE project SET repo_status='READY',deleted=1");
mvc.perform(
basic(
get(protocol(code) + "/info/refs")
.param("service", "git-upload-pack"),
owner,
secret))
.andExpect(status().isNotFound());
}
@Test
void lfsValidatesDownloadActionsIntegrityExpiryAndMetadataLimits() throws Exception {
Account owner = login("git-lfs-boundaries");
String code = create(owner), other = create(owner);
String rw = issue(owner, code, "READ_WRITE").path("token").asText();
String ro = issue(owner, code, "READ_ONLY").path("token").asText();
byte[] batch =
json.writeValueAsBytes(
Map.of(
"operation",
"download",
"objects",
List.of(Map.of("oid", OID, "size", OBJECT.length))));
MvcResult reply =
mvc.perform(
basic(post(protocol(code) + "/info/lfs/objects/batch"), owner, ro)
.contentType(LfsGateway.MEDIA)
.content(batch))
.andExpect(status().isOk())
.andReturn();
JsonNode action =
json.readTree(reply.getResponse().getContentAsByteArray())
.path("objects")
.get(0)
.path("actions")
.path("download");
String path = URI.create(action.path("href").asText()).getPath(),
authorization = action.path("header").path("Authorization").asText();
mvc.perform(get(path).header("Authorization", authorization))
.andExpect(status().isOk())
.andExpect(content().bytes(OBJECT));
mvc.perform(get(path.replace(code, other)).header("Authorization", authorization))
.andExpect(status().isUnauthorized());
mvc.perform(put(path).header("Authorization", authorization).content(OBJECT))
.andExpect(status().isUnauthorized());
String key = "git:lfs-action:" + path.substring(path.lastIndexOf('/') + 1);
com.fasterxml.jackson.databind.node.ObjectNode stored =
(com.fasterxml.jackson.databind.node.ObjectNode) json.readTree(state.get(key));
stored.put("expiresAt", java.time.Instant.now().minusSeconds(1).toString());
state.remove(key);
state.putIfAbsent(key, json.writeValueAsString(stored), java.time.Duration.ofSeconds(30));
mvc.perform(get(path).header("Authorization", authorization))
.andExpect(status().isUnauthorized());
reply =
mvc.perform(
basic(post(protocol(code) + "/info/lfs/objects/batch"), owner, rw)
.contentType(LfsGateway.MEDIA)
.content(
json.writeValueAsBytes(
Map.of(
"operation",
"upload",
"objects",
List.of(
Map.of(
"oid",
OID,
"size",
OBJECT.length))))))
.andExpect(status().isOk())
.andReturn();
action =
json.readTree(reply.getResponse().getContentAsByteArray())
.path("objects")
.get(0)
.path("actions")
.path("upload");
path = URI.create(action.path("href").asText()).getPath();
authorization = action.path("header").path("Authorization").asText();
mvc.perform(
put(path)
.header("Authorization", authorization)
.content(new byte[OBJECT.length - 1]))
.andExpect(status().isBadRequest());
mvc.perform(
put(path)
.header("Authorization", authorization)
.content(new byte[OBJECT.length]))
.andExpect(status().isBadRequest());
mvc.perform(
basic(post(protocol(code) + "/info/lfs/objects/batch"), owner, rw)
.contentType(LfsGateway.MEDIA)
.content("{"))
.andExpect(status().isBadRequest());
mvc.perform(
basic(post(protocol(code) + "/info/lfs/objects/batch"), owner, rw)
.contentType(LfsGateway.MEDIA)
.content(
json.writeValueAsBytes(
Map.of(
"operation",
"upload",
"objects",
List.of(
Map.of(
"oid",
OID,
"size",
1073741825L))))))
.andExpect(status().isPayloadTooLarge());
}
}
@@ -0,0 +1,515 @@
package com.linkerbot.vibemake.api;
import static org.assertj.core.api.Assertions.*;
import com.fasterxml.jackson.databind.*;
import com.linkerbot.vibemake.service.config.*;
import com.linkerbot.vibemake.service.config.RedisStateProperties;
import com.linkerbot.vibemake.service.gitea.GiteaClient;
import org.junit.jupiter.api.*;
import org.junit.jupiter.api.condition.EnabledIfEnvironmentVariable;
import org.junit.jupiter.api.io.TempDir;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.boot.test.web.server.LocalServerPort;
import org.springframework.context.annotation.Import;
import org.springframework.data.redis.core.*;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.test.context.ActiveProfiles;
import org.springframework.test.context.DynamicPropertyRegistry;
import org.springframework.test.context.DynamicPropertySource;
import java.net.*;
import java.net.http.*;
import java.nio.charset.StandardCharsets;
import java.nio.file.*;
import java.sql.*;
import java.time.Duration;
import java.util.*;
import java.util.concurrent.*;
import java.util.regex.*;
/** 只使用本测试创建的用户、仓库与临时 H2;不访问业务项目。 */
@SpringBootTest(
webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT,
properties = {
"spring.datasource.url=jdbc:h2:mem:gitlive;MODE=MySQL;DB_CLOSE_DELAY=-1",
"spring.datasource.username=sa",
"spring.datasource.password=",
"spring.datasource.driver-class-name=org.h2.Driver",
"vibemake.ai.enabled=false",
"vibemake.oss.enabled=false",
"vibemake.limits.enabled=false",
"vibemake.repository-write.worker-enabled=false",
"vibemake.git.enabled=true",
"vibemake.git.public-base-url=https://git.example/git",
"vibemake.git.upstream-username=fixture",
"vibemake.git.upstream-token=fixture",
"vibemake.redis.real-test=true",
"vibemake.redis.key-prefix=vibemake:test:git-live-${random.uuid}",
"logging.level.root=WARN"
})
@ActiveProfiles("test")
@Import(GiteaLiveTest.Config.class)
@EnabledIfEnvironmentVariable(named = "VIBEMAKE_LIVE_GITEA", matches = "true")
class GitGatewayLiveTest {
@LocalServerPort int port;
@Autowired ObjectMapper json;
@Autowired JdbcTemplate jdbc;
@Autowired VibemakeProperties config;
@Autowired GitGatewayProperties gitConfig;
@Autowired GiteaClient gitea;
@Autowired StringRedisTemplate redis;
@Autowired RedisStateProperties namespace;
static String mysqlDatabase;
static String mysqlUrl;
@DynamicPropertySource
static void isolatedMysql(DynamicPropertyRegistry registry) throws Exception {
if (!"true".equals(System.getenv("GIT_GATEWAY_MYSQL"))) return;
String original = System.getenv("DB_URL");
if (original == null || !original.startsWith("jdbc:mysql:"))
throw new IllegalStateException("Git 联调需要 DB_URL");
String requestedDatabase =
"vibemake_git_verify_" + UUID.randomUUID().toString().replace("-", "");
try (Connection connection =
DriverManager.getConnection(
original,
System.getenv("DB_USERNAME"),
System.getenv("DB_PASSWORD"));
Statement statement = connection.createStatement()) {
statement.execute(
"CREATE DATABASE "
+ requestedDatabase
+ " CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci");
mysqlDatabase = requestedDatabase;
}
String plain = original.split("\\?", 2)[0];
mysqlUrl =
plain.substring(0, plain.lastIndexOf('/') + 1)
+ mysqlDatabase
+ (original.contains("?") ? "?" + original.split("\\?", 2)[1] : "");
registry.add("spring.datasource.url", () -> mysqlUrl);
registry.add("spring.datasource.username", () -> System.getenv("DB_USERNAME"));
registry.add("spring.datasource.password", () -> System.getenv("DB_PASSWORD"));
registry.add("spring.datasource.driver-class-name", () -> "com.mysql.cj.jdbc.Driver");
}
@AfterAll
static void dropOnlyTemporaryDatabase() throws Exception {
if (mysqlDatabase == null) return;
if (!mysqlDatabase.matches("vibemake_git_verify_[a-f0-9]{32}"))
throw new IllegalStateException("拒绝清理非测试库");
try (Connection connection =
DriverManager.getConnection(
System.getenv("DB_URL"),
System.getenv("DB_USERNAME"),
System.getenv("DB_PASSWORD"));
Statement statement = connection.createStatement()) {
statement.execute("DROP DATABASE " + mysqlDatabase);
}
}
@TempDir Path directory;
final HttpClient http =
HttpClient.newBuilder()
.followRedirects(HttpClient.Redirect.NEVER)
.connectTimeout(Duration.ofSeconds(10))
.build();
String bot;
String siteToken;
String username;
String project;
String gitToken;
String gitAuth;
Path askpass;
boolean botCreated;
@BeforeEach
void setup() throws Exception {
if (mysqlUrl == null) SchemaSupport.reset(jdbc);
else
for (String doc : List.of("database.md", "admin-database.md")) {
Matcher tables =
Pattern.compile("CREATE TABLE [\\s\\S]*?;")
.matcher(Files.readString(Path.of("../docs/" + doc)));
while (tables.find()) jdbc.execute(tables.group());
}
String suffix = UUID.randomUUID().toString().replace("-", "").substring(0, 16);
bot = "vbg_test_" + suffix;
String password = "Vbg!" + UUID.randomUUID() + "a9";
admin(
"POST",
"/admin/users",
Map.of(
"username",
bot,
"email",
bot + "@example.invalid",
"password",
password,
"must_change_password",
false,
"send_notify",
false),
201);
botCreated = true;
HttpRequest mint =
HttpRequest.newBuilder(
URI.create(
config.getGitea().getBaseUrl()
+ "/api/v1/users/"
+ bot
+ "/tokens"))
.header(
"Authorization",
"Basic "
+ Base64.getEncoder()
.encodeToString(
(bot + ":" + password)
.getBytes(StandardCharsets.UTF_8)))
.header("Content-Type", "application/json")
.POST(
HttpRequest.BodyPublishers.ofByteArray(
json.writeValueAsBytes(
Map.of(
"name",
"gateway-live",
"scopes",
List.of("read:user", "write:repository")))))
.build();
HttpResponse<byte[]> minted = http.send(mint, HttpResponse.BodyHandlers.ofByteArray());
assertThat(minted.statusCode()).as("创建测试技术账号PAT").isEqualTo(201);
gitConfig.setUpstreamUsername(bot);
gitConfig.setUpstreamToken(json.readTree(minted.body()).path("sha1").asText());
// 仅回环联调使用明文 HTTP;生产配置启动校验仍强制公网 HTTPS。
gitConfig.setPublicBaseUrl("http://127.0.0.1:" + port + "/git");
JsonNode captcha = api("GET", "/api/auth/captcha", null, null);
JsonNode login =
api(
"POST",
"/api/auth/login",
Map.of(
"email",
"gateway-live-" + suffix + "@example.invalid",
"captchaCode",
captcha.path("captchaCode").asText(),
"captchaAnswer",
"ABCDE"),
null);
siteToken = login.path("token").asText();
username = login.path("user").path("userCode").asText();
project =
api(
"POST",
"/api/workspace/projects",
Map.of("name", "网关真实联调", "projectType", "GENERAL"),
siteToken)
.path("projectCode")
.asText();
gitToken =
api(
"POST",
"/api/workspace/projects/" + project + "/git/tokens",
Map.of("name", "测试本机", "scope", "READ_WRITE"),
siteToken)
.path("token")
.asText();
gitAuth =
"Basic "
+ Base64.getEncoder()
.encodeToString(
(username + ":" + gitToken)
.getBytes(StandardCharsets.UTF_8));
askpass = directory.resolve("askpass.sh");
Files.writeString(
askpass,
"#!/bin/sh\n"
+ "case \"$1\" in *Username*) printf '%s' \"$VBG_USERNAME\";; *) printf '%s'"
+ " \"$VBG_SECRET\";; esac\n");
Files.setPosixFilePermissions(
askpass, java.nio.file.attribute.PosixFilePermissions.fromString("rwx------"));
}
@AfterEach
void cleanup() throws Exception {
for (String name : jdbc.queryForList("SELECT gitea_repo_name FROM project", String.class)) {
if (!name.matches("p_([0-9a-f]{32}|[0-9abcdefghjkmnpqrstvwxyz]{12})"))
throw new IllegalStateException("拒绝清理非测试仓库");
admin("DELETE", "/repos/" + config.getGitea().getOwner() + "/" + name, null, 204);
}
if (botCreated) admin("DELETE", "/admin/users/" + bot, null, 204);
assertThat(namespace.getKeyPrefix()).startsWith("vibemake:test:git-live-");
try (Cursor<String> cursor =
redis.scan(
ScanOptions.scanOptions()
.match(namespace.prefix() + "*")
.count(100)
.build())) {
while (cursor.hasNext()) redis.delete(cursor.next());
}
}
JsonNode admin(String method, String path, Object body, int expected) throws Exception {
HttpRequest.Builder builder =
HttpRequest.newBuilder(
URI.create(config.getGitea().getBaseUrl() + "/api/v1" + path))
.timeout(Duration.ofSeconds(40))
.header("Authorization", "token " + config.getGitea().getToken())
.header("Content-Type", "application/json");
HttpResponse<byte[]> result =
http.send(
builder.method(
method,
body == null
? HttpRequest.BodyPublishers.noBody()
: HttpRequest.BodyPublishers.ofByteArray(
json.writeValueAsBytes(body)))
.build(),
HttpResponse.BodyHandlers.ofByteArray());
assertThat(result.statusCode()).as("Gitea 测试管理请求 %s %s", method, path).isEqualTo(expected);
return result.body().length == 0 ? json.nullNode() : json.readTree(result.body());
}
JsonNode api(String method, String path, Object body, String token) throws Exception {
HttpRequest.Builder builder =
HttpRequest.newBuilder(URI.create("http://127.0.0.1:" + port + path))
.timeout(Duration.ofSeconds(90))
.header("Content-Type", "application/json");
if (token != null) builder.header("Authorization", "Bearer " + token);
HttpResponse<byte[]> result =
http.send(
builder.method(
method,
body == null
? HttpRequest.BodyPublishers.noBody()
: HttpRequest.BodyPublishers.ofByteArray(
json.writeValueAsBytes(body)))
.build(),
HttpResponse.BodyHandlers.ofByteArray());
JsonNode parsed = json.readTree(result.body());
assertThat(result.statusCode())
.as("网站请求 %s %s code=%s", method, path, parsed.path("code").asText())
.isEqualTo(200);
return parsed.path("data");
}
record Command(int status, String output) {}
Command command(Path cwd, String... args) throws Exception {
List<String> command =
new ArrayList<>(
List.of(
"git",
"-c",
"credential.helper=",
"-c",
"credential.useHttpPath=true"));
command.addAll(List.of(args));
ProcessBuilder builder =
new ProcessBuilder(command).directory(cwd.toFile()).redirectErrorStream(true);
// 凭据只在环境中传递;不出现在命令行、URL、断言正文或日志中。
builder.environment().put("GIT_ASKPASS", askpass.toString());
builder.environment().put("VBG_USERNAME", username);
builder.environment().put("VBG_SECRET", gitToken);
builder.environment().put("GIT_TERMINAL_PROMPT", "0");
builder.environment().put("GIT_AUTHOR_NAME", "Gateway fixture");
builder.environment().put("GIT_AUTHOR_EMAIL", "fixture@example.invalid");
builder.environment().put("GIT_COMMITTER_NAME", "Gateway fixture");
builder.environment().put("GIT_COMMITTER_EMAIL", "fixture@example.invalid");
Process process = builder.start();
CompletableFuture<byte[]> output =
CompletableFuture.supplyAsync(
() -> {
try {
return process.getInputStream().readAllBytes();
} catch (Exception error) {
throw new IllegalStateException(error);
}
});
if (!process.waitFor(180, TimeUnit.SECONDS)) {
process.destroyForcibly();
throw new IllegalStateException("Git 联调超时");
}
return new Command(
process.exitValue(),
new String(output.get(5, TimeUnit.SECONDS), StandardCharsets.UTF_8));
}
String git(Path cwd, String... args) throws Exception {
Command result = command(cwd, args);
assertThat(result.status())
.as(
"Git %s: %s",
args[0],
result.output()
.replace(gitToken, "[redacted]")
.replace(gitConfig.getUpstreamToken(), "[redacted]")
.replaceAll(
"(?i)(Basic|Bearer) [A-Za-z0-9_+./=-]+", "$1 [redacted]"))
.isZero();
return result.output().trim();
}
@Test
void nativeGitAndLargeLfsRoundTripWithRefProtectionAndExplicitPublication() throws Exception {
String url = "http://127.0.0.1:" + port + "/git/projects/" + project + ".git";
String tree = "/api/workspace/projects/" + project + "/tree";
String first = api("GET", tree, null, siteToken).path("commitSha").asText();
api(
"PUT",
"/api/workspace/projects/" + project + "/publication",
Map.of("commitSha", first),
siteToken);
git(
directory,
"-c",
"protocol.version=0",
"clone",
"--single-branch",
"--branch",
"main",
url,
"one");
Path one = directory.resolve("one");
git(one, "config", "credential.useHttpPath", "true");
git(one, "lfs", "install", "--local");
git(one, "lfs", "track", "*.bin");
byte[] bytes = new byte[51 * 1024 * 1024 + 17];
new Random(71).nextBytes(bytes);
Files.write(one.resolve("large.bin"), bytes);
Files.createDirectories(one.resolve("子目录"));
Files.writeString(one.resolve("子目录/说明.md"), "本地提交\n");
git(one, "add", ".");
git(one, "commit", "-m", "本地 Git 和 LFS 更新");
String second = git(one, "rev-parse", "HEAD");
git(one, "push", "origin", "main");
assertThat(api("GET", tree, null, siteToken).path("commitSha").asText()).isEqualTo(second);
assertThat(
api("GET", "/api/community/projects/" + project + "/tree", null, null)
.path("commitSha")
.asText())
.isEqualTo(first);
assertThat(
api(
"GET",
"/api/workspace/projects/" + project + "/commits",
null,
siteToken)
.path("items")
.get(0)
.path("message")
.asText())
.contains("本地 Git 和 LFS 更新");
git(directory, "-c", "protocol.version=2", "clone", "--no-checkout", url, "two");
Path two = directory.resolve("two");
git(two, "lfs", "install", "--local");
git(two, "checkout", "main");
assertThat(Arrays.equals(Files.readAllBytes(two.resolve("large.bin")), bytes))
.as("LFS 实体逐字节一致")
.isTrue();
assertThat(Files.readString(two.resolve("子目录/说明.md"))).isEqualTo("本地提交\n");
assertThat(command(one, "push", "origin", "HEAD:refs/heads/other").status()).isNotZero();
assertThat(command(one, "push", "origin", "HEAD:refs/tags/test").status()).isNotZero();
assertThat(command(one, "push", "origin", ":main").status()).isNotZero();
git(one, "reset", "--hard", first);
assertThat(command(one, "push", "--force", "origin", "main").status()).isNotZero();
assertThat(api("GET", tree, null, siteToken).path("commitSha").asText()).isEqualTo(second);
api(
"PUT",
"/api/workspace/projects/" + project + "/publication",
Map.of("commitSha", second),
siteToken);
assertThat(
api("GET", "/api/community/projects/" + project + "/tree", null, null)
.path("commitSha")
.asText())
.isEqualTo(second);
Files.writeString(two.resolve("pull.txt"), "下一次更新");
git(two, "add", "pull.txt");
git(two, "commit", "-m", "继续更新");
git(two, "push", "origin", "main");
git(one, "pull", "--ff-only");
assertThat(Files.readString(one.resolve("pull.txt"))).isEqualTo("下一次更新");
// 非 LFS 的大 pack + 极小 postBuffer 强制走 HTTP 分块及认证 probe。
byte[] packData = new byte[2 * 1024 * 1024 + 7];
new Random(19).nextBytes(packData);
Files.write(one.resolve("native.dat"), packData);
git(one, "add", "native.dat");
git(one, "commit", "-m", "大 pack 更新");
git(one, "-c", "http.postBuffer=1024", "push", "origin", "main");
git(two, "pull", "--ff-only");
assertThat(Arrays.equals(Files.readAllBytes(two.resolve("native.dat")), packData)).isTrue();
git(two, "checkout", "-b", "local-only");
Files.writeString(two.resolve("left.txt"), "left");
git(two, "add", "left.txt");
git(two, "commit", "-m", "本地分支提交");
git(two, "checkout", "main");
Files.writeString(two.resolve("right.txt"), "right");
git(two, "add", "right.txt");
git(two, "commit", "-m", "main 提交");
git(two, "merge", "--no-ff", "local-only", "-m", "合并本地提交");
git(two, "push", "origin", "main");
Path signing = directory.resolve("signing-key");
Process keygen =
new ProcessBuilder(
"ssh-keygen",
"-q",
"-t",
"ed25519",
"-N",
"",
"-f",
signing.toString())
.redirectErrorStream(true)
.start();
assertThat(keygen.waitFor(10, TimeUnit.SECONDS)).isTrue();
assertThat(keygen.exitValue()).isZero();
git(
two,
"-c",
"gpg.format=ssh",
"-c",
"user.signingkey=" + signing,
"commit",
"--allow-empty",
"-S",
"-m",
"签名提交");
String signed = git(two, "rev-parse", "HEAD");
git(two, "push", "origin", "main");
assertThat(api("GET", tree, null, siteToken).path("commitSha").asText()).isEqualTo(signed);
assertThat(git(two, "cat-file", "-p", "HEAD"))
.contains("gpgsig -----BEGIN SSH SIGNATURE-----");
String originalToken = gitToken;
String other =
api(
"POST",
"/api/workspace/projects",
Map.of("name", "第二项目", "projectType", "GENERAL"),
siteToken)
.path("projectCode")
.asText();
gitToken =
api(
"POST",
"/api/workspace/projects/" + other + "/git/tokens",
Map.of("name", "第二项目凭据", "scope", "READ_WRITE"),
siteToken)
.path("token")
.asText();
String otherUrl = "http://127.0.0.1:" + port + "/git/projects/" + other + ".git";
git(directory, "clone", otherUrl, "other");
Path otherRepo = directory.resolve("other");
git(otherRepo, "lfs", "install", "--local");
git(otherRepo, "lfs", "track", "*.bin");
Files.write(otherRepo.resolve("other.bin"), Arrays.copyOf(bytes, 1024));
git(otherRepo, "add", ".");
git(otherRepo, "commit", "-m", "第二项目 LFS");
git(otherRepo, "push", "origin", "main");
assertThat(command(one, "fetch", "origin").status()).isNotZero();
gitToken = originalToken;
git(one, "fetch", "origin");
}
}
@@ -26,11 +26,12 @@ final class SchemaSupport {
"JSON NOT NULL DEFAULT (JSON_ARRAY())",
"CLOB NOT NULL DEFAULT '[]'")
.replace(" CHARACTER SET utf8mb4 COLLATE utf8mb4_bin", "")
.replace(" CHARACTER SET ascii COLLATE ascii_bin", "")
.replaceAll(" COMMENT '[^']*'", "")
.replaceAll("\\) ENGINE=InnoDB[^;]*;", ");");
jdbc.execute(ddl);
tables++;
}
if (tables != 6) throw new IllegalStateException("必须包含四张基础表、迁移任务表和仓库写入操作表");
if (tables != 7) throw new IllegalStateException("必须包含七张业务与审计表");
}
}
@@ -1263,7 +1263,7 @@ class WorkflowTest {
"PROJECT",
"PROJECT_COMMENT",
"PROJECT_LIKE",
"PROJECT_IMPORT_TASK", "REPOSITORY_WRITE_OPERATION");
"PROJECT_IMPORT_TASK", "REPOSITORY_WRITE_OPERATION", "PROJECT_GIT_TOKEN");
}
@Test
@@ -0,0 +1,49 @@
package com.linkerbot.vibemake.bean;
import jakarta.validation.constraints.Max;
import jakarta.validation.constraints.Min;
import jakarta.validation.constraints.NotBlank;
import jakarta.validation.constraints.NotNull;
import jakarta.validation.constraints.Size;
import java.time.LocalDateTime;
public final class GitContracts {
private GitContracts() {}
public enum Scope {
READ_ONLY,
READ_WRITE
}
public record TokenInput(
@NotBlank @Size(max = 64) String name,
@NotNull Scope scope,
@Min(1) @Max(365) Integer expiresInDays) {}
public record TokenView(
String tokenCode,
String name,
Scope scope,
String status,
LocalDateTime createdAt,
LocalDateTime expiresAt,
LocalDateTime revokedAt,
LocalDateTime lastUsedAt) {}
public record IssuedToken(TokenView credential, String token) {
@Override
public String toString() {
return "IssuedToken[credential=" + credential + ", token=[redacted]]";
}
}
public record ConnectionView(
String cloneUrl,
String username,
String defaultBranch,
String headCommitSha,
String publishedCommitSha,
String writeStatus,
boolean canPush) {}
}
@@ -0,0 +1,21 @@
package com.linkerbot.vibemake.bean.entity;
import lombok.Getter;
import lombok.Setter;
import java.time.LocalDateTime;
@Getter
@Setter
public class ProjectGitToken extends BaseEntity {
private String tokenCode;
private String projectCode;
private Long ownerUserId;
private String tokenName;
private String tokenHash;
private String permission;
private Long sessionVersion;
private LocalDateTime expiresAt;
private LocalDateTime revokedAt;
private LocalDateTime lastUsedAt;
}
@@ -12,6 +12,10 @@ import java.time.LocalDateTime;
public class RepositoryWriteOperation {
@TableId(type = IdType.INPUT)
private String operationCode;
private String operationKind;
private String targetCommitSha;
private String actorUserCode;
private String gitTokenCode;
private String projectCode;
private Long repositoryId;
private String repositoryOwner;
@@ -0,0 +1,136 @@
package com.linkerbot.vibemake.security;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.linkerbot.vibemake.common.BusinessException;
import com.linkerbot.vibemake.service.git.*;
import com.linkerbot.vibemake.service.state.ClientAddresses;
import jakarta.servlet.*;
import jakarta.servlet.http.*;
import lombok.RequiredArgsConstructor;
import org.springframework.stereotype.Component;
import org.springframework.web.filter.OncePerRequestFilter;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.util.*;
import java.util.regex.*;
@Component
@RequiredArgsConstructor
public class GitAuthenticationFilter extends OncePerRequestFilter {
private static final org.slf4j.Logger log =
org.slf4j.LoggerFactory.getLogger(GitAuthenticationFilter.class);
public static final String ACCESS = GitAuthenticationFilter.class.getName() + ".access";
public static final String ACTION = GitAuthenticationFilter.class.getName() + ".action";
private static final Pattern PATH =
Pattern.compile("/git/projects/([A-Za-z0-9]{1,32})\\.git/(.+)");
private final GitTokens tokens;
private final LfsGateway lfs;
private final GitLimits limits;
private final ClientAddresses addresses;
private final ObjectMapper json;
@Override
protected void doFilterInternal(
HttpServletRequest request, HttpServletResponse response, FilterChain chain)
throws IOException, ServletException {
response.setHeader("Cache-Control", "no-store");
String ip = addresses.resolve(request);
try {
tokens.enabled();
limits.authenticationAllowed(ip);
Matcher matcher =
PATH.matcher(
request.getRequestURI().substring(request.getContextPath().length()));
if (!matcher.matches()) throw BusinessException.notFound();
String code = matcher.group(1);
String endpoint = matcher.group(2);
String authorization = request.getHeader("Authorization");
GitAccess access;
if (endpoint.startsWith("info/lfs/actions/")) {
if (authorization == null || !authorization.startsWith("Bearer "))
throw GitTokens.unauthorized();
LfsGateway.Action action =
lfs.authenticate(
endpoint.substring("info/lfs/actions/".length()),
code,
authorization.substring(7),
request.getMethod());
access = action.access();
request.setAttribute(ACTION, action);
} else {
if (authorization == null
|| !authorization.startsWith("Basic ")
|| authorization.length() > 2048) throw GitTokens.unauthorized();
String decoded;
try {
decoded =
new String(
Base64.getDecoder().decode(authorization.substring(6)),
StandardCharsets.UTF_8);
} catch (IllegalArgumentException error) {
throw GitTokens.unauthorized();
}
int separator = decoded.indexOf(':');
if (separator < 1) throw GitTokens.unauthorized();
access =
tokens.authenticate(
decoded.substring(0, separator),
decoded.substring(separator + 1),
code);
}
limits.request(access, endpoint.startsWith("info/lfs/"));
request.setAttribute(ACCESS, access);
} catch (BusinessException error) {
if (request.getHeader("Authorization") != null)
log.warn("Git 认证或限流拒绝 code={} status={}", error.code(), error.status());
// Git 客户端的首次无凭据请求用于协商 Basic challenge,不计为密码认证失败。
if (error.status() == 401 && request.getHeader("Authorization") != null)
limits.authenticationFailed(ip);
failure(request, response, json, error);
return;
} catch (RuntimeException error) {
log.warn("Git 认证服务异常 errorType={}", error.getClass().getSimpleName());
failure(
request,
response,
json,
new BusinessException(503, "GIT_UNAVAILABLE", "Git 鉴权服务暂不可用"));
return;
}
chain.doFilter(request, response);
}
public static void failure(
HttpServletRequest request,
HttpServletResponse response,
ObjectMapper json,
BusinessException error)
throws IOException {
if (response.isCommitted()) {
response.getOutputStream().close();
return;
}
response.resetBuffer();
response.setHeader("Content-Length", null);
response.setStatus(error.status());
response.setHeader("Cache-Control", "no-store");
if (error.status() == 401)
response.setHeader(
"WWW-Authenticate", "Basic realm=\"Vibemake Git\", charset=\"UTF-8\"");
if (error.status() == 429) response.setHeader("Retry-After", "60");
if (request.getRequestURI().contains("/info/lfs/")) {
response.setContentType(LfsGateway.MEDIA + ";charset=UTF-8");
json.writeValue(response.getOutputStream(), Map.of("message", error.getMessage()));
} else {
response.setContentType("text/plain;charset=UTF-8");
response.getOutputStream()
.write(
("error: " + error.getMessage() + "\n")
.getBytes(StandardCharsets.UTF_8));
}
}
}
@@ -0,0 +1,40 @@
package com.linkerbot.vibemake.security;
import org.springframework.boot.web.servlet.FilterRegistrationBean;
import org.springframework.context.annotation.*;
import org.springframework.core.annotation.Order;
import org.springframework.security.config.annotation.web.builders.HttpSecurity;
import org.springframework.security.config.http.SessionCreationPolicy;
import org.springframework.security.web.SecurityFilterChain;
import org.springframework.security.web.authentication.UsernamePasswordAuthenticationFilter;
@Configuration
public class GitSecurityConfiguration {
@Bean
public FilterRegistrationBean<GitAuthenticationFilter> disableGitServletRegistration(
GitAuthenticationFilter filter) {
FilterRegistrationBean<GitAuthenticationFilter> registration =
new FilterRegistrationBean<>(filter);
registration.setEnabled(false);
return registration;
}
@Bean
@Order(1)
public SecurityFilterChain gitSecurityFilterChain(
HttpSecurity http, GitAuthenticationFilter filter) throws Exception {
return http.securityMatcher("/git/**")
.csrf(csrf -> csrf.disable())
.cors(cors -> cors.disable())
.sessionManagement(
session -> session.sessionCreationPolicy(SessionCreationPolicy.STATELESS))
.requestCache(cache -> cache.disable())
.formLogin(form -> form.disable())
.httpBasic(basic -> basic.disable())
.logout(logout -> logout.disable())
// 专用过滤器已完成每请求鉴权,不创建可被普通 Bearer 链复用的登录状态。
.authorizeHttpRequests(auth -> auth.anyRequest().permitAll())
.addFilterBefore(filter, UsernamePasswordAuthenticationFilter.class)
.build();
}
}
@@ -0,0 +1,95 @@
package com.linkerbot.vibemake.service.config;
import jakarta.validation.constraints.AssertTrue;
import jakarta.validation.constraints.Max;
import jakarta.validation.constraints.Min;
import jakarta.validation.constraints.Positive;
import lombok.Getter;
import lombok.Setter;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.stereotype.Component;
import org.springframework.validation.annotation.Validated;
import java.net.URI;
import java.time.Duration;
@Getter
@Setter
@Component
@Validated
@ConfigurationProperties("vibemake.git")
public class GitGatewayProperties {
private boolean enabled;
private String publicBaseUrl = "";
private String upstreamUsername = "";
private String upstreamToken = "";
private Duration timeout = Duration.ofMinutes(10);
@Positive private long maxPackBytes = 5L * 1024 * 1024 * 1024;
@Positive private long maxLfsObjectBytes = 1024L * 1024 * 1024;
@Min(1024)
@Max(16777216)
private int maxMetadataBytes = 8 * 1024 * 1024;
@Min(1)
@Max(500)
private int maxBatchObjects = 500;
@Min(1)
@Max(32)
private int readConcurrent = 4;
@Min(1)
@Max(32)
private int pushConcurrent = 2;
@Min(1)
@Max(64)
private int lfsConcurrent = 8;
@Min(1)
@Max(10)
private int maxActiveTokens = 10;
@Min(1)
@Max(10000)
private int gitRequestsPerMinute = 120;
@Min(1)
@Max(10000)
private int lfsRequestsPerMinute = 600;
@Min(1)
@Max(1000)
private int authFailuresPerMinute = 20;
@AssertTrue(message = "Git 网关启用时必须配置公网 HTTPS 基础地址(以 /git 结尾)和独立技术账号凭据")
public boolean isConfigurationValid() {
if (!enabled) return true;
try {
URI uri = URI.create(publicBaseUrl);
return "https".equals(uri.getScheme())
&& uri.getHost() != null
&& "/git".equals(uri.getRawPath())
&& uri.getRawUserInfo() == null
&& uri.getRawQuery() == null
&& uri.getRawFragment() == null
&& upstreamUsername != null
&& !upstreamUsername.isBlank()
&& !upstreamUsername.contains(":")
&& upstreamToken != null
&& !upstreamToken.isBlank();
} catch (IllegalArgumentException error) {
return false;
}
}
@AssertTrue(message = "Git 请求预算必须在 100 毫秒至 1 小时之间")
public boolean isTimeoutValid() {
return timeout != null
&& timeout.compareTo(Duration.ofMillis(100)) >= 0
&& timeout.compareTo(Duration.ofHours(1)) <= 0;
}
}
@@ -0,0 +1,5 @@
package com.linkerbot.vibemake.service.git;
/** 只在请求内部使用;不包含凭据明文,不作为 API 响应返回。 */
public record GitAccess(
long userId, String userCode, String tokenCode, String projectCode, boolean writable) {}
@@ -0,0 +1,294 @@
package com.linkerbot.vibemake.service.git;
import com.linkerbot.vibemake.bean.GitContracts.ConnectionView;
import com.linkerbot.vibemake.bean.entity.Project;
import com.linkerbot.vibemake.common.BusinessException;
import com.linkerbot.vibemake.service.ProjectService;
import com.linkerbot.vibemake.service.UserService;
import com.linkerbot.vibemake.service.config.GitGatewayProperties;
import com.linkerbot.vibemake.service.gitea.GiteaClient;
import com.linkerbot.vibemake.service.write.RepositoryWrites;
import jakarta.servlet.http.*;
import lombok.RequiredArgsConstructor;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.stereotype.Service;
import java.io.*;
import java.util.*;
import java.util.zip.GZIPInputStream;
@Service
@RequiredArgsConstructor
public class GitGateway {
private static final Logger log = LoggerFactory.getLogger(GitGateway.class);
private final GitTokens tokens;
private final GitTransport transport;
private final GitGatewayProperties properties;
private final ProjectService projects;
private final UserService users;
private final GiteaClient gitea;
private final GitPolicy policy;
private final RepositoryWrites writes;
private final GitLimits limits;
public ConnectionView connection(long userId, String code) {
tokens.enabled();
Project project = projects.readyOwner(userId, code);
boolean canPush =
"IDLE".equals(project.getRepoWriteStatus()) && writes.active(code) == null;
if (canPush) {
try {
policy.verify(project);
} catch (BusinessException error) {
canPush = false;
}
}
return new ConnectionView(
properties.getPublicBaseUrl() + "/projects/" + code + ".git",
users.require(userId).getUserCode(),
"main",
gitea.head(projects.repository(project)),
project.getPublishedCommitSha(),
project.getRepoWriteStatus(),
canPush);
}
public void serve(
GitAccess access,
String endpoint,
HttpServletRequest request,
HttpServletResponse response)
throws IOException {
String service = "info/refs".equals(endpoint) ? request.getParameter("service") : endpoint;
boolean advertisement = "info/refs".equals(endpoint);
if (!("git-upload-pack".equals(service) || "git-receive-pack".equals(service))
|| !(advertisement ? "GET" : "POST").equals(request.getMethod()))
throw GitIo.invalid();
if (request.getParameterMap().size() != (advertisement ? 1 : 0)) throw GitIo.invalid();
boolean push = "git-receive-pack".equals(service);
if (push) GitTokens.requireWrite(access);
long started = System.nanoTime();
java.util.concurrent.atomic.AtomicLong transferred =
new java.util.concurrent.atomic.AtomicLong();
log.info(
"Git 请求开始 projectCode={} userCode={} tokenCode={} endpoint={}",
access.projectCode(),
access.userCode(),
access.tokenCode(),
endpoint);
try (AutoCloseable permit = limits.acquire(push, false);
GitTransport.Budget budget = transport.budget()) {
budget.register(request.getInputStream());
budget.onTimeout(
() -> {
if (response.isCommitted()) response.getOutputStream().close();
});
Project project = projects.readyOwner(access.userId(), access.projectCode());
Map<String, String> headers = new HashMap<>();
headers.put(
"Accept",
"application/x-" + service + (advertisement ? "-advertisement" : "-result"));
if (!push && request.getHeader("Git-Protocol") != null) {
if (!request.getHeader("Git-Protocol").matches("version=[012]"))
throw GitIo.invalid();
headers.put("Git-Protocol", request.getHeader("Git-Protocol"));
}
String path =
transport.repositoryPath(project)
+ "/"
+ endpoint
+ (advertisement ? "?service=" + service : "");
if (push && !advertisement) {
requireContentType(request, "application/x-git-receive-pack-request");
InputStream input =
GitIo.counted(
decoded(request, properties.getMaxPackBytes(), budget),
transferred);
GitIo.PushCommand command = GitIo.command(input);
if (command == null) {
if (input.read() != -1) throw GitIo.invalid();
response.setContentType("application/x-git-receive-pack-result");
response.setContentLength(0);
return;
}
java.util.concurrent.atomic.AtomicReference<Project> pushing =
new java.util.concurrent.atomic.AtomicReference<>();
byte[] result =
writes.push(
access,
command,
() -> {
GitAccess current = tokens.recheck(access);
GitTokens.requireWrite(current);
Project fresh =
projects.readyOwner(
current.userId(), current.projectCode());
if (!command.oldSha()
.equals(gitea.head(projects.repository(fresh))))
throw new BusinessException(
409, "GIT_STALE_MAIN", "main 已变化,请先 pull、合并后再推送");
policy.verify(fresh);
budget.check();
pushing.set(fresh);
return fresh;
},
() -> {
headers.put(
"Content-Type",
"application/x-git-receive-pack-request");
InputStream body =
new SequenceInputStream(
new ByteArrayInputStream(command.prefix()),
input);
try (GitTransport.Response upstream =
transport.exchange(
transport.repositoryPath(pushing.get())
+ "/git-receive-pack",
"POST",
body,
-1,
headers,
budget)) {
requireSuccess(
upstream, "application/x-git-receive-pack-result");
return GitIo.metadata(
upstream.input(), properties.getMaxMetadataBytes());
} catch (IOException error) {
budget.check();
throw GitIo.upstreamInvalid();
}
});
response.setContentType("application/x-git-receive-pack-result");
response.setContentLength(result.length);
response.getOutputStream().write(result);
log.info(
"Git 推送响应完成 projectCode={} userCode={} tokenCode={} oldSha={} newSha={}"
+ " bytes={} durationMs={}",
access.projectCode(),
access.userCode(),
access.tokenCode(),
command.oldSha(),
command.newSha(),
transferred.get(),
(System.nanoTime() - started) / 1000000);
} else {
InputStream body = null;
if (!advertisement) {
requireContentType(request, "application/x-git-upload-pack-request");
body = decoded(request, properties.getMaxMetadataBytes(), budget);
headers.put("Content-Type", "application/x-git-upload-pack-request");
}
String type =
"application/x-" + service + (advertisement ? "-advertisement" : "-result");
try (GitTransport.Response upstream =
transport.exchange(path, request.getMethod(), body, -1, headers, budget)) {
requireSuccess(upstream, type);
response.setContentType(type);
if (push) {
byte[] result =
GitIo.receiveAdvertisement(
GitIo.metadata(
upstream.input(),
properties.getMaxMetadataBytes()));
response.setContentLength(result.length);
response.getOutputStream().write(result);
} else {
long bytes =
copy(
upstream.input(),
response,
properties.getMaxPackBytes(),
budget,
transferred);
log.info(
"Git 读取完成 projectCode={} userCode={} tokenCode={} bytes={}"
+ " durationMs={}",
access.projectCode(),
access.userCode(),
access.tokenCode(),
bytes,
(System.nanoTime() - started) / 1000000);
}
}
}
} catch (BusinessException | IOException error) {
log.warn(
"Git 请求失败或取消 projectCode={} userCode={} tokenCode={} endpoint={} errorType={}"
+ " bytes={} durationMs={}",
access.projectCode(),
access.userCode(),
access.tokenCode(),
endpoint,
error.getClass().getSimpleName(),
transferred.get(),
(System.nanoTime() - started) / 1000000);
throw error;
} catch (Exception error) {
throw GitIo.upstreamInvalid();
}
}
public static InputStream decoded(
HttpServletRequest request, long maximum, GitTransport.Budget budget)
throws IOException {
if (request.getContentLengthLong() > maximum)
throw new BusinessException(413, "GIT_TRANSFER_TOO_LARGE", "Git/LFS 请求过大");
InputStream raw = GitIo.limited(request.getInputStream(), maximum);
budget.register(raw);
String encoding = request.getHeader("Content-Encoding");
if (encoding != null && !"identity".equalsIgnoreCase(encoding)) {
if (!"gzip".equalsIgnoreCase(encoding)) throw GitIo.invalid();
raw = new GZIPInputStream(raw);
}
return GitIo.limited(raw, maximum);
}
public static void requireContentType(HttpServletRequest request, String type) {
if (request.getContentType() == null
|| !request.getContentType().split(";", 2)[0].equalsIgnoreCase(type))
throw GitIo.invalid();
}
public static void requireSuccess(GitTransport.Response response, String type) {
if (response.status() != 200
|| !response.headers()
.firstValue("Content-Type")
.orElse("")
.split(";", 2)[0]
.equalsIgnoreCase(type)) throw GitIo.upstreamInvalid();
}
public static long copy(
InputStream input,
HttpServletResponse response,
long maximum,
GitTransport.Budget budget)
throws IOException {
return copy(input, response, maximum, budget, new java.util.concurrent.atomic.AtomicLong());
}
public static long copy(
InputStream input,
HttpServletResponse response,
long maximum,
GitTransport.Budget budget,
java.util.concurrent.atomic.AtomicLong transferred)
throws IOException {
byte[] buffer = new byte[65536];
long total = 0;
int count;
InputStream bounded = GitIo.limited(input, maximum);
while ((count = bounded.read(buffer)) >= 0) {
budget.check();
response.getOutputStream().write(buffer, 0, count);
total += count;
transferred.addAndGet(count);
}
budget.check();
return total;
}
}
@@ -0,0 +1,186 @@
package com.linkerbot.vibemake.service.git;
import com.linkerbot.vibemake.common.BusinessException;
import java.io.*;
import java.nio.charset.StandardCharsets;
import java.util.Set;
/** Git pkt-line 的有界元数据处理;pack 正文始终以流处理。 */
public final class GitIo {
private GitIo() {}
public record PushCommand(String oldSha, String newSha, byte[] prefix) {}
private static final Set<String> PUSH_CAPS = Set.of("report-status", "ofs-delta", "quiet");
public static InputStream counted(
InputStream input, java.util.concurrent.atomic.AtomicLong count) {
return new FilterInputStream(input) {
@Override
public int read() throws IOException {
byte[] one = new byte[1];
return read(one, 0, 1) < 0 ? -1 : one[0] & 255;
}
@Override
public int read(byte[] bytes, int offset, int length) throws IOException {
int size = super.read(bytes, offset, length);
if (size > 0) count.addAndGet(size);
return size;
}
};
}
public static InputStream limited(InputStream input, long maximum) {
return new FilterInputStream(input) {
private long count;
@Override
public int read() throws IOException {
byte[] bytes = new byte[1];
return read(bytes, 0, 1) < 0 ? -1 : bytes[0] & 255;
}
@Override
public int read(byte[] bytes, int offset, int length) throws IOException {
if (length == 0) return 0;
long remaining = maximum - count;
int size =
super.read(
bytes, offset, remaining >= length ? length : (int) remaining + 1);
if (size > 0 && (count += size) > maximum)
throw new BusinessException(413, "GIT_TRANSFER_TOO_LARGE", "Git/LFS 传输超过大小限制");
return size;
}
};
}
public static byte[] metadata(InputStream input, int maximum) throws IOException {
byte[] bytes = input.readNBytes(maximum + 1);
if (bytes.length > maximum)
throw new BusinessException(413, "GIT_METADATA_TOO_LARGE", "Git/LFS 元数据超过限制");
return bytes;
}
public static byte[] packet(InputStream input) throws IOException {
byte[] header = input.readNBytes(4);
if (header.length != 4) throw invalid();
String hex = new String(header, StandardCharsets.US_ASCII);
if (!hex.matches("[0-9a-fA-F]{4}")) throw invalid();
int size = Integer.parseInt(hex, 16);
if (size == 0) return null;
if (size < 4 || size > 65520) throw invalid();
byte[] data = input.readNBytes(size - 4);
if (data.length != size - 4) throw invalid();
return data;
}
public static void packet(OutputStream output, byte[] data) throws IOException {
output.write(
String.format(java.util.Locale.ROOT, "%04x", data.length + 4)
.getBytes(StandardCharsets.US_ASCII));
output.write(data);
}
private static boolean capability(String value) {
return PUSH_CAPS.contains(value)
|| value.matches("agent=[!-~]{1,128}")
|| value.equals("object-format=sha1")
|| value.equals("object-format=sha256");
}
public static PushCommand command(InputStream input) throws IOException {
byte[] first = packet(input);
// Smart HTTP 在大 pack 分块上传前会单独 POST 一个 flush 进行认证探测。
if (first == null) return null;
if (first.length > 4096) throw invalid();
String command = new String(first, StandardCharsets.UTF_8).stripTrailing();
int nul = command.indexOf('\0');
if (nul < 0) throw invalid();
String[] fields = command.substring(0, nul).split(" ", -1);
if (fields.length != 3 || !"refs/heads/main".equals(fields[2])) throw invalid();
String oldSha = com.linkerbot.vibemake.common.FilePaths.sha(fields[0]);
String newSha = com.linkerbot.vibemake.common.FilePaths.sha(fields[1]);
if (oldSha.length() != newSha.length() || oldSha.matches("0+") || newSha.matches("0+"))
throw invalid();
// 原生 send-pack 会在 NUL 后的能力列表前保留一个空格。
String[] caps = command.substring(nul + 1).strip().split(" ");
boolean report = false;
for (String cap : caps) {
if (!capability(cap)) throw invalid();
if (cap.equals("report-status")) report = true;
}
if (!report || packet(input) != null) throw invalid();
ByteArrayOutputStream prefix = new ByteArrayOutputStream();
packet(prefix, first);
prefix.write("0000".getBytes(StandardCharsets.US_ASCII));
return new PushCommand(oldSha, newSha, prefix.toByteArray());
}
public static byte[] receiveAdvertisement(byte[] bytes) throws IOException {
ByteArrayInputStream input = new ByteArrayInputStream(bytes);
byte[] service = upstreamPacket(input);
if (service == null
|| !new String(service, StandardCharsets.US_ASCII)
.equals("# service=git-receive-pack\n")
|| upstreamPacket(input) != null) throw upstreamInvalid();
String main = null;
String caps = null;
byte[] entry;
while ((entry = upstreamPacket(input)) != null) {
String line = new String(entry, StandardCharsets.UTF_8).stripTrailing();
int nul = line.indexOf('\0');
if (nul >= 0) {
caps = line.substring(nul + 1);
line = line.substring(0, nul);
}
if (line.matches("[0-9a-fA-F]{40,64} refs/heads/main")) main = line;
}
if (main == null || caps == null || input.available() != 0) throw upstreamInvalid();
StringBuilder filtered = new StringBuilder();
for (String cap : caps.split(" ")) if (capability(cap)) filtered.append(cap).append(' ');
if (!java.util.Arrays.asList(caps.split(" ")).contains("report-status"))
throw upstreamInvalid();
ByteArrayOutputStream output = new ByteArrayOutputStream();
packet(output, service);
output.write("0000".getBytes(StandardCharsets.US_ASCII));
packet(
output,
(main + "\0" + filtered.toString().trim() + "\n").getBytes(StandardCharsets.UTF_8));
output.write("0000".getBytes(StandardCharsets.US_ASCII));
return output.toByteArray();
}
public static boolean pushSucceeded(byte[] bytes) throws IOException {
ByteArrayInputStream input = new ByteArrayInputStream(bytes);
byte[] unpack = upstreamPacket(input);
if (unpack == null) throw upstreamInvalid();
String unpackStatus = new String(unpack, StandardCharsets.UTF_8).stripTrailing();
byte[] ref = upstreamPacket(input);
if (ref == null) throw upstreamInvalid();
String refStatus = new String(ref, StandardCharsets.UTF_8).stripTrailing();
if (!unpackStatus.startsWith("unpack ")
|| !(refStatus.equals("ok refs/heads/main")
|| refStatus.startsWith("ng refs/heads/main "))
|| upstreamPacket(input) != null
|| input.available() != 0) throw upstreamInvalid();
return unpackStatus.equals("unpack ok") && refStatus.equals("ok refs/heads/main");
}
private static byte[] upstreamPacket(InputStream input) throws IOException {
try {
return packet(input);
} catch (BusinessException error) {
throw upstreamInvalid();
}
}
public static BusinessException invalid() {
return new BusinessException(400, "GIT_COMMAND_REJECTED", "仅支持一次 main 更新,禁止删除、其他引用及扩展推送选项");
}
public static BusinessException upstreamInvalid() {
return new BusinessException(502, "GIT_UPSTREAM_ERROR", "Git 上游响应不完整或格式异常");
}
}
@@ -0,0 +1,62 @@
package com.linkerbot.vibemake.service.git;
import com.linkerbot.vibemake.common.BusinessException;
import com.linkerbot.vibemake.common.state.TransientStateStore;
import com.linkerbot.vibemake.service.config.GitGatewayProperties;
import org.springframework.stereotype.Component;
import java.time.Duration;
import java.util.concurrent.Semaphore;
@Component
public class GitLimits {
private final TransientStateStore state;
private final GitGatewayProperties properties;
private final Semaphore reads;
private final Semaphore pushes;
private final Semaphore lfs;
public GitLimits(TransientStateStore state, GitGatewayProperties properties) {
this.state = state;
this.properties = properties;
reads = new Semaphore(properties.getReadConcurrent());
pushes = new Semaphore(properties.getPushConcurrent());
lfs = new Semaphore(properties.getLfsConcurrent());
}
public void authenticationAllowed(String ip) {
String count =
state.get("git:auth:" + com.linkerbot.vibemake.common.SecurityHashes.sha256(ip));
if (count != null && Long.parseLong(count) >= properties.getAuthFailuresPerMinute())
throw busy();
}
public void authenticationFailed(String ip) {
state.increment(
"git:auth:" + com.linkerbot.vibemake.common.SecurityHashes.sha256(ip),
Duration.ofMinutes(1),
false);
}
public void request(GitAccess access, boolean isLfs) {
int maximum =
isLfs ? properties.getLfsRequestsPerMinute() : properties.getGitRequestsPerMinute();
if (state.increment(
"git:rate:" + (isLfs ? "lfs:" : "git:") + access.userCode(),
Duration.ofMinutes(1),
false)
.count()
> maximum) throw busy();
}
public AutoCloseable acquire(boolean push, boolean isLfs) {
Semaphore pool = isLfs ? lfs : push ? pushes : reads;
if (!pool.tryAcquire()) throw busy();
return pool::release;
}
private BusinessException busy() {
return new BusinessException(429, "GIT_BUSY", "Git/LFS 请求过于频繁或服务繁忙,请稍后重试");
}
}
@@ -0,0 +1,79 @@
package com.linkerbot.vibemake.service.git;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.linkerbot.vibemake.bean.entity.Project;
import com.linkerbot.vibemake.common.BusinessException;
import com.linkerbot.vibemake.service.ProjectService;
import com.linkerbot.vibemake.service.config.GitGatewayProperties;
import com.linkerbot.vibemake.service.gitea.GiteaClient;
import lombok.RequiredArgsConstructor;
import org.springframework.stereotype.Service;
import java.io.IOException;
import java.util.Map;
@Service
@RequiredArgsConstructor
public class GitPolicy {
private final GiteaClient gitea;
private final ProjectService projects;
private final GitTransport transport;
private final GitGatewayProperties properties;
private final ObjectMapper json;
public void ensure(Project project) {
identity(project);
gitea.ensureGitProtection(projects.repository(project), properties.getUpstreamUsername());
}
public void provision(Project project) {
identity(project);
gitea.ensureGitAccess(projects.repository(project), properties.getUpstreamUsername());
}
public void verify(Project project) {
identity(project);
gitea.verifyGitProtection(projects.repository(project), properties.getUpstreamUsername());
}
private void identity(Project project) {
try (GitTransport.Budget budget = transport.budget();
GitTransport.Response response =
transport.exchange(
"/api/v1/user",
"GET",
null,
-1,
Map.of("Accept", "application/json"),
budget)) {
if (response.status() != 200) throw invalid();
JsonNode user =
json.readTree(
GitIo.metadata(response.input(), properties.getMaxMetadataBytes()));
if (user == null
|| !properties.getUpstreamUsername().equals(user.path("login").asText())
|| !user.has("is_admin")
|| user.path("is_admin").asBoolean()) throw invalid();
GiteaClient.RemoteRepository remote = gitea.inspect(project.getGiteaRepoName());
if (!java.util.Objects.equals(project.getGiteaRepoId(), remote.id())
|| !project.getGiteaOwner().equals(remote.owner())
|| !project.getGiteaRepoName().equals(remote.name())
|| !remote.privateRepo()
|| remote.mirror()
|| remote.empty()
|| !"main".equals(remote.defaultBranch()))
throw new BusinessException(
503, "GIT_POLICY_UNAVAILABLE", "Git 仓库身份或 main 配置不兼容,请联系管理员");
budget.check();
} catch (IOException error) {
throw invalid();
}
}
private BusinessException invalid() {
return new BusinessException(503, "GIT_POLICY_UNAVAILABLE", "Git 必须配置身份匹配的独立非管理员技术账号");
}
}
@@ -0,0 +1,222 @@
package com.linkerbot.vibemake.service.git;
import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper;
import com.baomidou.mybatisplus.core.conditions.update.LambdaUpdateWrapper;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
import com.linkerbot.vibemake.bean.GitContracts.*;
import com.linkerbot.vibemake.bean.entity.Project;
import com.linkerbot.vibemake.bean.entity.ProjectGitToken;
import com.linkerbot.vibemake.bean.entity.User;
import com.linkerbot.vibemake.common.*;
import com.linkerbot.vibemake.service.ProjectService;
import com.linkerbot.vibemake.service.UserService;
import com.linkerbot.vibemake.service.config.GitGatewayProperties;
import com.linkerbot.vibemake.service.mapper.ProjectGitTokenMapper;
import com.linkerbot.vibemake.service.support.ProjectLocks;
import com.linkerbot.vibemake.service.support.ProjectSupport;
import lombok.RequiredArgsConstructor;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.stereotype.Service;
import java.security.SecureRandom;
import java.time.LocalDateTime;
import java.util.Base64;
@Service
@RequiredArgsConstructor
public class GitTokens {
private static final Logger log = LoggerFactory.getLogger(GitTokens.class);
private final ProjectGitTokenMapper tokens;
private final ProjectService projects;
private final UserService users;
private final ProjectLocks locks;
private final GitPolicy policy;
private final GitGatewayProperties properties;
private final SecureRandom random = new SecureRandom();
public void enabled() {
if (!properties.isEnabled())
throw new BusinessException(503, "GIT_UNAVAILABLE", "Git 接入尚未启用");
}
public IssuedToken issue(long userId, String code, TokenInput input) {
enabled();
return locks.serial(
code,
() -> {
Project project = projects.readyOwner(userId, code);
User user = users.require(userId);
requireRole(user);
LocalDateTime now = ProjectSupport.now();
long active =
tokens.selectCount(
new LambdaQueryWrapper<ProjectGitToken>()
.eq(ProjectGitToken::getProjectCode, code)
.eq(ProjectGitToken::getOwnerUserId, userId)
.eq(
ProjectGitToken::getSessionVersion,
user.getSessionVersion())
.isNull(ProjectGitToken::getRevokedAt)
.gt(ProjectGitToken::getExpiresAt, now));
if (active >= properties.getMaxActiveTokens())
throw BusinessException.conflict("有效 Git 凭据已达上限,请先撤销旧凭据");
policy.provision(project);
if (input.scope() == Scope.READ_WRITE) policy.ensure(project);
byte[] bytes = new byte[32];
random.nextBytes(bytes);
String secret =
"vbg_" + Base64.getUrlEncoder().withoutPadding().encodeToString(bytes);
ProjectGitToken token = new ProjectGitToken();
token.setTokenCode(Codes.next());
token.setProjectCode(code);
token.setOwnerUserId(userId);
token.setTokenName(input.name().trim());
token.setTokenHash(SecurityHashes.sha256(secret));
token.setPermission(input.scope().name());
token.setSessionVersion(user.getSessionVersion());
token.setExpiresAt(
now.plusDays(
input.expiresInDays() == null ? 90 : input.expiresInDays()));
token.setCreatedAt(now);
token.setUpdatedAt(now);
tokens.insert(token);
log.info(
"Git 凭据创建 projectCode={} userCode={} tokenCode={} permission={}",
code,
user.getUserCode(),
token.getTokenCode(),
token.getPermission());
return new IssuedToken(view(token, user.getSessionVersion()), secret);
});
}
public PageResponse<TokenView> list(long userId, String code, long page, long size) {
projects.owner(userId, code);
if (page < 1 || page > 100000 || size < 1 || size > 100)
throw BusinessException.badRequest("分页参数不合法");
long version = users.require(userId).getSessionVersion();
Page<ProjectGitToken> result =
tokens.selectPage(
new Page<>(page, size),
new LambdaQueryWrapper<ProjectGitToken>()
.eq(ProjectGitToken::getProjectCode, code)
.eq(ProjectGitToken::getOwnerUserId, userId)
.orderByDesc(ProjectGitToken::getCreatedAt)
.orderByDesc(ProjectGitToken::getTokenCode));
return new PageResponse<>(
result.getRecords().stream().map(token -> view(token, version)).toList(),
page,
size,
result.getTotal());
}
public void revoke(long userId, String code, String tokenCode) {
projects.owner(userId, code);
ProjectGitToken token =
tokens.selectOne(
new LambdaQueryWrapper<ProjectGitToken>()
.eq(ProjectGitToken::getTokenCode, tokenCode)
.eq(ProjectGitToken::getProjectCode, code)
.eq(ProjectGitToken::getOwnerUserId, userId));
if (token == null) throw BusinessException.notFound();
LocalDateTime now = ProjectSupport.now();
tokens.update(
null,
new LambdaUpdateWrapper<ProjectGitToken>()
.eq(ProjectGitToken::getTokenCode, tokenCode)
.isNull(ProjectGitToken::getRevokedAt)
.set(ProjectGitToken::getRevokedAt, now)
.set(ProjectGitToken::getUpdatedAt, now));
log.info("Git 凭据撤销 projectCode={} tokenCode={}", code, tokenCode);
}
public GitAccess authenticate(String username, String secret, String code) {
enabled();
if (secret == null || !secret.matches("vbg_[A-Za-z0-9_-]{43}")) throw unauthorized();
ProjectGitToken token =
tokens.selectOne(
new LambdaQueryWrapper<ProjectGitToken>()
.eq(ProjectGitToken::getTokenHash, SecurityHashes.sha256(secret)));
GitAccess access = validate(token, code);
if (!access.userCode().equals(username)) throw unauthorized();
touch(token);
return access;
}
public GitAccess recheck(GitAccess access) {
enabled();
ProjectGitToken token =
tokens.selectOne(
new LambdaQueryWrapper<ProjectGitToken>()
.eq(ProjectGitToken::getTokenCode, access.tokenCode()));
GitAccess current = validate(token, access.projectCode());
if (current.userId() != access.userId()) throw unauthorized();
return current;
}
private GitAccess validate(ProjectGitToken token, String code) {
if (token == null
|| token.getRevokedAt() != null
|| !token.getExpiresAt().isAfter(ProjectSupport.now())) throw unauthorized();
if (!token.getProjectCode().equals(code)) throw BusinessException.notFound();
User user = users.require(token.getOwnerUserId());
requireRole(user);
if (!user.getSessionVersion().equals(token.getSessionVersion())) throw unauthorized();
projects.readyOwner(user.getId(), code);
if (!"READ_ONLY".equals(token.getPermission())
&& !"READ_WRITE".equals(token.getPermission())) throw unauthorized();
return new GitAccess(
user.getId(),
user.getUserCode(),
token.getTokenCode(),
code,
"READ_WRITE".equals(token.getPermission()));
}
private void touch(ProjectGitToken token) {
LocalDateTime now = ProjectSupport.now();
tokens.update(
null,
new LambdaUpdateWrapper<ProjectGitToken>()
.eq(ProjectGitToken::getTokenCode, token.getTokenCode())
.and(
query ->
query.isNull(ProjectGitToken::getLastUsedAt)
.or()
.lt(
ProjectGitToken::getLastUsedAt,
now.minusMinutes(5)))
.set(ProjectGitToken::getLastUsedAt, now));
}
private void requireRole(User user) {
if (!"USER".equals(user.getRole()) && !"ADMIN".equals(user.getRole())) throw unauthorized();
}
private TokenView view(ProjectGitToken token, long version) {
String status =
token.getRevokedAt() != null || token.getSessionVersion() != version
? "REVOKED"
: token.getExpiresAt().isAfter(ProjectSupport.now()) ? "ACTIVE" : "EXPIRED";
return new TokenView(
token.getTokenCode(),
token.getTokenName(),
Scope.valueOf(token.getPermission()),
status,
token.getCreatedAt(),
token.getExpiresAt(),
token.getRevokedAt(),
token.getLastUsedAt());
}
public static BusinessException unauthorized() {
return new BusinessException(401, "GIT_UNAUTHORIZED", "请使用网站用户编码和有效的项目 Git Token");
}
public static void requireWrite(GitAccess access) {
if (!access.writable()) throw new BusinessException(403, "GIT_READ_ONLY", "该 Git 凭据仅允许读取");
}
}
@@ -0,0 +1,242 @@
package com.linkerbot.vibemake.service.git;
import com.linkerbot.vibemake.common.BusinessException;
import com.linkerbot.vibemake.service.config.GitGatewayProperties;
import com.linkerbot.vibemake.service.config.VibemakeProperties;
import jakarta.annotation.PreDestroy;
import org.springframework.stereotype.Component;
import java.io.*;
import java.net.URI;
import java.net.URLEncoder;
import java.net.http.*;
import java.nio.charset.StandardCharsets;
import java.util.*;
import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicBoolean;
/** 仅访问配置的 Gitea;请求正文与响应正文都不落盘、不完整缓冲。 */
@Component
public class GitTransport implements AutoCloseable {
private final GitGatewayProperties properties;
private final VibemakeProperties config;
private final HttpClient http;
private final ScheduledThreadPoolExecutor timers =
new ScheduledThreadPoolExecutor(
2, Thread.ofPlatform().daemon().name("git-budget-", 0).factory());
public GitTransport(GitGatewayProperties properties, VibemakeProperties config) {
timers.setRemoveOnCancelPolicy(true);
this.properties = properties;
this.config = config;
this.http =
HttpClient.newBuilder()
.connectTimeout(config.getGitea().getConnectTimeout())
.followRedirects(HttpClient.Redirect.NEVER)
.build();
}
public final class Budget implements AutoCloseable {
private final long deadline = System.nanoTime() + properties.getTimeout().toNanos();
private final Set<AutoCloseable> resources = ConcurrentHashMap.newKeySet();
private final Set<AutoCloseable> timeoutResources = ConcurrentHashMap.newKeySet();
private final AtomicBoolean expired = new AtomicBoolean();
private final ScheduledFuture<?> timer;
private Budget() {
timer =
timers.schedule(
() -> {
expired.set(true);
for (AutoCloseable resource : resources)
Thread.startVirtualThread(() -> closeQuietly(resource));
for (AutoCloseable resource : timeoutResources)
Thread.startVirtualThread(() -> closeQuietly(resource));
},
properties.getTimeout().toNanos(),
TimeUnit.NANOSECONDS);
}
public void check() {
if (expired.get() || System.nanoTime() - deadline >= 0) throw timeout();
}
public void register(AutoCloseable resource) {
resources.add(resource);
if (expired.get()) {
closeQuietly(resource);
throw timeout();
}
}
public void onTimeout(AutoCloseable resource) {
timeoutResources.add(resource);
if (expired.get()) {
closeQuietly(resource);
throw timeout();
}
}
public java.time.Duration remaining() {
check();
return java.time.Duration.ofNanos(Math.max(1, deadline - System.nanoTime()));
}
@Override
public void close() {
timer.cancel(false);
resources.forEach(GitTransport::closeQuietly);
resources.clear();
timeoutResources.clear();
}
}
public record Response(int status, HttpHeaders headers, InputStream input)
implements AutoCloseable {
@Override
public void close() {
closeQuietly(input);
}
}
public Budget budget() {
return new Budget();
}
private static void closeQuietly(AutoCloseable resource) {
try {
resource.close();
} catch (Exception ignored) {
}
}
public URI base() {
URI base = URI.create(config.getGitea().getBaseUrl().replaceAll("/+$", ""));
if (!("http".equals(base.getScheme()) || "https".equals(base.getScheme()))
|| base.getHost() == null
|| base.getRawUserInfo() != null
|| base.getRawQuery() != null
|| base.getRawFragment() != null)
throw new BusinessException(503, "GIT_UNAVAILABLE", "Git 上游地址配置不合法");
return base;
}
public static String encode(String value) {
return URLEncoder.encode(value, StandardCharsets.UTF_8).replace("+", "%20");
}
public String repositoryPath(com.linkerbot.vibemake.bean.entity.Project project) {
return "/"
+ encode(project.getGiteaOwner())
+ "/"
+ encode(project.getGiteaRepoName())
+ ".git";
}
public Response exchange(
String path,
String method,
InputStream body,
long length,
Map<String, String> headers,
Budget budget) {
return exchange(path, method, body, length, headers, budget, null);
}
public Response exchange(
String path,
String method,
InputStream body,
long length,
Map<String, String> headers,
Budget budget,
Map<String, String> actionHeaders) {
budget.check();
if (!path.startsWith("/")
|| path.startsWith("//")
|| path.contains("\\")
|| path.contains("#")
|| path.contains("/../")
|| path.contains("/./")) throw GitIo.upstreamInvalid();
URI destination = URI.create(base().toString() + path);
String basic =
Base64.getEncoder()
.encodeToString(
(properties.getUpstreamUsername()
+ ":"
+ properties.getUpstreamToken())
.getBytes(StandardCharsets.UTF_8));
HttpRequest.Builder builder =
HttpRequest.newBuilder(destination)
.timeout(budget.remaining())
.header("Authorization", "Basic " + basic);
headers.forEach(builder::setHeader);
if (actionHeaders != null) actionHeaders.forEach(builder::setHeader);
HttpRequest.BodyPublisher publisher = HttpRequest.BodyPublishers.noBody();
if (body != null) {
budget.register(body);
publisher = HttpRequest.BodyPublishers.ofInputStream(() -> body);
if (length > 0) publisher = HttpRequest.BodyPublishers.fromPublisher(publisher, length);
}
CompletableFuture<HttpResponse<InputStream>> future =
http.sendAsync(
builder.method(method, publisher).build(),
HttpResponse.BodyHandlers.ofInputStream());
budget.register(() -> future.cancel(true));
try {
HttpResponse<InputStream> response =
future.get(budget.remaining().toNanos(), TimeUnit.NANOSECONDS);
budget.register(response.body());
budget.check();
if (response.statusCode() >= 300 && response.statusCode() < 400)
throw GitIo.upstreamInvalid();
return new Response(
response.statusCode(), response.headers(), checked(response.body(), budget));
} catch (InterruptedException error) {
Thread.currentThread().interrupt();
throw timeout();
} catch (TimeoutException error) {
future.cancel(true);
throw timeout();
} catch (ExecutionException | CancellationException error) {
budget.check();
throw GitIo.upstreamInvalid();
}
}
private InputStream checked(InputStream input, Budget budget) {
return new FilterInputStream(input) {
@Override
public int read() throws IOException {
byte[] b = new byte[1];
return read(b, 0, 1) < 0 ? -1 : b[0] & 255;
}
@Override
public int read(byte[] bytes, int offset, int length) throws IOException {
budget.check();
try {
int size = super.read(bytes, offset, length);
budget.check();
return size;
} catch (IOException error) {
budget.check();
throw error;
}
}
};
}
public static BusinessException timeout() {
return new BusinessException(504, "GIT_TIMEOUT", "Git/LFS 请求超时;涉及推送请先刷新工作区核对结果");
}
@PreDestroy
@Override
public void close() {
timers.shutdownNow();
http.shutdownNow();
}
}
@@ -0,0 +1,438 @@
package com.linkerbot.vibemake.service.git;
import com.fasterxml.jackson.databind.*;
import com.fasterxml.jackson.databind.node.*;
import com.linkerbot.vibemake.bean.entity.Project;
import com.linkerbot.vibemake.common.*;
import com.linkerbot.vibemake.common.state.TransientStateStore;
import com.linkerbot.vibemake.service.ProjectService;
import com.linkerbot.vibemake.service.config.GitGatewayProperties;
import jakarta.servlet.http.*;
import lombok.RequiredArgsConstructor;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.stereotype.Service;
import java.io.*;
import java.net.URI;
import java.security.*;
import java.time.*;
import java.util.*;
@Service
@RequiredArgsConstructor
public class LfsGateway {
public static final String MEDIA = "application/vnd.git-lfs+json";
private static final Logger log = LoggerFactory.getLogger(LfsGateway.class);
private final GitTokens tokens;
private final GitTransport transport;
private final GitGatewayProperties properties;
private final ProjectService projects;
private final TransientStateStore state;
private final ObjectMapper json;
private final GitLimits limits;
private final SecureRandom random = new SecureRandom();
public record Action(
GitAccess access,
String hash,
String oid,
long size,
String method,
String path,
Map<String, String> headers,
Instant expiresAt) {}
public Action authenticate(String code, String projectCode, String bearer, String method) {
if (!code.matches("[a-f0-9]{32}") || bearer == null || !bearer.matches("[A-Za-z0-9_-]{43}"))
throw GitTokens.unauthorized();
String stored = state.get("git:lfs-action:" + code);
try {
Action action = stored == null ? null : json.readValue(stored, Action.class);
if (action == null
|| !MessageDigest.isEqual(
action.hash().getBytes(java.nio.charset.StandardCharsets.US_ASCII),
SecurityHashes.sha256(bearer)
.getBytes(java.nio.charset.StandardCharsets.US_ASCII))
|| !action.expiresAt().isAfter(Instant.now())
|| !action.access().projectCode().equals(projectCode)
|| !action.method().equals(method)) throw GitTokens.unauthorized();
GitAccess access = tokens.recheck(action.access());
if (!"GET".equals(method)) GitTokens.requireWrite(access);
return action;
} catch (com.fasterxml.jackson.core.JsonProcessingException error) {
throw GitTokens.unauthorized();
}
}
public void serve(
GitAccess access,
String endpoint,
HttpServletRequest request,
HttpServletResponse response)
throws IOException {
try (AutoCloseable permit = limits.acquire(false, true);
GitTransport.Budget budget = transport.budget()) {
budget.register(request.getInputStream());
budget.onTimeout(
() -> {
if (response.isCommitted()) response.getOutputStream().close();
});
if ("locks/verify".equals(endpoint) && "POST".equals(request.getMethod())) {
GitIo.metadata(
GitGateway.decoded(request, properties.getMaxMetadataBytes(), budget),
properties.getMaxMetadataBytes());
ObjectNode emptyLocks = json.createObjectNode();
emptyLocks.set("ours", json.createArrayNode());
emptyLocks.set("theirs", json.createArrayNode());
write(response, emptyLocks);
return;
}
if (!"objects/batch".equals(endpoint) || !"POST".equals(request.getMethod()))
throw new BusinessException(501, "LFS_LOCKS_UNSUPPORTED", "暂不支持 LFS 文件锁或该协议接口");
GitGateway.requireContentType(request, MEDIA);
JsonNode input = clientJson(request, budget);
if (input == null || !input.isObject()) throw invalid();
String operation = input.path("operation").asText();
if (!("upload".equals(operation) || "download".equals(operation))) throw invalid();
if ("upload".equals(operation)) GitTokens.requireWrite(access);
JsonNode objects = input.path("objects");
if (!objects.isArray()
|| objects.isEmpty()
|| objects.size() > properties.getMaxBatchObjects()) throw invalid();
Map<String, Long> requested = new HashMap<>();
for (JsonNode object : objects) {
String oid = object.path("oid").asText();
long size = object.path("size").asLong(-1);
if (!oid.matches("[a-f0-9]{64}")
|| !object.path("size").isIntegralNumber()
|| !object.path("size").canConvertToLong()
|| size < 0) throw invalid();
if (size > properties.getMaxLfsObjectBytes())
throw new BusinessException(413, "LFS_TOO_LARGE", "LFS 对象超过大小限制");
if (requested.put(oid, size) != null) throw invalid();
}
// 明确协商 Basic Transfer;不向上游传入客户端自定义 transfer、ref 或 action。
ObjectNode sanitized = json.createObjectNode().put("operation", operation);
sanitized.set("objects", objects);
sanitized.putArray("transfers").add("basic");
Project project = projects.readyOwner(access.userId(), access.projectCode());
byte[] body = json.writeValueAsBytes(sanitized);
try (GitTransport.Response upstream =
transport.exchange(
transport.repositoryPath(project) + "/info/lfs/objects/batch",
"POST",
new ByteArrayInputStream(body),
body.length,
Map.of("Accept", MEDIA, "Content-Type", MEDIA),
budget)) {
GitGateway.requireSuccess(upstream, MEDIA);
JsonNode result =
json.readTree(
GitIo.metadata(upstream.input(), properties.getMaxMetadataBytes()));
if (result == null
|| !result.path("objects").isArray()
|| result.path("objects").size() != requested.size()
|| !"basic".equals(result.path("transfer").asText("basic")))
throw GitIo.upstreamInvalid();
ObjectNode output = json.createObjectNode().put("transfer", "basic");
ArrayNode results = output.putArray("objects");
Set<String> seen = new HashSet<>();
for (JsonNode object : result.path("objects")) {
String oid = object.path("oid").asText();
Long size = requested.get(oid);
if (size == null
|| !seen.add(oid)
|| !object.path("size").isIntegralNumber()
|| !object.path("size").canConvertToLong()
|| object.path("size").asLong(-1) != size)
throw GitIo.upstreamInvalid();
ObjectNode target =
results.addObject()
.put("oid", oid)
.put("size", size)
.put("authenticated", true);
if (object.has("error")) {
// 上游错误正文可能含内部地址,仅返回固定文本与错误码。
target.putObject("error")
.put("code", object.path("error").path("code").asInt(502))
.put("message", "LFS 对象不可用");
continue;
}
JsonNode actions = object.path("actions");
if (actions.isObject()) {
ObjectNode rewritten = target.putObject("actions");
Iterator<String> names = actions.fieldNames();
while (names.hasNext()) {
String name = names.next();
if (!(name.equals(operation)
|| name.equals("verify") && operation.equals("upload")))
throw GitIo.upstreamInvalid();
rewritten.set(
name,
issue(access, project, oid, size, name, actions.get(name)));
}
}
}
write(response, output);
log.info(
"LFS batch 完成 projectCode={} userCode={} tokenCode={} operation={}"
+ " objects={}",
access.projectCode(),
access.userCode(),
access.tokenCode(),
operation,
requested.size());
}
} catch (BusinessException | IOException error) {
throw error;
} catch (Exception error) {
throw GitIo.upstreamInvalid();
}
}
private ObjectNode issue(
GitAccess access, Project project, String oid, long size, String name, JsonNode source)
throws IOException {
String expectedPath =
transport.base().getRawPath()
+ transport.repositoryPath(project)
+ "/info/lfs/"
+ (name.equals("verify")
? "verify"
: "objects/" + oid + (name.equals("upload") ? "/" + size : ""));
URI uri;
try {
uri = URI.create(source.path("href").asText());
} catch (IllegalArgumentException error) {
throw GitIo.upstreamInvalid();
}
URI base = transport.base();
if (!Objects.equals(base.getScheme(), uri.getScheme())
|| !Objects.equals(base.getHost(), uri.getHost())
|| base.getPort() != uri.getPort()
|| uri.getRawUserInfo() != null
|| uri.getRawQuery() != null
|| uri.getRawFragment() != null
|| !expectedPath.equals(uri.getRawPath())) throw GitIo.upstreamInvalid();
Map<String, String> headers = new HashMap<>();
JsonNode sourceHeaders = source.path("header");
if (sourceHeaders.isObject()) {
Iterator<Map.Entry<String, JsonNode>> fields = sourceHeaders.fields();
while (fields.hasNext()) {
Map.Entry<String, JsonNode> field = fields.next();
// Gitea 原生上传 action 会给出此 hop-by-hop 提示。网关使用自己的
// BodyPublisher 决定传输方式,不能把上游的 Transfer-Encoding 直接设置到 JDK 客户端。
if (name.equals("upload")
&& field.getKey().equalsIgnoreCase("Transfer-Encoding")
&& field.getValue().isTextual()
&& "chunked".equalsIgnoreCase(field.getValue().asText())) continue;
if (!Set.of("authorization", "accept", "content-type")
.contains(field.getKey().toLowerCase(Locale.ROOT))
|| !field.getValue().isTextual()
|| field.getValue().asText().length() > 8192
|| field.getValue().asText().contains("\n")
|| field.getValue().asText().contains("\r")) throw GitIo.upstreamInvalid();
headers.put(field.getKey(), field.getValue().asText());
}
}
long ttl = 900;
if (source.has("expires_in")) ttl = Math.min(ttl, source.path("expires_in").asLong(0));
if (source.has("expires_at")) {
try {
ttl =
Math.min(
ttl,
Duration.between(
Instant.now(),
Instant.parse(source.path("expires_at").asText()))
.getSeconds());
} catch (java.time.format.DateTimeParseException error) {
throw GitIo.upstreamInvalid();
}
}
if (ttl < 1) throw GitIo.upstreamInvalid();
byte[] bytes = new byte[32];
random.nextBytes(bytes);
String secret = Base64.getUrlEncoder().withoutPadding().encodeToString(bytes);
String code = Codes.next();
String method = name.equals("download") ? "GET" : name.equals("upload") ? "PUT" : "POST";
String path =
transport.repositoryPath(project)
+ "/info/lfs/"
+ (name.equals("verify")
? "verify"
: "objects/" + oid + (name.equals("upload") ? "/" + size : ""));
Action action =
new Action(
access,
SecurityHashes.sha256(secret),
oid,
size,
method,
path,
headers,
Instant.now().plusSeconds(ttl));
if (!state.putIfAbsent(
"git:lfs-action:" + code, json.writeValueAsString(action), Duration.ofSeconds(ttl)))
throw GitIo.upstreamInvalid();
ObjectNode result =
json.createObjectNode()
.put(
"href",
properties.getPublicBaseUrl()
+ "/projects/"
+ access.projectCode()
+ ".git/info/lfs/actions/"
+ code)
.put("expires_in", ttl);
result.putObject("header").put("Authorization", "Bearer " + secret);
return result;
}
public void action(Action action, HttpServletRequest request, HttpServletResponse response)
throws IOException {
long started = System.nanoTime();
java.util.concurrent.atomic.AtomicLong transferred =
new java.util.concurrent.atomic.AtomicLong();
Project project =
projects.readyOwner(action.access().userId(), action.access().projectCode());
String expectedPath =
transport.repositoryPath(project)
+ "/info/lfs/"
+ (action.method().equals("POST")
? "verify"
: "objects/"
+ action.oid()
+ (action.method().equals("PUT")
? "/" + action.size()
: ""));
if (!expectedPath.equals(action.path())) throw GitTokens.unauthorized();
try (AutoCloseable permit = limits.acquire(false, true);
GitTransport.Budget budget = transport.budget()) {
budget.register(request.getInputStream());
budget.onTimeout(
() -> {
if (response.isCommitted()) response.getOutputStream().close();
});
InputStream body = null;
long length = -1;
String type = "application/octet-stream";
MessageDigest digest = MessageDigest.getInstance("SHA-256");
if (action.method().equals("PUT")) {
if (request.getContentLengthLong() >= 0
&& request.getContentLengthLong() != action.size()) throw invalid();
if (request.getHeader("Content-Encoding") != null) throw invalid();
body =
new DigestInputStream(
GitIo.counted(
GitIo.limited(request.getInputStream(), action.size()),
transferred),
digest);
length = action.size();
} else if (action.method().equals("POST")) {
JsonNode verification = clientJson(request, budget);
if (verification == null
|| !action.oid().equals(verification.path("oid").asText())
|| !verification.path("size").isIntegralNumber()
|| !verification.path("size").canConvertToLong()
|| verification.path("size").asLong(-1) != action.size()) throw invalid();
byte[] bytes =
json.writeValueAsBytes(Map.of("oid", action.oid(), "size", action.size()));
body = new ByteArrayInputStream(bytes);
length = bytes.length;
transferred.set(bytes.length);
type = MEDIA;
}
try (GitTransport.Response upstream =
transport.exchange(
action.path(),
action.method(),
body,
length,
Map.of("Accept", MEDIA, "Content-Type", type),
budget,
action.headers())) {
if (upstream.status() < 200 || upstream.status() >= 300)
throw GitIo.upstreamInvalid();
if (action.method().equals("GET")) {
long size = upstream.headers().firstValueAsLong("Content-Length").orElse(-1);
if (size >= 0 && size != action.size()) throw GitIo.upstreamInvalid();
response.setContentType("application/octet-stream");
response.setContentLengthLong(action.size());
long copied =
GitGateway.copy(
new DigestInputStream(upstream.input(), digest),
response,
action.size(),
budget,
transferred);
if (copied != action.size()
|| !HexFormat.of().formatHex(digest.digest()).equals(action.oid()))
throw GitIo.upstreamInvalid();
} else {
GitIo.metadata(upstream.input(), properties.getMaxMetadataBytes());
if (action.method().equals("PUT")
&& !HexFormat.of().formatHex(digest.digest()).equals(action.oid()))
throw invalid();
response.setStatus(200);
response.setContentType(MEDIA);
response.setContentLength(0);
}
log.info(
"LFS action 完成 projectCode={} userCode={} tokenCode={} method={} oid={}"
+ " bytes={} durationMs={}",
action.access().projectCode(),
action.access().userCode(),
action.access().tokenCode(),
action.method(),
action.oid(),
transferred.get(),
(System.nanoTime() - started) / 1000000);
}
} catch (BusinessException | IOException error) {
log.warn(
"LFS action 失败或取消 projectCode={} tokenCode={} method={} oid={} bytes={}"
+ " errorType={}",
action.access().projectCode(),
action.access().tokenCode(),
action.method(),
action.oid(),
transferred.get(),
error.getClass().getSimpleName());
throw error;
} catch (Exception error) {
throw GitIo.upstreamInvalid();
}
}
private JsonNode clientJson(HttpServletRequest request, GitTransport.Budget budget)
throws IOException {
GitGateway.requireContentType(request, MEDIA);
byte[] bytes =
GitIo.metadata(
GitGateway.decoded(request, properties.getMaxMetadataBytes(), budget),
properties.getMaxMetadataBytes());
try {
return json.reader()
.with(DeserializationFeature.FAIL_ON_TRAILING_TOKENS)
.readTree(bytes);
} catch (com.fasterxml.jackson.core.JsonProcessingException error) {
throw invalid();
}
}
private BusinessException invalid() {
return BusinessException.badRequest("LFS 请求的操作、OID、大小或 JSON 不合法");
}
private void write(HttpServletResponse response, JsonNode value) throws IOException {
response.setContentType(MEDIA);
byte[] bytes = json.writeValueAsBytes(value);
response.setContentLength(bytes.length);
response.getOutputStream().write(bytes);
}
}
@@ -10,8 +10,19 @@ import java.util.List;
import java.time.Duration;
public interface GiteaClient {
default void ensureGitAccess(Repository repository, String username) {
throw new UnsupportedOperationException("未实现 Git 技术账号仓库授权");
}
default String authenticatedUsername() { throw new UnsupportedOperationException("未实现服务账号查询"); }
default void ensureGitProtection(Repository repository, String username) {
throw new UnsupportedOperationException("未实现 Git main 保护初始化");
}
default void verifyGitProtection(Repository repository, String username) {
throw new UnsupportedOperationException("未实现 Git main 保护核验");
}
record Repository(long id, String owner, String name) {}
record StoredFile(String path, String blobSha, byte[] bytes) {}
@@ -49,6 +49,51 @@ public class RestGiteaClient implements GiteaClient {
return "/repos/" + encode(repo.owner()) + "/" + encode(repo.name());
}
@Override
public void ensureGitAccess(Repository repo, String username) {
String path = repoPath(repo) + "/collaborators/" + encode(username);
String permission = "none";
try { permission = parse(request("GET", path + "/permission", null).body(), false).path("permission").asText(); }
catch (GiteaFailure error) { if (error.status() != 404) throw error; }
if ("admin".equals(permission) || "owner".equals(permission)) throw gitPolicyFailure();
if (!"write".equals(permission)) request("PUT", path, Map.of("permission", "write"));
JsonNode current = parse(request("GET", path + "/permission", null).body(), false);
if (!"write".equals(current.path("permission").asText())) throw gitPolicyFailure();
}
@Override
public void ensureGitProtection(Repository repo, String username) {
JsonNode rules = parse(request("GET", repoPath(repo) + "/branch_protections", null).body(), false);
if (rules.isArray() && rules.isEmpty()) {
request("POST", repoPath(repo) + "/branch_protections", Map.of(
"branch_name", "main", "rule_name", "main", "enable_push", true, "enable_force_push", false));
}
verifyGitProtection(repo, username);
}
@Override
public void verifyGitProtection(Repository repo, String username) {
JsonNode permission = parse(request("GET", repoPath(repo) + "/collaborators/"
+ encode(username) + "/permission", null).body(), false);
if (!"write".equals(permission.path("permission").asText())) throw gitPolicyFailure();
JsonNode rules = parse(request("GET", repoPath(repo) + "/branch_protections", null).body(), false);
if (!rules.isArray() || rules.size() != 1) throw gitPolicyFailure();
JsonNode rule = rules.get(0);
String name = rule.path("rule_name").asText(rule.path("branch_name").asText());
if (!"main".equals(name) || !rule.path("enable_push").asBoolean()
|| !rule.has("enable_force_push") || rule.path("enable_force_push").asBoolean()) throw gitPolicyFailure();
if (rule.path("enable_push_whitelist").asBoolean()) {
boolean allowed = false;
for (JsonNode member : rule.path("push_whitelist_usernames"))
if (username.equals(member.asText())) allowed = true;
if (!allowed) throw gitPolicyFailure();
}
}
private BusinessException gitPolicyFailure() {
return new BusinessException(503, "GIT_POLICY_UNAVAILABLE", "main 保护规则或 Git 技术账号权限不兼容,请联系管理员");
}
private String filePath(String path) {
return Arrays.stream(path.split("/"))
.map(RestGiteaClient::encode)
@@ -78,6 +78,11 @@ public class ProjectLocks implements AutoCloseable {
}
public <T> T write(String code, Supplier<T> action) {
return write(code, () -> {}, action);
}
/** prepare 在写保护检查后、任何远端写入前执行,用于原子持久化协议写入凭据。 */
public <T> T write(String code, Runnable prepare, Supplier<T> action) {
return serial(
code,
() -> {
@@ -96,6 +101,7 @@ public class ProjectLocks implements AutoCloseable {
markUncertain(code);
throw blocked();
}
prepare.run();
state(code, "WRITING");
writing.get().add(code);
try {
@@ -6,21 +6,26 @@ import com.linkerbot.vibemake.bean.entity.Project;
import com.linkerbot.vibemake.bean.entity.RepositoryWriteOperation;
import com.linkerbot.vibemake.common.*;
import com.linkerbot.vibemake.service.ProjectService;
import com.linkerbot.vibemake.service.git.GitAccess;
import com.linkerbot.vibemake.service.git.GitIo;
import com.linkerbot.vibemake.service.gitea.*;
import com.linkerbot.vibemake.service.mapper.ProjectMapper;
import com.linkerbot.vibemake.service.mapper.RepositoryWriteOperationMapper;
import com.linkerbot.vibemake.service.support.ProjectActors;
import com.linkerbot.vibemake.service.support.ProjectLocks;
import com.linkerbot.vibemake.service.support.ProjectSupport;
import lombok.RequiredArgsConstructor;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.stereotype.Service;
import org.springframework.transaction.support.TransactionTemplate;
import java.time.LocalDateTime;
import java.util.List;
import java.util.Objects;
import java.util.function.Function;
import java.util.function.Supplier;
/** 数据库凭据先于远端写入落盘。未知结果只读取核对,绝不自动重发写请求。 */
@Service
@@ -37,61 +42,87 @@ public class RepositoryWrites {
public record Request(String code, String digest) {}
public CommitView execute(long userId, String projectCode, String key, String digest,
Function<Request, CommitView> action) {
String operationCode = key == null ? Codes.next() : key.replace("-", "").toLowerCase(java.util.Locale.ROOT);
public CommitView execute(
long userId,
String projectCode,
String key,
String digest,
Function<Request, CommitView> action) {
String operationCode =
key == null
? Codes.next()
: key.replace("-", "").toLowerCase(java.util.Locale.ROOT);
if (!operationCode.matches("[a-f0-9]{32}"))
throw BusinessException.badRequest("操作编码必须为 UUID 或 32 位十六进制编码");
return locks.serial(projectCode, () -> {
projects.readyOwner(userId, projectCode); // 重放也必须重新鉴权。
RepositoryWriteOperation previous = operations.selectById(operationCode);
if (previous != null) {
if (!projectCode.equals(previous.getProjectCode()) || !digest.equals(previous.getRequestDigest()))
throw new BusinessException(409, "IDEMPOTENCY_KEY_REUSED", "操作编码已用于不同请求,请勿复用");
return replay(previous);
}
RepositoryWriteOperation pending = active(projectCode);
if (pending != null)
throw pending(pending);
try {
CommitView result = action.apply(new Request(operationCode, digest));
RepositoryWriteOperation operation = operations.selectById(operationCode);
if (operation == null) throw new IllegalStateException("仓库写入缺少持久化凭据");
complete(operation, result.commitSha(), "SUCCEEDED");
return result;
} catch (RuntimeException error) {
RepositoryWriteOperation operation = operations.selectById(operationCode);
if (operation == null) throw error; // 参数校验、读取失败发生在任何远端写入之前。
if (!projectCode.equals(operation.getProjectCode()) || !digest.equals(operation.getRequestDigest()))
throw new BusinessException(409, "IDEMPOTENCY_KEY_REUSED", "操作编码已用于不同请求,请勿复用");
boolean definite = error instanceof GiteaFailure remote && !remote.uncertain();
if (definite) {
BusinessException business = (BusinessException) error;
operation.setStatus("FAILED");
operation.setFailureStatus(business.status());
operation.setLastErrorCode(business.code());
save(operation);
throw error;
}
locks.markUncertain(projectCode);
operation.setLastErrorCode(error instanceof BusinessException business
? business.code() : "WRITE_RESULT_UNKNOWN");
save(operation);
log.warn("仓库写入转后台核对 projectCode={} operationCode={} errorCode={} errorType={}",
projectCode, operationCode, operation.getLastErrorCode(), error.getClass().getSimpleName());
if (!(error instanceof BusinessException)) SafeExceptionLog.write(log, "仓库提交结果未知", error);
throw error;
}
});
return locks.serial(
projectCode,
() -> {
projects.readyOwner(userId, projectCode); // 重放也必须重新鉴权。
RepositoryWriteOperation previous = operations.selectById(operationCode);
if (previous != null) {
if (!projectCode.equals(previous.getProjectCode())
|| !digest.equals(previous.getRequestDigest()))
throw new BusinessException(
409, "IDEMPOTENCY_KEY_REUSED", "操作编码已用于不同请求,请勿复用");
return replay(previous);
}
RepositoryWriteOperation pending = active(projectCode);
if (pending != null) throw pending(pending);
try {
CommitView result = action.apply(new Request(operationCode, digest));
RepositoryWriteOperation operation = operations.selectById(operationCode);
if (operation == null) throw new IllegalStateException("仓库写入缺少持久化凭据");
complete(operation, result.commitSha(), "SUCCEEDED");
return result;
} catch (RuntimeException error) {
RepositoryWriteOperation operation = operations.selectById(operationCode);
if (operation == null) throw error; // 参数校验、读取失败发生在任何远端写入之前。
if (!projectCode.equals(operation.getProjectCode())
|| !digest.equals(operation.getRequestDigest()))
throw new BusinessException(
409, "IDEMPOTENCY_KEY_REUSED", "操作编码已用于不同请求,请勿复用");
boolean definite =
error instanceof GiteaFailure remote && !remote.uncertain();
if (definite) {
BusinessException business = (BusinessException) error;
operation.setStatus("FAILED");
operation.setFailureStatus(business.status());
operation.setLastErrorCode(business.code());
save(operation);
throw error;
}
locks.markUncertain(projectCode);
operation.setLastErrorCode(
error instanceof BusinessException business
? business.code()
: "WRITE_RESULT_UNKNOWN");
save(operation);
log.warn(
"仓库写入转后台核对 projectCode={} operationCode={} errorCode={}"
+ " errorType={}",
projectCode,
operationCode,
operation.getLastErrorCode(),
error.getClass().getSimpleName());
if (!(error instanceof BusinessException))
SafeExceptionLog.write(log, "仓库提交结果未知", error);
throw error;
}
});
}
/** 必须在 ProjectLocks.write 内调用;此处是唯一一次发起远端写请求的位置。 */
public CommitView commit(Request request, Project project, String baseSha,
List<GiteaClient.Change> changes, String message,
ProjectActors.Author author) {
public CommitView commit(
Request request,
Project project,
String baseSha,
List<GiteaClient.Change> changes,
String message,
ProjectActors.Author author) {
GiteaClient.Repository repository = projects.repository(project);
RepositoryWriteOperation operation = new RepositoryWriteOperation();
operation.setOperationCode(request.code());
operation.setOperationKind("WEB_COMMIT");
operation.setProjectCode(project.getProjectCode());
operation.setRepositoryId(repository.id());
operation.setRepositoryOwner(repository.owner());
@@ -109,11 +140,20 @@ public class RepositoryWrites {
} catch (org.springframework.dao.DuplicateKeyException collision) {
throw new BusinessException(409, "IDEMPOTENCY_KEY_REUSED", "操作编码已用于其他请求,请勿复用");
}
log.info("仓库写入开始 projectCode={} operationCode={} baseCommitSha={} fileCount={}",
project.getProjectCode(), request.code(), baseSha, changes.size());
log.info(
"仓库写入开始 projectCode={} operationCode={} baseCommitSha={} fileCount={}",
project.getProjectCode(),
request.code(),
baseSha,
changes.size());
try {
CommitView result = gitea.commit(repository, changes,
message + WriteReceipt.suffix(request.code(), request.digest()), author.nickname(), author.code());
CommitView result =
gitea.commit(
repository,
changes,
message + WriteReceipt.suffix(request.code(), request.digest()),
author.nickname(),
author.code());
return new CommitView(result.commitSha(), message);
} catch (GiteaFailure error) {
if (!error.uncertain()) throw error;
@@ -126,23 +166,141 @@ public class RepositoryWrites {
private CommitView replay(RepositoryWriteOperation operation) {
return switch (operation.getStatus()) {
case "SUCCEEDED" -> new CommitView(operation.getResultCommitSha(), operation.getCommitMessage());
case "FAILED" -> throw new BusinessException(operation.getFailureStatus(),
operation.getLastErrorCode(), "该操作已被仓库拒绝,请刷新后使用新的操作编码提交");
case "SUCCEEDED" ->
new CommitView(operation.getResultCommitSha(), operation.getCommitMessage());
case "FAILED" ->
throw new BusinessException(
operation.getFailureStatus(),
operation.getLastErrorCode(),
"该操作已被仓库拒绝,请刷新后使用新的操作编码提交");
default -> throw pending(operation);
};
}
private BusinessException pending(RepositoryWriteOperation operation) {
return new BusinessException(409, "REMOTE_RESULT_UNKNOWN", "仓库提交结果正在后台核对,请稍后刷新工作区,暂勿重复提交",
return new BusinessException(
409,
"REMOTE_RESULT_UNKNOWN",
"仓库提交结果正在后台核对,请稍后刷新工作区,暂勿重复提交",
java.util.Map.of("projectCode", operation.getProjectCode()));
}
public RepositoryWriteOperation active(String code) {
return operations.selectOne(new LambdaQueryWrapper<RepositoryWriteOperation>()
.eq(RepositoryWriteOperation::getProjectCode, code)
.in(RepositoryWriteOperation::getStatus, ACTIVE)
.orderByDesc(RepositoryWriteOperation::getCreatedAt).last("LIMIT 1"));
return operations.selectOne(
new LambdaQueryWrapper<RepositoryWriteOperation>()
.eq(RepositoryWriteOperation::getProjectCode, code)
.in(RepositoryWriteOperation::getStatus, ACTIVE)
.orderByDesc(RepositoryWriteOperation::getCreatedAt)
.last("LIMIT 1"));
}
/** 实时校验、写入以及命名锁释放全部纳入同一结果核对边界。 */
public byte[] push(
GitAccess access,
GitIo.PushCommand command,
Supplier<Project> validate,
Supplier<byte[]> transfer) {
java.util.concurrent.atomic.AtomicReference<RepositoryWriteOperation> registered =
new java.util.concurrent.atomic.AtomicReference<>();
try {
return locks.serial(
access.projectCode(),
() -> pushLocked(validate.get(), access, command, transfer, registered::set));
} catch (RuntimeException error) {
RepositoryWriteOperation operation = registered.get();
if (operation != null && operations.selectById(operation.getOperationCode()) != null) {
locks.markUncertain(access.projectCode());
operation.setStatus("PROCESSING");
operation.setLastErrorCode(
error instanceof BusinessException business
? business.code()
: "GIT_RESULT_UNKNOWN");
save(operation);
log.warn(
"Git 推送待核对 projectCode={} operationCode={} oldSha={} newSha={}"
+ " errorCode={}",
access.projectCode(),
operation.getOperationCode(),
command.oldSha(),
command.newSha(),
operation.getLastErrorCode());
}
throw error;
}
}
/** 调用者已在共享项目锁中完成实时鉴权、old SHA 和保护规则校验。 */
private byte[] pushLocked(
Project project,
GitAccess access,
GitIo.PushCommand command,
Supplier<byte[]> transfer,
java.util.function.Consumer<RepositoryWriteOperation> registered) {
RepositoryWriteOperation pending = active(project.getProjectCode());
if (pending != null) throw pending(pending);
GiteaClient.Repository repository = projects.repository(project);
RepositoryWriteOperation operation = new RepositoryWriteOperation();
operation.setOperationCode(Codes.next());
operation.setOperationKind("GIT_PUSH");
operation.setProjectCode(project.getProjectCode());
operation.setRepositoryId(repository.id());
operation.setRepositoryOwner(repository.owner());
operation.setRepositoryName(repository.name());
operation.setBaseCommitSha(command.oldSha());
operation.setTargetCommitSha(command.newSha());
operation.setActorUserCode(access.userCode());
operation.setGitTokenCode(access.tokenCode());
operation.setRequestDigest(
SecurityHashes.sha256(
repository.id()
+ ":refs/heads/main:"
+ command.oldSha()
+ ":"
+ command.newSha()));
operation.setCommitMessage("Git push main");
operation.setStatus("PROCESSING");
operation.setCheckCount(0);
operation.setCreatedAt(ProjectSupport.now());
operation.setUpdatedAt(operation.getCreatedAt());
operation.setNextCheckAt(operation.getCreatedAt().plusSeconds(15));
registered.accept(operation);
return locks.write(
project.getProjectCode(),
() ->
transactions.executeWithoutResult(
transaction -> {
operations.insert(operation);
project.setRepoWriteStatus("WRITING");
if (projectMapper.updateById(project) != 1)
throw new IllegalStateException("项目写保护保存失败");
}),
() -> {
log.info(
"Git 推送开始 projectCode={} userCode={} tokenCode={} operationCode={}"
+ " oldSha={} newSha={}",
project.getProjectCode(),
access.userCode(),
access.tokenCode(),
operation.getOperationCode(),
command.oldSha(),
command.newSha());
byte[] result = transfer.get();
boolean success;
try {
success = GitIo.pushSucceeded(result);
} catch (java.io.IOException error) {
throw GitIo.upstreamInvalid();
}
if (success && !command.newSha().equals(gitea.head(repository)))
throw new BusinessException(503, "REMOTE_RESULT_UNKNOWN", "推送结果尚未核对");
operation.setLastErrorCode(success ? null : "GIT_PUSH_REJECTED");
operation.setFailureStatus(success ? null : 409);
complete(
operation,
success ? command.newSha() : null,
success ? "SUCCEEDED" : "FAILED");
return result;
});
}
private boolean slowCheckDue(RepositoryWriteOperation operation) {
@@ -151,20 +309,29 @@ public class RepositoryWrites {
/** 重启后直接扫描数据库,无内存任务队列;多实例由相同项目命名锁串行核对。 */
public void reconcileDue() {
List<RepositoryWriteOperation> due = operations.selectList(new LambdaQueryWrapper<RepositoryWriteOperation>()
.in(RepositoryWriteOperation::getStatus, ACTIVE)
.le(RepositoryWriteOperation::getNextCheckAt, ProjectSupport.now())
.orderByAsc(RepositoryWriteOperation::getNextCheckAt).last("LIMIT 20"));
List<RepositoryWriteOperation> due =
operations.selectList(
new LambdaQueryWrapper<RepositoryWriteOperation>()
.in(RepositoryWriteOperation::getStatus, ACTIVE)
.le(RepositoryWriteOperation::getNextCheckAt, ProjectSupport.now())
.orderByAsc(RepositoryWriteOperation::getNextCheckAt)
.last("LIMIT 20"));
for (RepositoryWriteOperation candidate : due) {
try {
locks.serial(candidate.getProjectCode(), () -> {
reconcile(candidate.getOperationCode());
return null;
});
locks.serial(
candidate.getProjectCode(),
() -> {
reconcile(candidate.getOperationCode());
return null;
});
} catch (RuntimeException error) {
log.warn("仓库后台核对未完成 projectCode={} operationCode={} errorCode={} errorType={}",
candidate.getProjectCode(), candidate.getOperationCode(),
error instanceof BusinessException business ? business.code() : "CHECK_FAILED",
log.warn(
"仓库后台核对未完成 projectCode={} operationCode={} errorCode={} errorType={}",
candidate.getProjectCode(),
candidate.getOperationCode(),
error instanceof BusinessException business
? business.code()
: "CHECK_FAILED",
error.getClass().getSimpleName());
}
}
@@ -172,10 +339,13 @@ public class RepositoryWrites {
private void reconcile(String operationCode) {
RepositoryWriteOperation operation = operations.selectById(operationCode);
if (operation == null || !ACTIVE.contains(operation.getStatus())
if (operation == null
|| !ACTIVE.contains(operation.getStatus())
|| operation.getNextCheckAt().isAfter(ProjectSupport.now())) return;
Project project = projectMapper.selectOne(new LambdaQueryWrapper<Project>()
.eq(Project::getProjectCode, operation.getProjectCode()));
Project project =
projectMapper.selectOne(
new LambdaQueryWrapper<Project>()
.eq(Project::getProjectCode, operation.getProjectCode()));
String receipt = project == null ? null : receipt(operation, project);
operation.setCheckCount(operation.getCheckCount() + 1);
if (receipt != null) {
@@ -186,9 +356,13 @@ public class RepositoryWrites {
operation.setStatus(overdue ? "NEEDS_REVIEW" : "PROCESSING");
operation.setNextCheckAt(ProjectSupport.now().plusSeconds(overdue ? 60 : 15));
save(operation);
log.info("仓库写入继续核对 projectCode={} operationCode={} status={} checkCount={} errorCode={}",
operation.getProjectCode(), operationCode, operation.getStatus(),
operation.getCheckCount(), operation.getLastErrorCode());
log.info(
"仓库写入继续核对 projectCode={} operationCode={} status={} checkCount={} errorCode={}",
operation.getProjectCode(),
operationCode,
operation.getStatus(),
operation.getCheckCount(),
operation.getLastErrorCode());
}
private String receipt(RepositoryWriteOperation operation, Project project) {
@@ -202,34 +376,61 @@ public class RepositoryWrites {
if (remote.id() != operation.getRepositoryId()
|| !remote.owner().equals(operation.getRepositoryOwner())
|| !remote.name().equals(operation.getRepositoryName())
|| !remote.privateRepo() || remote.mirror() || remote.empty()
|| !remote.privateRepo()
|| remote.mirror()
|| remote.empty()
|| !"main".equals(remote.defaultBranch()))
throw new GiteaFailure(409, "WRITE_REPOSITORY_MISMATCH", "远端仓库身份发生变化", false);
String sha = gitea.findWriteReceipt(remote.repository(), operation.getBaseCommitSha(),
operation.getOperationCode(), operation.getRequestDigest());
String sha;
if ("GIT_PUSH".equals(operation.getOperationKind())) {
String head = gitea.head(remote.repository());
sha = Objects.equals(head, operation.getTargetCommitSha()) ? head : null;
// main 仍为旧 SHA 不能证明接收任务已经结束,继续保持写保护。
} else {
sha =
gitea.findWriteReceipt(
remote.repository(),
operation.getBaseCommitSha(),
operation.getOperationCode(),
operation.getRequestDigest());
}
if (sha == null) operation.setLastErrorCode("WRITE_RECEIPT_NOT_FOUND");
return sha;
} catch (RuntimeException error) {
operation.setLastErrorCode(error instanceof BusinessException business
? business.code() : "WRITE_RECEIPT_READ_FAILED");
log.warn("仓库写入凭据读取未确认 projectCode={} operationCode={} errorCode={} errorType={}",
operation.getProjectCode(), operation.getOperationCode(), operation.getLastErrorCode(),
operation.setLastErrorCode(
error instanceof BusinessException business
? business.code()
: "WRITE_RECEIPT_READ_FAILED");
log.warn(
"仓库写入凭据读取未确认 projectCode={} operationCode={} errorCode={} errorType={}",
operation.getProjectCode(),
operation.getOperationCode(),
operation.getLastErrorCode(),
error.getClass().getSimpleName());
if (!(error instanceof BusinessException)) SafeExceptionLog.write(log, "仓库凭据核对异常", error);
if (!(error instanceof BusinessException))
SafeExceptionLog.write(log, "仓库凭据核对异常", error);
return null;
}
}
private void complete(RepositoryWriteOperation operation, String sha, String status) {
// 项目恢复与操作终态在同一事务提交;任务获得锁后重读终态,旧任务不会清除新操作的保护。
transactions.executeWithoutResult(transaction -> {
operation.setStatus(status);
operation.setResultCommitSha(sha);
save(operation);
locks.confirmKnownResult(operation.getProjectCode());
});
log.info("仓库写入结果已落盘 projectCode={} operationCode={} status={} commitSha={}",
operation.getProjectCode(), operation.getOperationCode(), status, sha);
transactions.executeWithoutResult(
transaction -> {
operation.setStatus(status);
operation.setResultCommitSha(sha);
save(operation);
// 持锁连接释放异常可能使上一条终态重新进入核对;不能解除另一条操作的保护。
if (active(operation.getProjectCode()) == null)
locks.confirmKnownResult(operation.getProjectCode());
else locks.markUncertain(operation.getProjectCode());
});
log.info(
"仓库写入结果已落盘 projectCode={} operationCode={} status={} commitSha={}",
operation.getProjectCode(),
operation.getOperationCode(),
status,
sha);
}
private void save(RepositoryWriteOperation operation) {
@@ -0,0 +1,99 @@
package com.linkerbot.vibemake.service.git;
import static org.assertj.core.api.Assertions.*;
import com.linkerbot.vibemake.common.BusinessException;
import org.junit.jupiter.api.Test;
import java.io.*;
import java.nio.charset.StandardCharsets;
import java.util.List;
class GitIoTest {
private byte[] pkt(String... lines) throws Exception {
ByteArrayOutputStream out = new ByteArrayOutputStream();
for (String line : lines)
if (line == null) out.write("0000".getBytes());
else GitIo.packet(out, line.getBytes(StandardCharsets.UTF_8));
return out.toByteArray();
}
@Test
void commandLeavesPackUnreadAndRejectsAllOtherRefsAndExtensions() throws Exception {
String command =
"a".repeat(40)
+ " "
+ "b".repeat(40)
+ " refs/heads/main\0 report-status ofs-delta agent=git/test";
ByteArrayOutputStream body = new ByteArrayOutputStream();
body.write(pkt(command, null));
body.write("PACK".getBytes());
ByteArrayInputStream input = new ByteArrayInputStream(body.toByteArray());
assertThat(GitIo.command(input).newSha()).isEqualTo("b".repeat(40));
assertThat(input.readAllBytes()).isEqualTo("PACK".getBytes());
for (String bad :
List.of(
command.replace("main", "other"),
command.replace("refs/heads/main", "refs/tags/v1"),
command.replace("b".repeat(40), "0".repeat(40)),
command + " push-options",
command + " side-band-64k",
command + " report-status-v2")) {
assertThatThrownBy(() -> GitIo.command(new ByteArrayInputStream(pkt(bad, null))))
.isInstanceOf(BusinessException.class);
}
assertThatThrownBy(
() -> GitIo.command(new ByteArrayInputStream(pkt(command, command, null))))
.isInstanceOf(BusinessException.class);
}
@Test
void advertisementNegotiatesOnlyAuditedReportStatusAndMain() throws Exception {
byte[] advertisement =
pkt(
"# service=git-receive-pack\n",
null,
"a".repeat(40)
+ " refs/heads/main\0report-status report-status-v2 delete-refs"
+ " side-band-64k ofs-delta push-options\n",
"b".repeat(40) + " refs/tags/v1\n",
null);
String result =
new String(GitIo.receiveAdvertisement(advertisement), StandardCharsets.UTF_8);
assertThat(result)
.contains("report-status", "ofs-delta", "refs/heads/main")
.doesNotContain(
"side-band",
"delete-refs",
"push-options",
"report-status-v2",
"refs/tags");
}
@Test
void statusMustBeGitSuccessNotMerelyHttp200() throws Exception {
assertThat(GitIo.pushSucceeded(pkt("unpack ok\n", "ok refs/heads/main\n", null))).isTrue();
assertThat(
GitIo.pushSucceeded(
pkt("unpack ok\n", "ng refs/heads/main non-fast-forward\n", null)))
.isFalse();
assertThatThrownBy(() -> GitIo.pushSucceeded("HTTP 200".getBytes()))
.isInstanceOf(BusinessException.class);
assertThatThrownBy(() -> GitIo.pushSucceeded(pkt("unpack ok\n", "ok refs/heads/main\n")))
.isInstanceOf(BusinessException.class);
}
@Test
void actualBytesAndMetadataAreBoundedWithoutReadAll() throws Exception {
assertThatThrownBy(
() ->
GitIo.limited(new ByteArrayInputStream(new byte[1025]), 1024)
.transferTo(OutputStream.nullOutputStream()))
.isInstanceOf(BusinessException.class)
.extracting("status")
.isEqualTo(413);
assertThatThrownBy(() -> GitIo.metadata(new ByteArrayInputStream(new byte[1025]), 1024))
.isInstanceOf(BusinessException.class);
}
}
@@ -0,0 +1,46 @@
package com.linkerbot.vibemake.service.git;
import static org.assertj.core.api.Assertions.*;
import com.linkerbot.vibemake.common.BusinessException;
import com.linkerbot.vibemake.service.config.GitGatewayProperties;
import com.linkerbot.vibemake.testing.InMemoryStateStore;
import org.junit.jupiter.api.Test;
class GitLimitsTest {
@Test
void poolsAreIndependentAndReleaseAfterCancellationAndUserAndIpAreSeparate() throws Exception {
GitGatewayProperties config = new GitGatewayProperties();
config.setReadConcurrent(1);
config.setPushConcurrent(1);
config.setLfsConcurrent(1);
config.setGitRequestsPerMinute(1);
config.setAuthFailuresPerMinute(1);
GitLimits limits = new GitLimits(new InMemoryStateStore(), config);
try (AutoCloseable read = limits.acquire(false, false);
AutoCloseable push = limits.acquire(true, false);
AutoCloseable lfs = limits.acquire(false, true)) {
assertThatThrownBy(() -> limits.acquire(false, false))
.isInstanceOf(BusinessException.class);
assertThatThrownBy(() -> limits.acquire(true, false))
.isInstanceOf(BusinessException.class);
assertThatThrownBy(() -> limits.acquire(false, true))
.isInstanceOf(BusinessException.class);
}
try (AutoCloseable read = limits.acquire(false, false)) {
assertThat(read).isNotNull();
}
GitAccess first = new GitAccess(1, "user-one", "token-one", "project", true);
GitAccess second = new GitAccess(2, "user-two", "token-two", "project", true);
limits.request(first, false);
limits.request(second, false);
limits.request(first, true);
assertThatThrownBy(() -> limits.request(first, false))
.isInstanceOf(BusinessException.class);
limits.authenticationFailed("127.0.0.1");
limits.authenticationAllowed("127.0.0.2");
assertThatThrownBy(() -> limits.authenticationAllowed("127.0.0.1"))
.isInstanceOf(BusinessException.class);
}
}
@@ -0,0 +1,70 @@
package com.linkerbot.vibemake.service.git;
import static org.assertj.core.api.Assertions.*;
import static org.mockito.Mockito.*;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.linkerbot.vibemake.bean.entity.Project;
import com.linkerbot.vibemake.common.BusinessException;
import com.linkerbot.vibemake.service.ProjectService;
import com.linkerbot.vibemake.service.config.*;
import com.linkerbot.vibemake.service.gitea.GiteaClient;
import com.sun.net.httpserver.HttpServer;
import org.junit.jupiter.api.Test;
import java.net.InetSocketAddress;
import java.util.concurrent.atomic.AtomicBoolean;
class GitPolicyTest {
@Test
void independentlyVerifiesBotIdentityAndRepositoryIdentityBeforeGrantingAnyAccess()
throws Exception {
AtomicBoolean admin = new AtomicBoolean(false);
HttpServer server = HttpServer.create(new InetSocketAddress("127.0.0.1", 0), 0);
server.createContext(
"/api/v1/user",
exchange -> {
byte[] bytes =
("{\"login\":\"bot\",\"is_admin\":" + admin.get() + "}").getBytes();
exchange.sendResponseHeaders(200, bytes.length);
exchange.getResponseBody().write(bytes);
exchange.close();
});
server.start();
GitGatewayProperties config = new GitGatewayProperties();
config.setUpstreamUsername("bot");
config.setUpstreamToken("fixture");
VibemakeProperties general = new VibemakeProperties();
general.getGitea().setBaseUrl("http://127.0.0.1:" + server.getAddress().getPort());
GiteaClient remote = mock(GiteaClient.class);
ProjectService projects = mock(ProjectService.class);
Project project = new Project();
project.setGiteaRepoId(9L);
project.setGiteaOwner("owner");
project.setGiteaRepoName("repo");
GiteaClient.Repository repo = new GiteaClient.Repository(9, "owner", "repo");
when(projects.repository(project)).thenReturn(repo);
when(remote.inspect("repo"))
.thenReturn(
new GiteaClient.RemoteRepository(
9, "owner", "repo", true, false, false, null, "main"));
try (GitTransport transport = new GitTransport(config, general)) {
GitPolicy policy =
new GitPolicy(remote, projects, transport, config, new ObjectMapper());
policy.provision(project);
verify(remote).ensureGitAccess(repo, "bot");
admin.set(true);
assertThatThrownBy(() -> policy.verify(project)).isInstanceOf(BusinessException.class);
admin.set(false);
when(remote.inspect("repo"))
.thenReturn(
new GiteaClient.RemoteRepository(
10, "owner", "repo", true, false, false, null, "main"));
assertThatThrownBy(() -> policy.verify(project)).isInstanceOf(BusinessException.class);
verify(remote, never()).verifyGitProtection(repo, "bot");
} finally {
server.stop(0);
}
}
}
@@ -0,0 +1,72 @@
package com.linkerbot.vibemake.service.git;
import static org.assertj.core.api.Assertions.*;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.linkerbot.vibemake.common.BusinessException;
import com.linkerbot.vibemake.service.config.VibemakeProperties;
import com.linkerbot.vibemake.service.gitea.*;
import com.sun.net.httpserver.HttpServer;
import org.junit.jupiter.api.Test;
import java.net.InetSocketAddress;
import java.util.concurrent.atomic.*;
class GitProtectionTest {
@Test
void strictExistingRulesAreNeverOverwrittenAndAdminOrForcePushAreRejected() throws Exception {
AtomicReference<String> rule = new AtomicReference<>("[]");
AtomicReference<String> permission = new AtomicReference<>("write");
AtomicInteger writes = new AtomicInteger();
HttpServer server = HttpServer.create(new InetSocketAddress("127.0.0.1", 0), 0);
server.createContext(
"/",
exchange -> {
String response;
if (exchange.getRequestURI().getPath().endsWith("/permission"))
response = "{\"permission\":\"" + permission.get() + "\"}";
else {
if (exchange.getRequestMethod().equals("POST")) {
writes.incrementAndGet();
rule.set(
"[{\"rule_name\":\"main\",\"enable_push\":true,\"enable_force_push\":false}]");
}
response = rule.get();
}
byte[] bytes = response.getBytes();
exchange.sendResponseHeaders(200, bytes.length);
exchange.getResponseBody().write(bytes);
exchange.close();
});
server.start();
try {
VibemakeProperties config = new VibemakeProperties();
config.getGitea().setBaseUrl("http://127.0.0.1:" + server.getAddress().getPort());
config.getGitea().setToken("fixture");
GiteaClient client = new RestGiteaClient(config, new ObjectMapper());
GiteaClient.Repository repo = new GiteaClient.Repository(1, "fixture", "repo");
client.ensureGitProtection(repo, "bot");
assertThat(writes.get()).isEqualTo(1);
client.ensureGitProtection(repo, "bot");
assertThat(writes.get()).isEqualTo(1);
permission.set("admin");
assertThatThrownBy(() -> client.verifyGitProtection(repo, "bot"))
.isInstanceOf(BusinessException.class);
permission.set("write");
for (String bad :
new String[] {
"[{\"rule_name\":\"main\",\"enable_push\":true,\"enable_force_push\":true}]",
"[{\"rule_name\":\"*\",\"enable_push\":true,\"enable_force_push\":false}]",
"[{\"rule_name\":\"main\",\"enable_push\":false,\"enable_force_push\":false}]"
}) {
rule.set(bad);
assertThatThrownBy(() -> client.ensureGitProtection(repo, "bot"))
.isInstanceOf(BusinessException.class);
}
assertThat(writes.get()).isEqualTo(1);
} finally {
server.stop(0);
}
}
}
@@ -0,0 +1,72 @@
package com.linkerbot.vibemake.service.git;
import static org.assertj.core.api.Assertions.*;
import com.linkerbot.vibemake.common.BusinessException;
import com.linkerbot.vibemake.service.config.*;
import com.sun.net.httpserver.HttpServer;
import org.junit.jupiter.api.Test;
import java.io.*;
import java.net.InetSocketAddress;
import java.time.Duration;
import java.util.Map;
import java.util.concurrent.*;
class GitTransportTest {
@Test
void budgetCoversStalledResponseBodyAndNeverFollowsRedirects() throws Exception {
HttpServer server = HttpServer.create(new InetSocketAddress("127.0.0.1", 0), 0);
server.setExecutor(Executors.newVirtualThreadPerTaskExecutor());
CountDownLatch release = new CountDownLatch(1);
server.createContext(
"/stall",
exchange -> {
exchange.sendResponseHeaders(200, 0);
exchange.getResponseBody().write(1);
exchange.getResponseBody().flush();
try {
release.await(5, TimeUnit.SECONDS);
} catch (InterruptedException error) {
Thread.currentThread().interrupt();
}
exchange.close();
});
server.createContext(
"/redirect",
exchange -> {
exchange.getResponseHeaders().set("Location", "http://untrusted.invalid/");
exchange.sendResponseHeaders(302, -1);
exchange.close();
});
server.start();
GitGatewayProperties properties = new GitGatewayProperties();
properties.setTimeout(Duration.ofMillis(300));
VibemakeProperties config = new VibemakeProperties();
config.getGitea().setBaseUrl("http://127.0.0.1:" + server.getAddress().getPort());
try (GitTransport transport = new GitTransport(properties, config)) {
try (GitTransport.Budget budget = transport.budget();
GitTransport.Response result =
transport.exchange("/stall", "GET", null, -1, Map.of(), budget)) {
long started = System.nanoTime();
assertThatThrownBy(() -> result.input().transferTo(OutputStream.nullOutputStream()))
.isInstanceOf(BusinessException.class)
.extracting("code")
.isEqualTo("GIT_TIMEOUT");
assertThat(Duration.ofNanos(System.nanoTime() - started))
.isLessThan(Duration.ofSeconds(3));
}
try (GitTransport.Budget budget = transport.budget()) {
assertThatThrownBy(
() ->
transport.exchange(
"/redirect", "GET", null, -1, Map.of(), budget))
.isInstanceOf(BusinessException.class);
}
} finally {
release.countDown();
server.stop(0);
}
}
}