git clone功能
网站登录 → 创建项目 Git Token → clone → 本地修改、commit → push main → 网站工作区刷新后显示最新内容 → 网站手动发布到社区
This commit is contained in:
@@ -304,3 +304,27 @@ bash scripts/test-admin-mysql.sh
|
||||
项目响应采用数据库批量查询,不缓存业务列表。列表和 AI 检索不读取体积较大的 URDF 文件清单,Fork 来源仅批量读取标识和编码。按业务服务层统计(不包含安全过滤器独立账号查询),20 条带 Fork 来源、登录用户点赞和导入来源的项目列表最多 8 次 SQL,评论列表最多 4 次,后台用户列表最多 3 次;避免记录数增加造成逐条查询。昵称、互动和下架可见性仍读取当前数据。
|
||||
|
||||
普通 `AI_ENABLED=false OSS_ENABLED=false mvn verify` 使用只存在于 tests 分类制品中的状态存储,不依赖 Redis,生产 jar 不包含该实现。真实 Redis 联调不自动读取 `.env`,通过 `REDIS_TEST_HOST`、`REDIS_TEST_PORT`、`REDIS_TEST_USERNAME`、`REDIS_TEST_PASSWORD`、`REDIS_TEST_DATABASE`、`REDIS_TEST_SSL_ENABLED` 连接独立测试实例,执行 `bash scripts/test-redis.sh`;只使用随机测试 Key 前缀,不执行 FLUSHDB。真实联调验证 TTL、消费竞争、计数、断开后重建连接、会话撤销与账号禁用。
|
||||
|
||||
## HTTPS Git 网关
|
||||
|
||||
网站身份可通过项目独立凭据 clone、fetch、pull、push main,支持 LFS Basic Transfer;社区仍手动发布。默认关闭,普通用户无需 Gitea 账号。先执行 [Git 凭据建表 SQL](docs/database.md#项目-git-凭据) 和 [写入记录扩展 SQL](docs/admin-database.md#https-git-写入记录扩展),同时部署社区端和管理端兼容版本,再设置 `GIT_ENABLED=true`。应用不自动迁库,本次不修改 `.sql` 文件。
|
||||
|
||||
| 环境变量 | 默认值/含义 |
|
||||
| --- | --- |
|
||||
| GIT_ENABLED | false |
|
||||
| GIT_PUBLIC_BASE_URL | 必填的固定公网 HTTPS 地址,例如 `https://git.example.com/git`,不根据 Host 构造 |
|
||||
| GIT_UPSTREAM_USERNAME / GIT_UPSTREAM_TOKEN | 独立非管理员、非组织所有者技术账号及其 PAT;需要 read:user、write:repository 范围,与仓库管理 GITEA_TOKEN 分离 |
|
||||
| GIT_REQUEST_TIMEOUT | 10m,覆盖请求及响应流 |
|
||||
| GIT_MAX_PACK_BYTES / GIT_MAX_LFS_OBJECT_BYTES | 5368709120 / 1073741824 |
|
||||
| GIT_MAX_METADATA_BYTES / GIT_MAX_BATCH_OBJECTS | 8388608 / 500 |
|
||||
| GIT_READ_CONCURRENT / GIT_PUSH_CONCURRENT / GIT_LFS_CONCURRENT | 4 / 2 / 8,单实例 |
|
||||
| GIT_REQUESTS_PER_MINUTE / GIT_LFS_REQUESTS_PER_MINUTE | 120 / 600,每用户 |
|
||||
| GIT_AUTH_FAILURES_PER_MINUTE | 20,每IP;仅信任已配置代理 |
|
||||
|
||||
技术账号不能是仓库管理员;首次签发项目 Token 时后端管理凭据会给它授予该仓库 write 权限。首个读写 Token 初始化 main 保护,已有规则必须兼容;每次 push 重新核验,非快进和删除由 Gitea 接收钩子拒绝。仓库主存储、对象完整性和 LFS 保留在 Gitea。部署必须确保 Git/LFS 原生端点直接响应,无重定向、且上游 LFS action 使用配置的 Gitea origin 和本项目路径;不支持外部对象存储 action 直链。
|
||||
|
||||
代理 `/git/` 关闭请求/响应缓冲、缓存,透传 Authorization、Git-Protocol、Content-Type、Accept、Content-Encoding;请求大小允许 pack/LFS 额度,代理与 Gitea 超时至少覆盖10分钟预算,默认读取超时不低于660秒。服务端不得记录认证头或 action Bearer。使用 HTTPS 对外开放;本机真实联调才允许回环 HTTP。保持当前社区端、管理端各单实例部署范围。
|
||||
|
||||
前端对接见 [HTTPS Git 与本地开发](docs/vibemake-api.md#https-git-与本地开发),运维恢复与回滚见 [Git 网关运维](docs/admin-deployment.md#https-git-网关运维)。Git Token 继承现有网页登录身份边界:邮箱加图片验证码目前不验证邮箱归属。
|
||||
|
||||
验证:`AI_ENABLED=false VIBEMAKE_AI_ENABLED=false OSS_ENABLED=false mvn verify`。`python3 scripts/test-git-gateway.py` 读取 `.env`,在随机测试仓库及临时技术账号上运行真实 Git/Gitea/LFS(含大于50 MiB对象)闭环,测试完成后清理;需要 GITEA_TOKEN 具备创建/删除测试用户权限,以及可连接的 Redis(只使用随机 git-live 前缀)。配置 DB_URL 时还会创建随机 `vibemake_git_verify_*` 临时 MySQL 库,需建库权限;`GIT_GATEWAY_MYSQL=false` 可单独使用 H2 验证 Git/Redis。MySQL 锁测试只创建随机临时库,Redis 测试使用独立随机前缀,均不修改已有业务数据。
|
||||
|
||||
@@ -163,6 +163,10 @@ ALTER TABLE project_import_task
|
||||
```sql
|
||||
CREATE TABLE repository_write_operation (
|
||||
operation_code CHAR(32) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL COMMENT '写入操作业务编码,同时作为幂等键',
|
||||
operation_kind VARCHAR(16) NOT NULL DEFAULT 'WEB_COMMIT' COMMENT '操作类型 WEB_COMMIT网页提交 GIT_PUSH本地推送',
|
||||
target_commit_sha VARCHAR(64) NULL COMMENT 'Git推送请求的目标提交SHA,网页提交可为空',
|
||||
actor_user_code VARCHAR(32) NULL COMMENT '实际发起推送的网站用户业务编码',
|
||||
git_token_code CHAR(32) CHARACTER SET ascii COLLATE ascii_bin NULL COMMENT '本次推送使用的Git凭据业务编码,不保存明文',
|
||||
project_code CHAR(32) NOT NULL COMMENT '项目业务编码',
|
||||
repository_id BIGINT NOT NULL COMMENT '操作发起时Gitea仓库内部标识,仅核验使用',
|
||||
repository_owner VARCHAR(255) NOT NULL COMMENT '操作发起时仓库所属账号',
|
||||
@@ -260,3 +264,16 @@ ALTER TABLE repository_write_operation
|
||||
```
|
||||
|
||||
旧导入状态首次由新版读取时,将未声明时区的 `reservedAt` 从 UTC 转为上海时间,原子保存并标记 `timeZone=Asia/Shanghai`,后续不会再次转换。应在业务库完成转换后再续跑;未迁移的数据库时间与清单不一致时,现有归属校验会阻止接管项目。回滚应用前需要同时恢复数据库和导入状态备份,并恢复前端的旧时间解析约定。
|
||||
|
||||
|
||||
## HTTPS Git 写入记录扩展
|
||||
|
||||
已有 `repository_write_operation` 表手工执行下列 SQL;新库使用上方完整建表语句,不重复执行 ALTER。先升级社区端和管理端兼容代码,再开启 Git 网关。既有记录默认归类为 WEB_COMMIT;Git 推送不使用网页提交尾注核对。
|
||||
|
||||
```sql
|
||||
ALTER TABLE repository_write_operation
|
||||
ADD COLUMN operation_kind VARCHAR(16) NOT NULL DEFAULT 'WEB_COMMIT' COMMENT '操作类型 WEB_COMMIT网页提交 GIT_PUSH本地推送',
|
||||
ADD COLUMN target_commit_sha VARCHAR(64) NULL COMMENT 'Git推送请求的目标提交SHA,网页提交可为空',
|
||||
ADD COLUMN actor_user_code VARCHAR(32) NULL COMMENT '实际发起推送的网站用户业务编码',
|
||||
ADD COLUMN git_token_code CHAR(32) CHARACTER SET ascii COLLATE ascii_bin NULL COMMENT '本次推送使用的Git凭据业务编码,不保存明文';
|
||||
```
|
||||
|
||||
@@ -203,3 +203,23 @@ Gitea 原始文件有界读取,contents JSON 上限按单文件 Base64 膨胀
|
||||
项目 SQL 从 122 降到 8,评论列表从 23 降到 4,20 用户后台列表从 22 降到 3。MySQL 四并发 P95 从 2414.935ms 降到 221.998ms,约降低 91%。本地微基准受运行波动影响,不能将 SQL 数变化等同于所有环境的固定延迟收益。堆池峰值是各 heap memory pool 的峰值之和,并非进程 RSS;普通小请求的内存收益不稳定,本期内存保护主要针对超大正文、无界上游响应和大文件并发。
|
||||
|
||||
复测:`AI_ENABLED=false OSS_ENABLED=false mvn -pl vibemake-api -am -Dtest=QueryEfficiencyTest -Dsurefire.failIfNoSpecifiedTests=false '-DargLine=-Xms128m -Xmx512m' test`。默认使用 H2;显式设置 BENCHMARK_MYSQL_URL、BENCHMARK_MYSQL_USERNAME、BENCHMARK_MYSQL_PASSWORD 时,自动创建并最终删除 vibemake_benchmark_* 随机临时库,执行 DDL 前验证连接 catalog 必须等于本次随机库。账号需要创建和删除临时库权限,禁止将基准改为直接重置业务库。该小样本不代表正式环境容量,部署仍需结合真实项目体量验证。
|
||||
|
||||
## HTTPS Git 网关运维
|
||||
|
||||
仅社区 API 暴露 `/git/**`;管理端继续使用后台认证。按 [README 配置](../README.md#https-git-网关) 部署独立 Git 技术账号、HTTPS 地址与无缓冲代理。功能默认关闭;先人工执行 [Git Token 表](database.md#项目-git-凭据) 与 [operation_kind 等字段扩展](admin-database.md#https-git-写入记录扩展),再排空旧进程并升级两个 API,最后对测试项目验证后开放。不能让旧版后台核对器消费 GIT_PUSH 操作。
|
||||
|
||||
Git 技术账号须非 site admin、非组织所有者、非仓库管理员,PAT 只给 read:user、write:repository。首次 Token 签发会按项目给技术账号授予 write,不提升其已有 admin/owner 权限;权限不兼容必须先人工处理。已有 main 保护不覆盖,通配/多规则配置会保守拒绝开通;每次 push 复核。运维直接修改仓库引用或 main 保护前应停写,不能绕过网关并发控制。LFS action 以短期 Redis 状态承接,上游地址与认证头只保留服务端。Redis 丢失 action 后需重新请求 batch;原 Git 凭据存 MySQL,可撤销。
|
||||
|
||||
网页与本地写入使用同一 ProjectLocks 和 repo_write_status,跨进程由 MySQL 命名锁串行。操作记录必须在转发 receive-pack 前提交,operation_kind=GIT_PUSH 记录 base_commit_sha、target_commit_sha、actor_user_code、git_token_code,摘要不含 pack。HTTP200 不是成功证明;仅接受完整 report-status,成功后再次核对 main,并原子保存终态与 IDLE。推送作者来自本地提交,审计 actor_user_code 是网站发起者,两者可能不同。
|
||||
|
||||
后台核对按类型区分:WEB_COMMIT 继续查找原有尾注;GIT_PUSH 先校验项目/远端仓库身份、私有属性及 main,再仅在 main 精确等于 target_commit_sha 时确认成功。main 仍等于 base_commit_sha、为其他SHA、上游不可用或读取失败均保持 REVIEW,不能自动判失败。10分钟后状态为 NEEDS_REVIEW,每分钟继续只读核对,不重发 push。
|
||||
|
||||
人工恢复顺序:
|
||||
|
||||
1. 关闭 Git 入口、停止并排空社区/后台仓库写入和发布,暂停核对 worker,保留数据库和日志备份。
|
||||
2. 根据 requestId、projectCode、operationCode、actor_user_code、git_token_code、旧新 SHA 关联网关与 Gitea 接收日志。先确认对应 receive-pack/钩子任务已经结束;单次旧 SHA 查询不能作为结束证明。
|
||||
3. 核对仓库ID、owner/name、私有、非镜像、main 及对象可达性。精确目标SHA且验证完成可确认 SUCCEEDED;确认任务停止且 main 仍为旧 SHA、未发生引用变更时才可人工标记 FAILED。main 为其他 SHA 或身份变化继续待核对,不改历史、不重推。
|
||||
4. 在维护事务中同步更新操作 status/result_commit_sha/last_error_code/updated_at 和项目 repo_write_status=IDLE;必须校验项目只有这一条待核对操作,不能解除其他操作的保护。留存审批/排查依据和恢复审计,不能只执行孤立的 IDLE 更新。
|
||||
5. 重启恢复进程、恢复 worker,验证网页及本地正常写入后再开放 Git。不自动发布社区。
|
||||
|
||||
回滚先关闭 Git 功能并排空接收请求,核对处理中操作后再回退应用。旧版本不能消费未解决的 GIT_PUSH;未解决前保持维护停写。保留新增表/列/凭据/审计、仓库内容及 main 保护,不删对象或撤销保护。功能关闭后仍可在网站查询、撤销 Git Token。
|
||||
|
||||
+29
-1
@@ -1,6 +1,6 @@
|
||||
# 数据库初始化
|
||||
|
||||
业务数据库名为 `vibemake`,基础业务包含以下四张表;完整部署还需执行 [管理端增量 SQL](admin-database.md),新增迁移任务与仓库写入操作表。由人工执行;应用不运行迁移或自动建表。以下 SQL 面向 MySQL 8.0.13 及以上(URDF JSON 默认值使用表达式)。
|
||||
业务数据库名为 `vibemake`,基础业务包含以下四张表及项目 Git 凭据表;完整部署还需执行 [管理端增量 SQL](admin-database.md),新增迁移任务与仓库写入操作表。由人工执行;应用不运行迁移或自动建表。以下 SQL 面向 MySQL 8.0.13 及以上(URDF JSON 默认值使用表达式)。
|
||||
|
||||
```sql
|
||||
CREATE DATABASE IF NOT EXISTS vibemake CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
|
||||
@@ -310,3 +310,31 @@ WHERE LEFT(cover_url, 16) = '/project-covers/'
|
||||
```
|
||||
|
||||
最后查询必须为空,随后验证旧格式地址并回退应用,再恢复写入。旧版不识别相对路径,禁止只回退代码而跳过数据转换。
|
||||
|
||||
|
||||
## 项目 Git 凭据
|
||||
|
||||
HTTPS Git 功能默认关闭。已有数据库在启用前手工执行下列建表,以及 [仓库写入记录扩展](admin-database.md#https-git-写入记录扩展)。不自动迁库;回滚保留表和审计记录。所有时间为上海时间。
|
||||
|
||||
```sql
|
||||
CREATE TABLE project_git_token (
|
||||
id BIGINT NOT NULL AUTO_INCREMENT COMMENT '内部主键,不对用户端暴露',
|
||||
token_code CHAR(32) CHARACTER SET ascii COLLATE ascii_bin NOT NULL COMMENT 'Git凭据唯一业务编码',
|
||||
project_code CHAR(32) NOT NULL COMMENT '绑定的项目业务编码',
|
||||
owner_user_id BIGINT NOT NULL COMMENT '所属用户内部标识,仅内部关联',
|
||||
token_name VARCHAR(64) NOT NULL COMMENT '用户填写的凭据名称',
|
||||
token_hash CHAR(64) CHARACTER SET ascii COLLATE ascii_bin NOT NULL COMMENT '凭据明文的SHA256摘要,不保存明文',
|
||||
permission VARCHAR(16) NOT NULL COMMENT '权限 READ_ONLY只读 READ_WRITE读写',
|
||||
session_version BIGINT NOT NULL COMMENT '签发时账号会话版本,用于账号变更后失效',
|
||||
expires_at DATETIME(3) NOT NULL COMMENT '过期时间,上海时间',
|
||||
revoked_at DATETIME(3) NULL COMMENT '撤销时间,未撤销为空',
|
||||
last_used_at DATETIME(3) NULL COMMENT '最近成功认证时间,上海时间',
|
||||
created_at DATETIME(3) NOT NULL COMMENT '创建时间,上海时间',
|
||||
updated_at DATETIME(3) NOT NULL COMMENT '最近更新时间,上海时间',
|
||||
PRIMARY KEY (id),
|
||||
UNIQUE KEY uk_git_token_code (token_code),
|
||||
UNIQUE KEY uk_git_token_hash (token_hash),
|
||||
KEY idx_git_token_project_owner (project_code, owner_user_id, created_at)
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4
|
||||
COLLATE=utf8mb4_unicode_ci COMMENT='项目级Git访问凭据';
|
||||
```
|
||||
|
||||
@@ -18,6 +18,7 @@
|
||||
- [项目 URDF 文件夹](#项目-urdf-文件夹)
|
||||
- [目录和文件读取](#目录和文件读取)
|
||||
- [文件编辑与上传提交](#文件编辑与上传提交)
|
||||
- [HTTPS Git 与本地开发](#https-git-与本地开发)
|
||||
- [社区发布和撤回](#社区发布和撤回)
|
||||
- [社区详情和 Fork](#社区详情和-fork)
|
||||
- [评论和点赞](#评论和点赞)
|
||||
@@ -128,6 +129,10 @@ Git SHA 使用完整的 40 或 64 位十六进制字符串,不使用缩写。
|
||||
| GET | `/api/workspace/projects/{projectCode}/tree` | 是 | Tree |
|
||||
| GET | `/api/workspace/projects/{projectCode}/files` | 是 | ProjectFile 或下载二进制 |
|
||||
| GET | `/api/workspace/projects/{projectCode}/archive` | 是 | 当前或指定版本的 ZIP、TAR.GZ 仓库归档 |
|
||||
| GET | `/api/workspace/projects/{projectCode}/git` | 是 | GitConnection |
|
||||
| GET | `/api/workspace/projects/{projectCode}/git/tokens` | 是 | PageResult<GitTokenView> |
|
||||
| POST | `/api/workspace/projects/{projectCode}/git/tokens` | 是 | credential 和仅返回一次的 token |
|
||||
| DELETE | `/api/workspace/projects/{projectCode}/git/tokens/{tokenCode}` | 是 | null |
|
||||
| POST | `/api/workspace/projects/{projectCode}/commits` | 是 | Commit |
|
||||
| GET | `/api/workspace/projects/{projectCode}/commits` | 是 | PageResult<HistoryEntry> |
|
||||
| POST | `/api/workspace/projects/{projectCode}/uploads/commit` | 是 | Commit |
|
||||
@@ -1625,3 +1630,106 @@ export default {
|
||||
Fork 每用户每分钟 10 次、URDF 批次签名 20 次、封面签名 30 次,均为独立请求尝试次数;一个 URDF 文件夹的一次批次签名算一次,不按文件数计。Fork 同时执行仍最多 2 个,文件读取/目录/历史/提交/上传合计最多 4 个。普通 JSON 256 KiB、URDF 清单 2 MiB、提交 JSON 64 MiB,原有单文件和总内容约束继续保留。全部规则和可配置变量见 [部署说明](admin-deployment.md#redis-安全与资源保护)。
|
||||
|
||||
登录和签名响应显式禁止缓存。CORS 新增暴露 Retry-After,不启用跨域 Cookie;正文、Token、邮箱及验证码答案不得写入前端日志。
|
||||
|
||||
## HTTPS Git 与本地开发
|
||||
|
||||
Git 网关默认关闭,开启后仅本人工作区可 clone/fetch/pull/push;社区项目先 Fork。网站身份沿用邮箱加图片验证码,当前不验证邮箱归属,Git Token 继承此身份校验边界。普通用户不需要 Gitea 账号。
|
||||
|
||||
### 接口清单与数据
|
||||
|
||||
以下接口均携带网站 `Authorization: Bearer <平台 Token>`,仅所有者可访问。所有成功响应仍为平台 ApiResponse,均设置 `Cache-Control: no-store`;Git 协议请求使用独立认证。
|
||||
|
||||
| 接口 | 作用 |
|
||||
| --- | --- |
|
||||
| `GET /api/workspace/projects/{projectCode}/git` | 获取克隆信息及写入状态 |
|
||||
| `GET /api/workspace/projects/{projectCode}/git/tokens?page=1&size=20` | 分页查询 Git 凭据元数据 |
|
||||
| `POST /api/workspace/projects/{projectCode}/git/tokens` | 创建凭据,明文只返回一次 |
|
||||
| `DELETE /api/workspace/projects/{projectCode}/git/tokens/{tokenCode}` | 撤销凭据,重复撤销成功 |
|
||||
|
||||
克隆信息 data:
|
||||
|
||||
```json
|
||||
{
|
||||
"cloneUrl": "https://git.example.com/git/projects/7m4k9x2p8q6r.git",
|
||||
"username": "111111111111",
|
||||
"defaultBranch": "main",
|
||||
"headCommitSha": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
|
||||
"publishedCommitSha": null,
|
||||
"writeStatus": "IDLE",
|
||||
"canPush": true
|
||||
}
|
||||
```
|
||||
|
||||
`canPush` 表示当前项目状态、main 保护及技术账号配置允许写入,不保证后续请求一定成功;仍需读写权限的 Git Token。`writeStatus=WRITING` 表示正在写入,`REVIEW` 表示结果待核对。仅刷新页面即可看到已推送内容,不使用项目 updatedAt 判断版本。
|
||||
|
||||
创建请求:
|
||||
|
||||
```json
|
||||
{"name":"我的笔记本","scope":"READ_WRITE","expiresInDays":90}
|
||||
```
|
||||
|
||||
name 必填、非空、最多64字符;scope 为 READ_ONLY 或 READ_WRITE;expiresInDays 省略默认90,允许1~365。每项目最多10个有效凭据。读写凭据创建前会核验或初始化 main 保护;配置不兼容返回错误,不覆盖已有规则。首次签发会由后端管理账号给独立技术账号授予该仓库 write 权限,用户侧权限仍由项目 Token 决定。
|
||||
|
||||
创建成功 data 为 `{ "credential": GitTokenView, "token": "vbg_..." }`;列表 data 为 `PageResult<GitTokenView>`。GitTokenView 只有 tokenCode、name、scope、status、createdAt、expiresAt、revokedAt、lastUsedAt。status 为 ACTIVE、EXPIRED、REVOKED(包括账号会话版本变化失效)。时间沿用本文上海时间约定。最近使用时间最多每5分钟写回一次。
|
||||
|
||||
明文仅在创建时展示一次,不能再次查询;遗失后撤销并新建。列表、日志、数据库均不保存或返回明文。普通退出网站不撤销 Git Token;撤销、过期、禁用账号或账号 sessionVersion 改变后,新 Git/LFS 请求会被拒绝。已经受理的传输不保证立即中止或回滚。
|
||||
|
||||
### 网站交互与本地使用
|
||||
|
||||
工作区提供“克隆与本地开发”入口,显示克隆 URL、用户名、main SHA、发布 SHA 和写入状态;提供 Token 创建、列表、撤销和一次性明文展示。明文只保存在当前展示状态,不写 localStorage、不放克隆 URL、查询参数或日志。用户可用操作系统凭据管理器保存。
|
||||
|
||||
Git 使用 HTTP Basic:用户名为网站 userCode,密码为项目 Git Token;网站 Bearer Token 不能用于 Git,Git Token 不能用于普通 `/api`。同域名不同项目必须配置 HTTP 路径级凭据:
|
||||
|
||||
```bash
|
||||
git lfs install
|
||||
git -c credential.useHttpPath=true clone --single-branch --branch main <cloneUrl>
|
||||
cd <本地仓库目录>
|
||||
git config credential.useHttpPath true
|
||||
|
||||
git add .
|
||||
git commit -m "更新项目"
|
||||
git push origin main
|
||||
```
|
||||
|
||||
支持标准 Git v0/v2 读取、main 正常快进更新和合并提交,保留本地提交作者、邮箱、时间、签名及 SHA。禁止强推、删除 main、创建其他分支或标签、多引用推送、push-options、signed-push 扩展;普通签名提交支持。仓库已有其他分支和标签可以读取,本期不清理历史数据。LFS Basic Transfer 支持实体上传、下载和 verify;不支持 LFS 文件锁,locks/verify 返回空集合,锁创建/删除返回501。不递归下载子模块,独立绑定的 OSS 封面、URDF、STEP、模型等资源不会自动进入仓库。
|
||||
|
||||
本地 push 后重新请求工作区 tree、files、commits 显示最新内容;社区仍固定 publishedCommitSha,需用户明确点击发布。网页编辑基准落后继续返回409,保留用户输入;本地 main 落后时先 pull、合并再 push。REVIEW 期间阻止仓库写入及发布,后台只读核对,不自动重推。网络断开或超时不能视为失败,先刷新核对。
|
||||
|
||||
前端创建示例(复用本文 request):
|
||||
|
||||
```ts
|
||||
interface GitTokenView {
|
||||
tokenCode: string;
|
||||
name: string;
|
||||
scope: 'READ_ONLY' | 'READ_WRITE';
|
||||
status: 'ACTIVE' | 'EXPIRED' | 'REVOKED';
|
||||
createdAt: string;
|
||||
expiresAt: string;
|
||||
revokedAt: string | null;
|
||||
lastUsedAt: string | null;
|
||||
}
|
||||
async function createGitToken(projectCode: string, token: string) {
|
||||
return request<{credential: GitTokenView; token: string}>(
|
||||
`/workspace/projects/${encodeURIComponent(projectCode)}/git/tokens`, {
|
||||
method: 'POST', token,
|
||||
body: {name: '我的笔记本', scope: 'READ_WRITE', expiresInDays: 90},
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
协议路径 `/git/**` 成功响应为 Git pkt-line/pack 或 LFS 标准结构;错误使用 Git 文本或 LFS `{message}`,不使用平台包装。401 含 Basic challenge,429 含 Retry-After。不要用浏览器平台 JSON 请求封装访问协议路径。
|
||||
|
||||
| 错误码/状态 | 处理 |
|
||||
| --- | --- |
|
||||
| GIT_UNAVAILABLE / 503 | 功能关闭或依赖不可用,保留网站编辑入口;仍可查询、撤销已有凭据 |
|
||||
| GIT_POLICY_UNAVAILABLE / 503 | 技术账号或 main 保护不兼容,联系运维,不放宽规则 |
|
||||
| CONFLICT / 409 | 有效凭据已达上限,先撤销旧凭据 |
|
||||
| GIT_UNAUTHORIZED / 401 | 用户名或项目 Git Token 无效、过期或撤销 |
|
||||
| GIT_READ_ONLY / 403 | 使用读写 Token 后再推送或上传 LFS |
|
||||
| GIT_COMMAND_REJECTED / 400 | 仅推送 main,移除不支持的引用/扩展 |
|
||||
| GIT_STALE_MAIN / 409 | 先拉取、合并,再推送 |
|
||||
| GIT_BUSY / 429 | 用户限流、IP认证失败限流或并发已满,稍后手动重试 |
|
||||
| GIT_TRANSFER_TOO_LARGE、GIT_METADATA_TOO_LARGE、LFS_TOO_LARGE / 413 | 缩小请求或联系运维核对配置 |
|
||||
| GIT_TIMEOUT / 504、GIT_UPSTREAM_ERROR / 502、REMOTE_RESULT_UNKNOWN / 409或503 | 涉及 push 时先刷新工作区及历史,结果未确认前不能重复写入 |
|
||||
|
||||
默认 pack 5 GiB、LFS 单对象1 GiB、元数据8 MiB、batch最多500个对象;每次 Git/LFS 请求预算10分钟,含请求及响应流。读取/push/LFS 每实例并发4/2/8;每用户每分钟 Git120次、LFS600次,每IP每分钟最多20次认证失败。均可配置,独立于 JSON、multipart、在线50 MiB文件读取和归档下载限制。中途超限、超时或断开会终止传输;不提供任务、进度查询或断点续传接口。
|
||||
|
||||
@@ -0,0 +1,26 @@
|
||||
#!/usr/bin/env python3
|
||||
"""使用随机测试用户/仓库验证真实 HTTPS Git 接入;不修改已有项目。"""
|
||||
import os
|
||||
from pathlib import Path
|
||||
import shlex
|
||||
import subprocess
|
||||
|
||||
root = Path(__file__).resolve().parent.parent
|
||||
environment = os.environ.copy()
|
||||
config = root / '.env'
|
||||
if config.exists():
|
||||
for line in config.read_text().splitlines():
|
||||
if not line or line.startswith('#') or '=' not in line:
|
||||
continue
|
||||
key, value = line.split('=', 1)
|
||||
parsed = shlex.split(value)
|
||||
if parsed:
|
||||
environment[key] = parsed[0]
|
||||
if not environment.get('GITEA_TOKEN'):
|
||||
raise SystemExit('请配置 GITEA_BASE_URL、GITEA_ORGANIZATION 与可创建测试用户的 GITEA_TOKEN')
|
||||
if environment.get('DB_URL', '').startswith('jdbc:mysql:'):
|
||||
environment.setdefault('GIT_GATEWAY_MYSQL', 'true')
|
||||
environment.update(VIBEMAKE_LIVE_GITEA='true', AI_ENABLED='false', VIBEMAKE_AI_ENABLED='false', OSS_ENABLED='false')
|
||||
raise SystemExit(subprocess.call([
|
||||
'mvn', '-q', '-Dtest=GitGatewayLiveTest', '-Dsurefire.failIfNoSpecifiedTests=false', 'test'
|
||||
], cwd=root, env=environment))
|
||||
@@ -126,6 +126,7 @@ class AdminWorkflowTest {
|
||||
"JSON NOT NULL DEFAULT (JSON_ARRAY())",
|
||||
"CLOB NOT NULL DEFAULT '[]'")
|
||||
.replace(" CHARACTER SET utf8mb4 COLLATE utf8mb4_bin", "")
|
||||
.replace(" CHARACTER SET ascii COLLATE ascii_bin", "")
|
||||
.replaceAll(" COMMENT '[^']*'", "")
|
||||
.replaceAll("\\) ENGINE=InnoDB[^;]*;", ");"));
|
||||
}
|
||||
|
||||
+74
@@ -0,0 +1,74 @@
|
||||
package com.linkerbot.vibemake.api.controller;
|
||||
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
import com.linkerbot.vibemake.common.BusinessException;
|
||||
import com.linkerbot.vibemake.security.GitAuthenticationFilter;
|
||||
import com.linkerbot.vibemake.service.git.*;
|
||||
|
||||
import jakarta.servlet.http.*;
|
||||
|
||||
import lombok.RequiredArgsConstructor;
|
||||
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
import org.springframework.web.bind.annotation.*;
|
||||
|
||||
import java.io.IOException;
|
||||
|
||||
@io.swagger.v3.oas.annotations.Hidden
|
||||
@RestController
|
||||
@RequiredArgsConstructor
|
||||
public class GitProtocolController {
|
||||
private static final Logger log = LoggerFactory.getLogger(GitProtocolController.class);
|
||||
private final GitGateway git;
|
||||
private final LfsGateway lfs;
|
||||
private final ObjectMapper json;
|
||||
|
||||
@RequestMapping("/git/projects/{projectCode}.git/{*endpoint}")
|
||||
public void protocol(
|
||||
@PathVariable String projectCode,
|
||||
@PathVariable String endpoint,
|
||||
HttpServletRequest request,
|
||||
HttpServletResponse response)
|
||||
throws IOException {
|
||||
GitAccess access = (GitAccess) request.getAttribute(GitAuthenticationFilter.ACCESS);
|
||||
try {
|
||||
if (access == null || !projectCode.equals(access.projectCode()))
|
||||
throw GitTokens.unauthorized();
|
||||
String path = endpoint.startsWith("/") ? endpoint.substring(1) : endpoint;
|
||||
if (path.startsWith("info/lfs/actions/")) {
|
||||
LfsGateway.Action action =
|
||||
(LfsGateway.Action) request.getAttribute(GitAuthenticationFilter.ACTION);
|
||||
if (action == null) throw GitTokens.unauthorized();
|
||||
lfs.action(action, request, response);
|
||||
} else if (path.startsWith("info/lfs/"))
|
||||
lfs.serve(access, path.substring(9), request, response);
|
||||
else git.serve(access, path, request, response);
|
||||
} catch (BusinessException error) {
|
||||
log.warn(
|
||||
"Git 协议请求拒绝 projectCode={} code={} status={}",
|
||||
projectCode,
|
||||
error.code(),
|
||||
error.status());
|
||||
fail(request, response, error);
|
||||
} catch (Exception error) {
|
||||
log.warn(
|
||||
"Git 协议请求中断 projectCode={} errorType={}",
|
||||
projectCode,
|
||||
error.getClass().getSimpleName());
|
||||
fail(
|
||||
request,
|
||||
response,
|
||||
new BusinessException(502, "GIT_UPSTREAM_ERROR", "Git 传输中断,请刷新工作区核对结果"));
|
||||
}
|
||||
}
|
||||
|
||||
private void fail(
|
||||
HttpServletRequest request, HttpServletResponse response, BusinessException error) {
|
||||
try {
|
||||
GitAuthenticationFilter.failure(request, response, json, error);
|
||||
} catch (IOException ignored) {
|
||||
/* 已断开连接不再进入平台 JSON 异常处理器。 */
|
||||
}
|
||||
}
|
||||
}
|
||||
+55
@@ -0,0 +1,55 @@
|
||||
package com.linkerbot.vibemake.api.controller;
|
||||
|
||||
import com.linkerbot.vibemake.bean.GitContracts.*;
|
||||
import com.linkerbot.vibemake.common.*;
|
||||
import com.linkerbot.vibemake.security.CurrentUser;
|
||||
import com.linkerbot.vibemake.service.git.*;
|
||||
|
||||
import jakarta.servlet.http.HttpServletResponse;
|
||||
import jakarta.validation.Valid;
|
||||
|
||||
import lombok.RequiredArgsConstructor;
|
||||
|
||||
import org.springframework.web.bind.annotation.*;
|
||||
|
||||
@RestController
|
||||
@RequiredArgsConstructor
|
||||
@RequestMapping("/api/workspace/projects/{projectCode}/git")
|
||||
public class ProjectGitController {
|
||||
private final GitGateway gateway;
|
||||
private final GitTokens tokens;
|
||||
|
||||
@ModelAttribute
|
||||
public void noStore(HttpServletResponse response) {
|
||||
response.setHeader("Cache-Control", "no-store");
|
||||
}
|
||||
|
||||
@GetMapping
|
||||
public ApiResponse<ConnectionView> connection(@PathVariable String projectCode) {
|
||||
return ApiResponse.success(gateway.connection(CurrentUser.required(), projectCode));
|
||||
}
|
||||
|
||||
@GetMapping("/tokens")
|
||||
public ApiResponse<PageResponse<TokenView>> list(
|
||||
@PathVariable String projectCode,
|
||||
@RequestParam(defaultValue = "1") long page,
|
||||
@RequestParam(defaultValue = "20") long size) {
|
||||
return ApiResponse.success(tokens.list(CurrentUser.required(), projectCode, page, size));
|
||||
}
|
||||
|
||||
@PostMapping("/tokens")
|
||||
public ApiResponse<IssuedToken> issue(
|
||||
@PathVariable String projectCode,
|
||||
@Valid @RequestBody TokenInput input,
|
||||
HttpServletResponse response) {
|
||||
response.setHeader("Cache-Control", "no-store");
|
||||
return ApiResponse.success(tokens.issue(CurrentUser.required(), projectCode, input));
|
||||
}
|
||||
|
||||
@DeleteMapping("/tokens/{tokenCode}")
|
||||
public ApiResponse<Void> revoke(
|
||||
@PathVariable String projectCode, @PathVariable String tokenCode) {
|
||||
tokens.revoke(CurrentUser.required(), projectCode, tokenCode);
|
||||
return ApiResponse.success(null);
|
||||
}
|
||||
}
|
||||
@@ -98,6 +98,22 @@ vibemake:
|
||||
gitea:
|
||||
connect-timeout: 5s
|
||||
request-timeout: 30s
|
||||
git:
|
||||
enabled: ${GIT_ENABLED:true}
|
||||
public-base-url: ${GIT_PUBLIC_BASE_URL:}
|
||||
upstream-username: ${GIT_UPSTREAM_USERNAME:}
|
||||
upstream-token: ${GIT_UPSTREAM_TOKEN:}
|
||||
timeout: ${GIT_REQUEST_TIMEOUT:10m}
|
||||
max-pack-bytes: ${GIT_MAX_PACK_BYTES:5368709120}
|
||||
max-lfs-object-bytes: ${GIT_MAX_LFS_OBJECT_BYTES:1073741824}
|
||||
max-metadata-bytes: ${GIT_MAX_METADATA_BYTES:8388608}
|
||||
max-batch-objects: ${GIT_MAX_BATCH_OBJECTS:500}
|
||||
read-concurrent: ${GIT_READ_CONCURRENT:4}
|
||||
push-concurrent: ${GIT_PUSH_CONCURRENT:2}
|
||||
lfs-concurrent: ${GIT_LFS_CONCURRENT:8}
|
||||
git-requests-per-minute: ${GIT_REQUESTS_PER_MINUTE:120}
|
||||
lfs-requests-per-minute: ${GIT_LFS_REQUESTS_PER_MINUTE:600}
|
||||
auth-failures-per-minute: ${GIT_AUTH_FAILURES_PER_MINUTE:20}
|
||||
archive:
|
||||
max-bytes: ${ARCHIVE_MAX_BYTES:1073741824}
|
||||
timeout: ${ARCHIVE_TIMEOUT:5m}
|
||||
|
||||
@@ -0,0 +1,728 @@
|
||||
package com.linkerbot.vibemake.api;
|
||||
|
||||
import static org.assertj.core.api.Assertions.*;
|
||||
import static org.mockito.ArgumentMatchers.*;
|
||||
import static org.mockito.Mockito.*;
|
||||
import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.*;
|
||||
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.*;
|
||||
|
||||
import com.fasterxml.jackson.databind.*;
|
||||
import com.linkerbot.vibemake.common.SecurityHashes;
|
||||
import com.linkerbot.vibemake.service.git.*;
|
||||
import com.linkerbot.vibemake.service.write.RepositoryWrites;
|
||||
import com.sun.net.httpserver.HttpServer;
|
||||
|
||||
import org.junit.jupiter.api.*;
|
||||
import org.springframework.beans.factory.annotation.Autowired;
|
||||
import org.springframework.boot.test.autoconfigure.web.servlet.*;
|
||||
import org.springframework.boot.test.context.SpringBootTest;
|
||||
import org.springframework.context.annotation.Import;
|
||||
import org.springframework.jdbc.core.JdbcTemplate;
|
||||
import org.springframework.test.context.*;
|
||||
import org.springframework.test.context.bean.override.mockito.MockitoBean;
|
||||
import org.springframework.test.context.bean.override.mockito.MockitoSpyBean;
|
||||
import org.springframework.test.web.servlet.*;
|
||||
import org.springframework.test.web.servlet.request.MockHttpServletRequestBuilder;
|
||||
|
||||
import java.io.*;
|
||||
import java.net.*;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.util.*;
|
||||
import java.util.concurrent.atomic.AtomicReference;
|
||||
|
||||
@SpringBootTest(
|
||||
properties = {
|
||||
"spring.datasource.url=jdbc:h2:mem:gitapi;MODE=MySQL;DB_CLOSE_DELAY=-1",
|
||||
"spring.datasource.username=sa",
|
||||
"spring.datasource.password=",
|
||||
"spring.datasource.driver-class-name=org.h2.Driver",
|
||||
"vibemake.ai.enabled=false",
|
||||
"vibemake.oss.enabled=false",
|
||||
"vibemake.limits.enabled=false",
|
||||
"vibemake.gitea.owner=vibemake",
|
||||
"vibemake.repository-write.worker-enabled=false",
|
||||
"vibemake.git.enabled=true",
|
||||
"vibemake.git.public-base-url=https://git.example/git",
|
||||
"vibemake.git.upstream-username=fixture-bot",
|
||||
"vibemake.git.upstream-token=fixture-only-secret",
|
||||
"vibemake.git.auth-failures-per-minute=1000",
|
||||
"logging.level.root=WARN"
|
||||
})
|
||||
@ActiveProfiles("test")
|
||||
@AutoConfigureMockMvc(print = MockMvcPrint.NONE)
|
||||
@Import(WorkflowTest.Config.class)
|
||||
class GitApiTest {
|
||||
static HttpServer server;
|
||||
static AtomicReference<String> mode = new AtomicReference<>("ok");
|
||||
static AtomicReference<byte[]> received = new AtomicReference<>();
|
||||
static java.util.concurrent.CountDownLatch pushEntered;
|
||||
static java.util.concurrent.CountDownLatch pushRelease;
|
||||
static String upstream;
|
||||
static final byte[] OBJECT = "LFS binary 中文\0fixture".getBytes(StandardCharsets.UTF_8);
|
||||
static final String OID = java.util.HexFormat.of().formatHex(hash(OBJECT));
|
||||
|
||||
private static byte[] hash(byte[] bytes) {
|
||||
try {
|
||||
return java.security.MessageDigest.getInstance("SHA-256").digest(bytes);
|
||||
} catch (Exception error) {
|
||||
throw new IllegalStateException(error);
|
||||
}
|
||||
}
|
||||
|
||||
@DynamicPropertySource
|
||||
static void upstream(DynamicPropertyRegistry registry) throws Exception {
|
||||
server = HttpServer.create(new InetSocketAddress("127.0.0.1", 0), 0);
|
||||
upstream = "http://127.0.0.1:" + server.getAddress().getPort();
|
||||
server.createContext(
|
||||
"/",
|
||||
exchange -> {
|
||||
String path = exchange.getRequestURI().getPath();
|
||||
byte[] result;
|
||||
String type = "application/x-git-upload-pack-advertisement";
|
||||
if (path.endsWith("objects/batch")) {
|
||||
type = LfsGateway.MEDIA;
|
||||
JsonNode input = new ObjectMapper().readTree(exchange.getRequestBody());
|
||||
String op = input.path("operation").asText();
|
||||
String prefix = path.substring(0, path.length() - "objects/batch".length());
|
||||
String href =
|
||||
upstream
|
||||
+ prefix
|
||||
+ "objects/"
|
||||
+ OID
|
||||
+ (op.equals("upload") ? "/" + OBJECT.length : "");
|
||||
if (mode.get().equals("evil")) href = "https://untrusted.invalid/" + OID;
|
||||
result =
|
||||
("{\"transfer\":\"basic\",\"objects\":[{\"oid\":\""
|
||||
+ OID
|
||||
+ "\",\"size\":"
|
||||
+ OBJECT.length
|
||||
+ ",\"actions\":{\""
|
||||
+ op
|
||||
+ "\":{\"href\":\""
|
||||
+ href
|
||||
+ "\",\"header\":{\"Authorization\":\"Bearer"
|
||||
+ " upstream-secret\"}}}}]}")
|
||||
.getBytes();
|
||||
} else if (path.contains("/info/lfs/objects/")) {
|
||||
type = "application/octet-stream";
|
||||
if (exchange.getRequestMethod().equals("PUT")) {
|
||||
received.set(exchange.getRequestBody().readAllBytes());
|
||||
result = new byte[0];
|
||||
} else result = OBJECT;
|
||||
} else if (path.endsWith("git-receive-pack")) {
|
||||
received.set(exchange.getRequestBody().readAllBytes());
|
||||
if (mode.get().equals("block")) {
|
||||
pushEntered.countDown();
|
||||
try {
|
||||
pushRelease.await(15, java.util.concurrent.TimeUnit.SECONDS);
|
||||
} catch (InterruptedException error) {
|
||||
Thread.currentThread().interrupt();
|
||||
}
|
||||
}
|
||||
type = "application/x-git-receive-pack-result";
|
||||
result =
|
||||
mode.get().equals("broken")
|
||||
? "not status".getBytes()
|
||||
: packets(
|
||||
"unpack ok\n",
|
||||
"ng refs/heads/main non-fast-forward\n",
|
||||
null);
|
||||
} else
|
||||
result =
|
||||
packets(
|
||||
"# service=git-upload-pack\n",
|
||||
null,
|
||||
"a".repeat(40) + " refs/heads/main\0multi_ack\n",
|
||||
null);
|
||||
exchange.getResponseHeaders().set("Content-Type", type);
|
||||
exchange.sendResponseHeaders(200, result.length == 0 ? -1 : result.length);
|
||||
exchange.getResponseBody().write(result);
|
||||
exchange.close();
|
||||
});
|
||||
server.start();
|
||||
registry.add("vibemake.gitea.base-url", () -> upstream);
|
||||
}
|
||||
|
||||
@AfterAll
|
||||
static void closeUpstream() {
|
||||
server.stop(0);
|
||||
}
|
||||
|
||||
@Autowired MockMvc mvc;
|
||||
@Autowired ObjectMapper json;
|
||||
@Autowired JdbcTemplate jdbc;
|
||||
@MockitoSpyBean FakeGiteaClient gitea;
|
||||
@Autowired RepositoryWrites writes;
|
||||
@Autowired com.linkerbot.vibemake.common.state.TransientStateStore state;
|
||||
@MockitoBean GitPolicy policy;
|
||||
|
||||
@BeforeEach
|
||||
void schema() throws Exception {
|
||||
SchemaSupport.reset(jdbc);
|
||||
reset(gitea);
|
||||
gitea.reset();
|
||||
mode.set("ok");
|
||||
received.set(null);
|
||||
pushEntered = new java.util.concurrent.CountDownLatch(1);
|
||||
pushRelease = new java.util.concurrent.CountDownLatch(1);
|
||||
}
|
||||
|
||||
static byte[] packets(String... lines) throws IOException {
|
||||
ByteArrayOutputStream bytes = new ByteArrayOutputStream();
|
||||
for (String line : lines)
|
||||
if (line == null) bytes.write("0000".getBytes());
|
||||
else GitIo.packet(bytes, line.getBytes(StandardCharsets.UTF_8));
|
||||
return bytes.toByteArray();
|
||||
}
|
||||
|
||||
JsonNode ok(MockHttpServletRequestBuilder request) throws Exception {
|
||||
return json.readTree(
|
||||
mvc.perform(request)
|
||||
.andExpect(status().isOk())
|
||||
.andReturn()
|
||||
.getResponse()
|
||||
.getContentAsByteArray())
|
||||
.path("data");
|
||||
}
|
||||
|
||||
MockHttpServletRequestBuilder body(MockHttpServletRequestBuilder request, Object value)
|
||||
throws Exception {
|
||||
return request.contentType("application/json").content(json.writeValueAsBytes(value));
|
||||
}
|
||||
|
||||
MockHttpServletRequestBuilder auth(MockHttpServletRequestBuilder request, String token) {
|
||||
return request.header("Authorization", "Bearer " + token);
|
||||
}
|
||||
|
||||
record Account(String token, String username) {}
|
||||
|
||||
Account login(String suffix) throws Exception {
|
||||
String captcha = ok(get("/api/auth/captcha")).path("captchaCode").asText();
|
||||
JsonNode response =
|
||||
ok(
|
||||
body(
|
||||
post("/api/auth/login"),
|
||||
Map.of(
|
||||
"email",
|
||||
suffix + "@example.invalid",
|
||||
"captchaCode",
|
||||
captcha,
|
||||
"captchaAnswer",
|
||||
"ABCDE")));
|
||||
return new Account(
|
||||
response.path("token").asText(), response.path("user").path("userCode").asText());
|
||||
}
|
||||
|
||||
String create(Account account) throws Exception {
|
||||
return ok(body(
|
||||
auth(post("/api/workspace/projects"), account.token()),
|
||||
Map.of("name", "Git 项目", "projectType", "GENERAL")))
|
||||
.path("projectCode")
|
||||
.asText();
|
||||
}
|
||||
|
||||
JsonNode issue(Account account, String code, String scope) throws Exception {
|
||||
return ok(
|
||||
body(
|
||||
auth(post(api(code) + "/tokens"), account.token()),
|
||||
Map.of("name", "笔记本", "scope", scope)));
|
||||
}
|
||||
|
||||
String api(String code) {
|
||||
return "/api/workspace/projects/" + code + "/git";
|
||||
}
|
||||
|
||||
String protocol(String code) {
|
||||
return "/git/projects/" + code + ".git";
|
||||
}
|
||||
|
||||
MockHttpServletRequestBuilder basic(
|
||||
MockHttpServletRequestBuilder request, Account account, String secret) {
|
||||
return request.header(
|
||||
"Authorization",
|
||||
"Basic "
|
||||
+ Base64.getEncoder()
|
||||
.encodeToString(
|
||||
(account.username() + ":" + secret)
|
||||
.getBytes(StandardCharsets.UTF_8)));
|
||||
}
|
||||
|
||||
@Test
|
||||
void credentialsAreProjectBoundHashedAndSeparateFromWebsiteSessions() throws Exception {
|
||||
Account owner = login("git-owner"), other = login("git-other");
|
||||
String code = create(owner), second = create(owner);
|
||||
JsonNode issued = issue(owner, code, "READ_ONLY");
|
||||
String secret = issued.path("token").asText();
|
||||
assertThat(secret).startsWith("vbg_");
|
||||
assertThat(jdbc.queryForObject("SELECT token_hash FROM project_git_token", String.class))
|
||||
.isEqualTo(SecurityHashes.sha256(secret));
|
||||
String list = ok(auth(get(api(code) + "/tokens"), owner.token())).toString();
|
||||
assertThat(list).doesNotContain(secret, "tokenHash", "ownerUserId", "\"id\"");
|
||||
mvc.perform(
|
||||
basic(
|
||||
get(protocol(code) + "/info/refs")
|
||||
.param("service", "git-upload-pack"),
|
||||
owner,
|
||||
secret))
|
||||
.andExpect(status().isOk());
|
||||
mvc.perform(
|
||||
basic(
|
||||
get(protocol(second) + "/info/refs")
|
||||
.param("service", "git-upload-pack"),
|
||||
owner,
|
||||
secret))
|
||||
.andExpect(status().isNotFound());
|
||||
mvc.perform(
|
||||
basic(
|
||||
get(protocol(code) + "/info/refs")
|
||||
.param("service", "git-upload-pack"),
|
||||
other,
|
||||
secret))
|
||||
.andExpect(status().isUnauthorized());
|
||||
mvc.perform(auth(get(api(code)), secret)).andExpect(status().isUnauthorized());
|
||||
mvc.perform(
|
||||
auth(
|
||||
get(protocol(code) + "/info/refs")
|
||||
.param("service", "git-upload-pack"),
|
||||
owner.token()))
|
||||
.andExpect(status().isUnauthorized())
|
||||
.andExpect(
|
||||
header().string(
|
||||
"WWW-Authenticate",
|
||||
org.hamcrest.Matchers.containsString("Basic")));
|
||||
mvc.perform(
|
||||
basic(
|
||||
get(protocol(code) + "/info/refs")
|
||||
.param("service", "git-receive-pack"),
|
||||
owner,
|
||||
secret))
|
||||
.andExpect(status().isForbidden());
|
||||
mvc.perform(auth(get(api(code) + "/tokens"), other.token()))
|
||||
.andExpect(status().isNotFound());
|
||||
// 普通退出只清网页会话。
|
||||
ok(auth(post("/api/auth/logout"), owner.token()));
|
||||
mvc.perform(
|
||||
basic(
|
||||
get(protocol(code) + "/info/refs")
|
||||
.param("service", "git-upload-pack"),
|
||||
owner,
|
||||
secret))
|
||||
.andExpect(status().isOk());
|
||||
jdbc.update(
|
||||
"UPDATE sys_user SET session_version=session_version+1 WHERE user_code=?",
|
||||
owner.username());
|
||||
mvc.perform(
|
||||
basic(
|
||||
get(protocol(code) + "/info/refs")
|
||||
.param("service", "git-upload-pack"),
|
||||
owner,
|
||||
secret))
|
||||
.andExpect(status().isUnauthorized());
|
||||
}
|
||||
|
||||
@Test
|
||||
void expirationRevocationAndTokenQuota() throws Exception {
|
||||
Account owner = login("git-token-lifecycle");
|
||||
String code = create(owner);
|
||||
JsonNode first = issue(owner, code, "READ_WRITE");
|
||||
String tokenCode = first.path("credential").path("tokenCode").asText();
|
||||
for (int index = 1; index < 10; index++) issue(owner, code, "READ_ONLY");
|
||||
mvc.perform(
|
||||
body(
|
||||
auth(post(api(code) + "/tokens"), owner.token()),
|
||||
Map.of("name", "超过", "scope", "READ_ONLY")))
|
||||
.andExpect(status().isConflict());
|
||||
ok(auth(delete(api(code) + "/tokens/" + tokenCode), owner.token()));
|
||||
ok(auth(delete(api(code) + "/tokens/" + tokenCode), owner.token()));
|
||||
mvc.perform(
|
||||
basic(
|
||||
get(protocol(code) + "/info/refs")
|
||||
.param("service", "git-upload-pack"),
|
||||
owner,
|
||||
first.path("token").asText()))
|
||||
.andExpect(status().isUnauthorized());
|
||||
String replacement = issue(owner, code, "READ_ONLY").path("token").asText();
|
||||
jdbc.update("UPDATE project_git_token SET expires_at=DATEADD('DAY',-1,CURRENT_TIMESTAMP)");
|
||||
mvc.perform(
|
||||
basic(
|
||||
get(protocol(code) + "/info/refs")
|
||||
.param("service", "git-upload-pack"),
|
||||
owner,
|
||||
replacement))
|
||||
.andExpect(status().isUnauthorized());
|
||||
for (Map<String, Object> invalid :
|
||||
List.of(
|
||||
Map.<String, Object>of("name", " ", "scope", "READ_ONLY"),
|
||||
Map.<String, Object>of(
|
||||
"name", "x", "scope", "READ_ONLY", "expiresInDays", 366)))
|
||||
mvc.perform(body(auth(post(api(code) + "/tokens"), owner.token()), invalid))
|
||||
.andExpect(status().isBadRequest());
|
||||
}
|
||||
|
||||
@Test
|
||||
void lfsActionsHideUpstreamBindProjectAndRecheckRevocation() throws Exception {
|
||||
Account owner = login("git-lfs");
|
||||
String code = create(owner);
|
||||
JsonNode issued = issue(owner, code, "READ_WRITE");
|
||||
String secret = issued.path("token").asText();
|
||||
Map<String, Object> batch =
|
||||
Map.of(
|
||||
"operation",
|
||||
"upload",
|
||||
"objects",
|
||||
List.of(Map.of("oid", OID, "size", OBJECT.length)));
|
||||
MvcResult result =
|
||||
mvc.perform(
|
||||
basic(
|
||||
post(protocol(code) + "/info/lfs/objects/batch"),
|
||||
owner,
|
||||
secret)
|
||||
.contentType(LfsGateway.MEDIA)
|
||||
.content(json.writeValueAsBytes(batch)))
|
||||
.andExpect(status().isOk())
|
||||
.andReturn();
|
||||
String raw = result.getResponse().getContentAsString();
|
||||
assertThat(raw).doesNotContain(upstream, "upstream-secret", "fixture-only-secret");
|
||||
JsonNode action = json.readTree(raw).path("objects").get(0).path("actions").path("upload");
|
||||
String path = URI.create(action.path("href").asText()).getPath();
|
||||
String auth = action.path("header").path("Authorization").asText();
|
||||
mvc.perform(
|
||||
put(path)
|
||||
.header("Authorization", auth)
|
||||
.contentType("application/octet-stream")
|
||||
.content(OBJECT))
|
||||
.andExpect(status().isOk());
|
||||
assertThat(received.get()).isEqualTo(OBJECT);
|
||||
String readonly = issue(owner, code, "READ_ONLY").path("token").asText();
|
||||
mvc.perform(
|
||||
basic(post(protocol(code) + "/info/lfs/objects/batch"), owner, readonly)
|
||||
.contentType(LfsGateway.MEDIA)
|
||||
.content(json.writeValueAsBytes(batch)))
|
||||
.andExpect(status().isForbidden());
|
||||
mode.set("evil");
|
||||
mvc.perform(
|
||||
basic(post(protocol(code) + "/info/lfs/objects/batch"), owner, secret)
|
||||
.contentType(LfsGateway.MEDIA)
|
||||
.content(json.writeValueAsBytes(batch)))
|
||||
.andExpect(status().isBadGateway());
|
||||
ok(
|
||||
auth(
|
||||
delete(
|
||||
api(code)
|
||||
+ "/tokens/"
|
||||
+ issued.path("credential").path("tokenCode").asText()),
|
||||
owner.token()));
|
||||
mvc.perform(put(path).header("Authorization", auth).content(OBJECT))
|
||||
.andExpect(status().isUnauthorized());
|
||||
}
|
||||
|
||||
@Test
|
||||
void explicitGitRejectionUnlocksButUnknownStatusRequiresReview() throws Exception {
|
||||
Account owner = login("git-push");
|
||||
String code = create(owner);
|
||||
String secret = issue(owner, code, "READ_WRITE").path("token").asText();
|
||||
String old =
|
||||
ok(auth(get("/api/workspace/projects/" + code + "/tree"), owner.token()))
|
||||
.path("commitSha")
|
||||
.asText();
|
||||
byte[] body =
|
||||
packets(
|
||||
old + " " + "b".repeat(40) + " refs/heads/main\0report-status ofs-delta",
|
||||
null);
|
||||
mvc.perform(
|
||||
basic(post(protocol(code) + "/git-receive-pack"), owner, secret)
|
||||
.contentType("application/x-git-receive-pack-request")
|
||||
.content(body))
|
||||
.andExpect(status().isOk());
|
||||
assertThat(
|
||||
jdbc.queryForObject(
|
||||
"SELECT status FROM repository_write_operation", String.class))
|
||||
.isEqualTo("FAILED");
|
||||
assertThat(jdbc.queryForObject("SELECT repo_write_status FROM project", String.class))
|
||||
.isEqualTo("IDLE");
|
||||
mode.set("broken");
|
||||
mvc.perform(
|
||||
basic(post(protocol(code) + "/git-receive-pack"), owner, secret)
|
||||
.contentType("application/x-git-receive-pack-request")
|
||||
.content(body))
|
||||
.andExpect(status().isBadGateway());
|
||||
assertThat(jdbc.queryForObject("SELECT repo_write_status FROM project", String.class))
|
||||
.isEqualTo("REVIEW");
|
||||
jdbc.update(
|
||||
"UPDATE repository_write_operation SET"
|
||||
+ " next_check_at=DATEADD('SECOND',-1,CURRENT_TIMESTAMP),created_at=DATEADD('MINUTE',-11,CURRENT_TIMESTAMP)"
|
||||
+ " WHERE status='PROCESSING'");
|
||||
writes.reconcileDue();
|
||||
assertThat(
|
||||
jdbc.queryForObject(
|
||||
"SELECT status FROM repository_write_operation WHERE"
|
||||
+ " status<>'FAILED'",
|
||||
String.class))
|
||||
.isEqualTo("NEEDS_REVIEW");
|
||||
assertThat(jdbc.queryForObject("SELECT repo_write_status FROM project", String.class))
|
||||
.isEqualTo("REVIEW");
|
||||
doReturn("b".repeat(40)).when(gitea).head(any());
|
||||
jdbc.update(
|
||||
"UPDATE repository_write_operation SET"
|
||||
+ " next_check_at=DATEADD('SECOND',-1,CURRENT_TIMESTAMP) WHERE"
|
||||
+ " status='NEEDS_REVIEW'");
|
||||
writes.reconcileDue();
|
||||
assertThat(
|
||||
jdbc.queryForObject(
|
||||
"SELECT status FROM repository_write_operation WHERE"
|
||||
+ " status<>'FAILED'",
|
||||
String.class))
|
||||
.isEqualTo("SUCCEEDED");
|
||||
assertThat(jdbc.queryForObject("SELECT repo_write_status FROM project", String.class))
|
||||
.isEqualTo("IDLE");
|
||||
}
|
||||
|
||||
@Test
|
||||
void nativePushHoldsSameLockAsWebsiteWritesAndPersistsBeforeForwarding() throws Exception {
|
||||
Account owner = login("git-shared-lock");
|
||||
String code = create(owner);
|
||||
String secret = issue(owner, code, "READ_WRITE").path("token").asText();
|
||||
String old =
|
||||
ok(auth(get("/api/workspace/projects/" + code + "/tree"), owner.token()))
|
||||
.path("commitSha")
|
||||
.asText();
|
||||
byte[] payload =
|
||||
packets(
|
||||
old + " " + "b".repeat(40) + " refs/heads/main\0 report-status ofs-delta",
|
||||
null);
|
||||
mode.set("block");
|
||||
try (java.util.concurrent.ExecutorService executor =
|
||||
java.util.concurrent.Executors.newSingleThreadExecutor()) {
|
||||
java.util.concurrent.Future<MvcResult> pushing =
|
||||
executor.submit(
|
||||
() ->
|
||||
mvc.perform(
|
||||
basic(
|
||||
post(
|
||||
protocol(code)
|
||||
+ "/git-receive-pack"),
|
||||
owner,
|
||||
secret)
|
||||
.contentType(
|
||||
"application/x-git-receive-pack-request")
|
||||
.content(payload))
|
||||
.andReturn());
|
||||
try {
|
||||
assertThat(pushEntered.await(5, java.util.concurrent.TimeUnit.SECONDS)).isTrue();
|
||||
assertThat(
|
||||
jdbc.queryForObject(
|
||||
"SELECT repo_write_status FROM project", String.class))
|
||||
.isEqualTo("WRITING");
|
||||
assertThat(
|
||||
jdbc.queryForObject(
|
||||
"SELECT operation_kind FROM repository_write_operation",
|
||||
String.class))
|
||||
.isEqualTo("GIT_PUSH");
|
||||
ok(auth(get("/api/workspace/projects/" + code + "/tree"), owner.token()));
|
||||
mvc.perform(
|
||||
body(
|
||||
auth(
|
||||
post(
|
||||
"/api/workspace/projects/"
|
||||
+ code
|
||||
+ "/commits"),
|
||||
owner.token()),
|
||||
Map.of(
|
||||
"expectedCommitSha",
|
||||
old,
|
||||
"message",
|
||||
"竞争网页提交",
|
||||
"changes",
|
||||
List.of(
|
||||
Map.of(
|
||||
"action",
|
||||
"UPDATE_FILE",
|
||||
"path",
|
||||
"README.md",
|
||||
"content",
|
||||
"保留输入")))))
|
||||
.andExpect(status().isConflict());
|
||||
assertThat(
|
||||
jdbc.queryForObject(
|
||||
"SELECT COUNT(*) FROM repository_write_operation",
|
||||
Integer.class))
|
||||
.isEqualTo(1);
|
||||
} finally {
|
||||
pushRelease.countDown();
|
||||
}
|
||||
assertThat(
|
||||
pushing.get(5, java.util.concurrent.TimeUnit.SECONDS)
|
||||
.getResponse()
|
||||
.getStatus())
|
||||
.isEqualTo(200);
|
||||
assertThat(jdbc.queryForObject("SELECT repo_write_status FROM project", String.class))
|
||||
.isEqualTo("IDLE");
|
||||
}
|
||||
}
|
||||
|
||||
@Test
|
||||
void acceptsGzipAndFlushProbeButRejectsDisabledDeletedAndNonMainUpdates() throws Exception {
|
||||
Account owner = login("git-wire");
|
||||
String code = create(owner);
|
||||
String secret = issue(owner, code, "READ_WRITE").path("token").asText();
|
||||
mvc.perform(
|
||||
basic(post(protocol(code) + "/git-receive-pack"), owner, secret)
|
||||
.contentType("application/x-git-receive-pack-request")
|
||||
.content("0000"))
|
||||
.andExpect(status().isOk());
|
||||
assertThat(
|
||||
jdbc.queryForObject(
|
||||
"SELECT COUNT(*) FROM repository_write_operation", Integer.class))
|
||||
.isZero();
|
||||
String old =
|
||||
ok(auth(get("/api/workspace/projects/" + code + "/tree"), owner.token()))
|
||||
.path("commitSha")
|
||||
.asText();
|
||||
byte[] payload =
|
||||
packets(
|
||||
old + " " + "b".repeat(40) + " refs/heads/main\0 report-status ofs-delta",
|
||||
null);
|
||||
ByteArrayOutputStream compressed = new ByteArrayOutputStream();
|
||||
try (java.util.zip.GZIPOutputStream gzip = new java.util.zip.GZIPOutputStream(compressed)) {
|
||||
gzip.write(payload);
|
||||
}
|
||||
mvc.perform(
|
||||
basic(post(protocol(code) + "/git-receive-pack"), owner, secret)
|
||||
.contentType("application/x-git-receive-pack-request")
|
||||
.header("Content-Encoding", "gzip")
|
||||
.content(compressed.toByteArray()))
|
||||
.andExpect(status().isOk());
|
||||
assertThat(received.get()).isEqualTo(payload);
|
||||
jdbc.update("UPDATE sys_user SET status='DISABLED' WHERE user_code=?", owner.username());
|
||||
assertThat(
|
||||
mvc.perform(
|
||||
basic(
|
||||
get(protocol(code) + "/info/refs")
|
||||
.param("service", "git-upload-pack"),
|
||||
owner,
|
||||
secret))
|
||||
.andReturn()
|
||||
.getResponse()
|
||||
.getStatus())
|
||||
.isIn(401, 403);
|
||||
jdbc.update("UPDATE sys_user SET status='ACTIVE' WHERE user_code=?", owner.username());
|
||||
jdbc.update("UPDATE project SET repo_status='CREATING'");
|
||||
mvc.perform(
|
||||
basic(
|
||||
get(protocol(code) + "/info/refs")
|
||||
.param("service", "git-upload-pack"),
|
||||
owner,
|
||||
secret))
|
||||
.andExpect(status().isConflict());
|
||||
jdbc.update("UPDATE project SET repo_status='READY',deleted=1");
|
||||
mvc.perform(
|
||||
basic(
|
||||
get(protocol(code) + "/info/refs")
|
||||
.param("service", "git-upload-pack"),
|
||||
owner,
|
||||
secret))
|
||||
.andExpect(status().isNotFound());
|
||||
}
|
||||
|
||||
@Test
|
||||
void lfsValidatesDownloadActionsIntegrityExpiryAndMetadataLimits() throws Exception {
|
||||
Account owner = login("git-lfs-boundaries");
|
||||
String code = create(owner), other = create(owner);
|
||||
String rw = issue(owner, code, "READ_WRITE").path("token").asText();
|
||||
String ro = issue(owner, code, "READ_ONLY").path("token").asText();
|
||||
byte[] batch =
|
||||
json.writeValueAsBytes(
|
||||
Map.of(
|
||||
"operation",
|
||||
"download",
|
||||
"objects",
|
||||
List.of(Map.of("oid", OID, "size", OBJECT.length))));
|
||||
MvcResult reply =
|
||||
mvc.perform(
|
||||
basic(post(protocol(code) + "/info/lfs/objects/batch"), owner, ro)
|
||||
.contentType(LfsGateway.MEDIA)
|
||||
.content(batch))
|
||||
.andExpect(status().isOk())
|
||||
.andReturn();
|
||||
JsonNode action =
|
||||
json.readTree(reply.getResponse().getContentAsByteArray())
|
||||
.path("objects")
|
||||
.get(0)
|
||||
.path("actions")
|
||||
.path("download");
|
||||
String path = URI.create(action.path("href").asText()).getPath(),
|
||||
authorization = action.path("header").path("Authorization").asText();
|
||||
mvc.perform(get(path).header("Authorization", authorization))
|
||||
.andExpect(status().isOk())
|
||||
.andExpect(content().bytes(OBJECT));
|
||||
mvc.perform(get(path.replace(code, other)).header("Authorization", authorization))
|
||||
.andExpect(status().isUnauthorized());
|
||||
mvc.perform(put(path).header("Authorization", authorization).content(OBJECT))
|
||||
.andExpect(status().isUnauthorized());
|
||||
String key = "git:lfs-action:" + path.substring(path.lastIndexOf('/') + 1);
|
||||
com.fasterxml.jackson.databind.node.ObjectNode stored =
|
||||
(com.fasterxml.jackson.databind.node.ObjectNode) json.readTree(state.get(key));
|
||||
stored.put("expiresAt", java.time.Instant.now().minusSeconds(1).toString());
|
||||
state.remove(key);
|
||||
state.putIfAbsent(key, json.writeValueAsString(stored), java.time.Duration.ofSeconds(30));
|
||||
mvc.perform(get(path).header("Authorization", authorization))
|
||||
.andExpect(status().isUnauthorized());
|
||||
reply =
|
||||
mvc.perform(
|
||||
basic(post(protocol(code) + "/info/lfs/objects/batch"), owner, rw)
|
||||
.contentType(LfsGateway.MEDIA)
|
||||
.content(
|
||||
json.writeValueAsBytes(
|
||||
Map.of(
|
||||
"operation",
|
||||
"upload",
|
||||
"objects",
|
||||
List.of(
|
||||
Map.of(
|
||||
"oid",
|
||||
OID,
|
||||
"size",
|
||||
OBJECT.length))))))
|
||||
.andExpect(status().isOk())
|
||||
.andReturn();
|
||||
action =
|
||||
json.readTree(reply.getResponse().getContentAsByteArray())
|
||||
.path("objects")
|
||||
.get(0)
|
||||
.path("actions")
|
||||
.path("upload");
|
||||
path = URI.create(action.path("href").asText()).getPath();
|
||||
authorization = action.path("header").path("Authorization").asText();
|
||||
mvc.perform(
|
||||
put(path)
|
||||
.header("Authorization", authorization)
|
||||
.content(new byte[OBJECT.length - 1]))
|
||||
.andExpect(status().isBadRequest());
|
||||
mvc.perform(
|
||||
put(path)
|
||||
.header("Authorization", authorization)
|
||||
.content(new byte[OBJECT.length]))
|
||||
.andExpect(status().isBadRequest());
|
||||
mvc.perform(
|
||||
basic(post(protocol(code) + "/info/lfs/objects/batch"), owner, rw)
|
||||
.contentType(LfsGateway.MEDIA)
|
||||
.content("{"))
|
||||
.andExpect(status().isBadRequest());
|
||||
mvc.perform(
|
||||
basic(post(protocol(code) + "/info/lfs/objects/batch"), owner, rw)
|
||||
.contentType(LfsGateway.MEDIA)
|
||||
.content(
|
||||
json.writeValueAsBytes(
|
||||
Map.of(
|
||||
"operation",
|
||||
"upload",
|
||||
"objects",
|
||||
List.of(
|
||||
Map.of(
|
||||
"oid",
|
||||
OID,
|
||||
"size",
|
||||
1073741825L))))))
|
||||
.andExpect(status().isPayloadTooLarge());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,515 @@
|
||||
package com.linkerbot.vibemake.api;
|
||||
|
||||
import static org.assertj.core.api.Assertions.*;
|
||||
|
||||
import com.fasterxml.jackson.databind.*;
|
||||
import com.linkerbot.vibemake.service.config.*;
|
||||
import com.linkerbot.vibemake.service.config.RedisStateProperties;
|
||||
import com.linkerbot.vibemake.service.gitea.GiteaClient;
|
||||
|
||||
import org.junit.jupiter.api.*;
|
||||
import org.junit.jupiter.api.condition.EnabledIfEnvironmentVariable;
|
||||
import org.junit.jupiter.api.io.TempDir;
|
||||
import org.springframework.beans.factory.annotation.Autowired;
|
||||
import org.springframework.boot.test.context.SpringBootTest;
|
||||
import org.springframework.boot.test.web.server.LocalServerPort;
|
||||
import org.springframework.context.annotation.Import;
|
||||
import org.springframework.data.redis.core.*;
|
||||
import org.springframework.jdbc.core.JdbcTemplate;
|
||||
import org.springframework.test.context.ActiveProfiles;
|
||||
import org.springframework.test.context.DynamicPropertyRegistry;
|
||||
import org.springframework.test.context.DynamicPropertySource;
|
||||
|
||||
import java.net.*;
|
||||
import java.net.http.*;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.nio.file.*;
|
||||
import java.sql.*;
|
||||
import java.time.Duration;
|
||||
import java.util.*;
|
||||
import java.util.concurrent.*;
|
||||
import java.util.regex.*;
|
||||
|
||||
/** 只使用本测试创建的用户、仓库与临时 H2;不访问业务项目。 */
|
||||
@SpringBootTest(
|
||||
webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT,
|
||||
properties = {
|
||||
"spring.datasource.url=jdbc:h2:mem:gitlive;MODE=MySQL;DB_CLOSE_DELAY=-1",
|
||||
"spring.datasource.username=sa",
|
||||
"spring.datasource.password=",
|
||||
"spring.datasource.driver-class-name=org.h2.Driver",
|
||||
"vibemake.ai.enabled=false",
|
||||
"vibemake.oss.enabled=false",
|
||||
"vibemake.limits.enabled=false",
|
||||
"vibemake.repository-write.worker-enabled=false",
|
||||
"vibemake.git.enabled=true",
|
||||
"vibemake.git.public-base-url=https://git.example/git",
|
||||
"vibemake.git.upstream-username=fixture",
|
||||
"vibemake.git.upstream-token=fixture",
|
||||
"vibemake.redis.real-test=true",
|
||||
"vibemake.redis.key-prefix=vibemake:test:git-live-${random.uuid}",
|
||||
"logging.level.root=WARN"
|
||||
})
|
||||
@ActiveProfiles("test")
|
||||
@Import(GiteaLiveTest.Config.class)
|
||||
@EnabledIfEnvironmentVariable(named = "VIBEMAKE_LIVE_GITEA", matches = "true")
|
||||
class GitGatewayLiveTest {
|
||||
@LocalServerPort int port;
|
||||
@Autowired ObjectMapper json;
|
||||
@Autowired JdbcTemplate jdbc;
|
||||
@Autowired VibemakeProperties config;
|
||||
@Autowired GitGatewayProperties gitConfig;
|
||||
@Autowired GiteaClient gitea;
|
||||
@Autowired StringRedisTemplate redis;
|
||||
@Autowired RedisStateProperties namespace;
|
||||
static String mysqlDatabase;
|
||||
static String mysqlUrl;
|
||||
|
||||
@DynamicPropertySource
|
||||
static void isolatedMysql(DynamicPropertyRegistry registry) throws Exception {
|
||||
if (!"true".equals(System.getenv("GIT_GATEWAY_MYSQL"))) return;
|
||||
String original = System.getenv("DB_URL");
|
||||
if (original == null || !original.startsWith("jdbc:mysql:"))
|
||||
throw new IllegalStateException("Git 联调需要 DB_URL");
|
||||
String requestedDatabase =
|
||||
"vibemake_git_verify_" + UUID.randomUUID().toString().replace("-", "");
|
||||
try (Connection connection =
|
||||
DriverManager.getConnection(
|
||||
original,
|
||||
System.getenv("DB_USERNAME"),
|
||||
System.getenv("DB_PASSWORD"));
|
||||
Statement statement = connection.createStatement()) {
|
||||
statement.execute(
|
||||
"CREATE DATABASE "
|
||||
+ requestedDatabase
|
||||
+ " CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci");
|
||||
mysqlDatabase = requestedDatabase;
|
||||
}
|
||||
String plain = original.split("\\?", 2)[0];
|
||||
mysqlUrl =
|
||||
plain.substring(0, plain.lastIndexOf('/') + 1)
|
||||
+ mysqlDatabase
|
||||
+ (original.contains("?") ? "?" + original.split("\\?", 2)[1] : "");
|
||||
registry.add("spring.datasource.url", () -> mysqlUrl);
|
||||
registry.add("spring.datasource.username", () -> System.getenv("DB_USERNAME"));
|
||||
registry.add("spring.datasource.password", () -> System.getenv("DB_PASSWORD"));
|
||||
registry.add("spring.datasource.driver-class-name", () -> "com.mysql.cj.jdbc.Driver");
|
||||
}
|
||||
|
||||
@AfterAll
|
||||
static void dropOnlyTemporaryDatabase() throws Exception {
|
||||
if (mysqlDatabase == null) return;
|
||||
if (!mysqlDatabase.matches("vibemake_git_verify_[a-f0-9]{32}"))
|
||||
throw new IllegalStateException("拒绝清理非测试库");
|
||||
try (Connection connection =
|
||||
DriverManager.getConnection(
|
||||
System.getenv("DB_URL"),
|
||||
System.getenv("DB_USERNAME"),
|
||||
System.getenv("DB_PASSWORD"));
|
||||
Statement statement = connection.createStatement()) {
|
||||
statement.execute("DROP DATABASE " + mysqlDatabase);
|
||||
}
|
||||
}
|
||||
|
||||
@TempDir Path directory;
|
||||
final HttpClient http =
|
||||
HttpClient.newBuilder()
|
||||
.followRedirects(HttpClient.Redirect.NEVER)
|
||||
.connectTimeout(Duration.ofSeconds(10))
|
||||
.build();
|
||||
String bot;
|
||||
String siteToken;
|
||||
String username;
|
||||
String project;
|
||||
String gitToken;
|
||||
String gitAuth;
|
||||
Path askpass;
|
||||
boolean botCreated;
|
||||
|
||||
@BeforeEach
|
||||
void setup() throws Exception {
|
||||
if (mysqlUrl == null) SchemaSupport.reset(jdbc);
|
||||
else
|
||||
for (String doc : List.of("database.md", "admin-database.md")) {
|
||||
Matcher tables =
|
||||
Pattern.compile("CREATE TABLE [\\s\\S]*?;")
|
||||
.matcher(Files.readString(Path.of("../docs/" + doc)));
|
||||
while (tables.find()) jdbc.execute(tables.group());
|
||||
}
|
||||
String suffix = UUID.randomUUID().toString().replace("-", "").substring(0, 16);
|
||||
bot = "vbg_test_" + suffix;
|
||||
String password = "Vbg!" + UUID.randomUUID() + "a9";
|
||||
admin(
|
||||
"POST",
|
||||
"/admin/users",
|
||||
Map.of(
|
||||
"username",
|
||||
bot,
|
||||
"email",
|
||||
bot + "@example.invalid",
|
||||
"password",
|
||||
password,
|
||||
"must_change_password",
|
||||
false,
|
||||
"send_notify",
|
||||
false),
|
||||
201);
|
||||
botCreated = true;
|
||||
HttpRequest mint =
|
||||
HttpRequest.newBuilder(
|
||||
URI.create(
|
||||
config.getGitea().getBaseUrl()
|
||||
+ "/api/v1/users/"
|
||||
+ bot
|
||||
+ "/tokens"))
|
||||
.header(
|
||||
"Authorization",
|
||||
"Basic "
|
||||
+ Base64.getEncoder()
|
||||
.encodeToString(
|
||||
(bot + ":" + password)
|
||||
.getBytes(StandardCharsets.UTF_8)))
|
||||
.header("Content-Type", "application/json")
|
||||
.POST(
|
||||
HttpRequest.BodyPublishers.ofByteArray(
|
||||
json.writeValueAsBytes(
|
||||
Map.of(
|
||||
"name",
|
||||
"gateway-live",
|
||||
"scopes",
|
||||
List.of("read:user", "write:repository")))))
|
||||
.build();
|
||||
HttpResponse<byte[]> minted = http.send(mint, HttpResponse.BodyHandlers.ofByteArray());
|
||||
assertThat(minted.statusCode()).as("创建测试技术账号PAT").isEqualTo(201);
|
||||
gitConfig.setUpstreamUsername(bot);
|
||||
gitConfig.setUpstreamToken(json.readTree(minted.body()).path("sha1").asText());
|
||||
// 仅回环联调使用明文 HTTP;生产配置启动校验仍强制公网 HTTPS。
|
||||
gitConfig.setPublicBaseUrl("http://127.0.0.1:" + port + "/git");
|
||||
JsonNode captcha = api("GET", "/api/auth/captcha", null, null);
|
||||
JsonNode login =
|
||||
api(
|
||||
"POST",
|
||||
"/api/auth/login",
|
||||
Map.of(
|
||||
"email",
|
||||
"gateway-live-" + suffix + "@example.invalid",
|
||||
"captchaCode",
|
||||
captcha.path("captchaCode").asText(),
|
||||
"captchaAnswer",
|
||||
"ABCDE"),
|
||||
null);
|
||||
siteToken = login.path("token").asText();
|
||||
username = login.path("user").path("userCode").asText();
|
||||
project =
|
||||
api(
|
||||
"POST",
|
||||
"/api/workspace/projects",
|
||||
Map.of("name", "网关真实联调", "projectType", "GENERAL"),
|
||||
siteToken)
|
||||
.path("projectCode")
|
||||
.asText();
|
||||
gitToken =
|
||||
api(
|
||||
"POST",
|
||||
"/api/workspace/projects/" + project + "/git/tokens",
|
||||
Map.of("name", "测试本机", "scope", "READ_WRITE"),
|
||||
siteToken)
|
||||
.path("token")
|
||||
.asText();
|
||||
gitAuth =
|
||||
"Basic "
|
||||
+ Base64.getEncoder()
|
||||
.encodeToString(
|
||||
(username + ":" + gitToken)
|
||||
.getBytes(StandardCharsets.UTF_8));
|
||||
askpass = directory.resolve("askpass.sh");
|
||||
Files.writeString(
|
||||
askpass,
|
||||
"#!/bin/sh\n"
|
||||
+ "case \"$1\" in *Username*) printf '%s' \"$VBG_USERNAME\";; *) printf '%s'"
|
||||
+ " \"$VBG_SECRET\";; esac\n");
|
||||
Files.setPosixFilePermissions(
|
||||
askpass, java.nio.file.attribute.PosixFilePermissions.fromString("rwx------"));
|
||||
}
|
||||
|
||||
@AfterEach
|
||||
void cleanup() throws Exception {
|
||||
for (String name : jdbc.queryForList("SELECT gitea_repo_name FROM project", String.class)) {
|
||||
if (!name.matches("p_([0-9a-f]{32}|[0-9abcdefghjkmnpqrstvwxyz]{12})"))
|
||||
throw new IllegalStateException("拒绝清理非测试仓库");
|
||||
admin("DELETE", "/repos/" + config.getGitea().getOwner() + "/" + name, null, 204);
|
||||
}
|
||||
if (botCreated) admin("DELETE", "/admin/users/" + bot, null, 204);
|
||||
assertThat(namespace.getKeyPrefix()).startsWith("vibemake:test:git-live-");
|
||||
try (Cursor<String> cursor =
|
||||
redis.scan(
|
||||
ScanOptions.scanOptions()
|
||||
.match(namespace.prefix() + "*")
|
||||
.count(100)
|
||||
.build())) {
|
||||
while (cursor.hasNext()) redis.delete(cursor.next());
|
||||
}
|
||||
}
|
||||
|
||||
JsonNode admin(String method, String path, Object body, int expected) throws Exception {
|
||||
HttpRequest.Builder builder =
|
||||
HttpRequest.newBuilder(
|
||||
URI.create(config.getGitea().getBaseUrl() + "/api/v1" + path))
|
||||
.timeout(Duration.ofSeconds(40))
|
||||
.header("Authorization", "token " + config.getGitea().getToken())
|
||||
.header("Content-Type", "application/json");
|
||||
HttpResponse<byte[]> result =
|
||||
http.send(
|
||||
builder.method(
|
||||
method,
|
||||
body == null
|
||||
? HttpRequest.BodyPublishers.noBody()
|
||||
: HttpRequest.BodyPublishers.ofByteArray(
|
||||
json.writeValueAsBytes(body)))
|
||||
.build(),
|
||||
HttpResponse.BodyHandlers.ofByteArray());
|
||||
assertThat(result.statusCode()).as("Gitea 测试管理请求 %s %s", method, path).isEqualTo(expected);
|
||||
return result.body().length == 0 ? json.nullNode() : json.readTree(result.body());
|
||||
}
|
||||
|
||||
JsonNode api(String method, String path, Object body, String token) throws Exception {
|
||||
HttpRequest.Builder builder =
|
||||
HttpRequest.newBuilder(URI.create("http://127.0.0.1:" + port + path))
|
||||
.timeout(Duration.ofSeconds(90))
|
||||
.header("Content-Type", "application/json");
|
||||
if (token != null) builder.header("Authorization", "Bearer " + token);
|
||||
HttpResponse<byte[]> result =
|
||||
http.send(
|
||||
builder.method(
|
||||
method,
|
||||
body == null
|
||||
? HttpRequest.BodyPublishers.noBody()
|
||||
: HttpRequest.BodyPublishers.ofByteArray(
|
||||
json.writeValueAsBytes(body)))
|
||||
.build(),
|
||||
HttpResponse.BodyHandlers.ofByteArray());
|
||||
JsonNode parsed = json.readTree(result.body());
|
||||
assertThat(result.statusCode())
|
||||
.as("网站请求 %s %s code=%s", method, path, parsed.path("code").asText())
|
||||
.isEqualTo(200);
|
||||
return parsed.path("data");
|
||||
}
|
||||
|
||||
record Command(int status, String output) {}
|
||||
|
||||
Command command(Path cwd, String... args) throws Exception {
|
||||
List<String> command =
|
||||
new ArrayList<>(
|
||||
List.of(
|
||||
"git",
|
||||
"-c",
|
||||
"credential.helper=",
|
||||
"-c",
|
||||
"credential.useHttpPath=true"));
|
||||
command.addAll(List.of(args));
|
||||
ProcessBuilder builder =
|
||||
new ProcessBuilder(command).directory(cwd.toFile()).redirectErrorStream(true);
|
||||
// 凭据只在环境中传递;不出现在命令行、URL、断言正文或日志中。
|
||||
builder.environment().put("GIT_ASKPASS", askpass.toString());
|
||||
builder.environment().put("VBG_USERNAME", username);
|
||||
builder.environment().put("VBG_SECRET", gitToken);
|
||||
builder.environment().put("GIT_TERMINAL_PROMPT", "0");
|
||||
builder.environment().put("GIT_AUTHOR_NAME", "Gateway fixture");
|
||||
builder.environment().put("GIT_AUTHOR_EMAIL", "fixture@example.invalid");
|
||||
builder.environment().put("GIT_COMMITTER_NAME", "Gateway fixture");
|
||||
builder.environment().put("GIT_COMMITTER_EMAIL", "fixture@example.invalid");
|
||||
Process process = builder.start();
|
||||
CompletableFuture<byte[]> output =
|
||||
CompletableFuture.supplyAsync(
|
||||
() -> {
|
||||
try {
|
||||
return process.getInputStream().readAllBytes();
|
||||
} catch (Exception error) {
|
||||
throw new IllegalStateException(error);
|
||||
}
|
||||
});
|
||||
if (!process.waitFor(180, TimeUnit.SECONDS)) {
|
||||
process.destroyForcibly();
|
||||
throw new IllegalStateException("Git 联调超时");
|
||||
}
|
||||
return new Command(
|
||||
process.exitValue(),
|
||||
new String(output.get(5, TimeUnit.SECONDS), StandardCharsets.UTF_8));
|
||||
}
|
||||
|
||||
String git(Path cwd, String... args) throws Exception {
|
||||
Command result = command(cwd, args);
|
||||
assertThat(result.status())
|
||||
.as(
|
||||
"Git %s: %s",
|
||||
args[0],
|
||||
result.output()
|
||||
.replace(gitToken, "[redacted]")
|
||||
.replace(gitConfig.getUpstreamToken(), "[redacted]")
|
||||
.replaceAll(
|
||||
"(?i)(Basic|Bearer) [A-Za-z0-9_+./=-]+", "$1 [redacted]"))
|
||||
.isZero();
|
||||
return result.output().trim();
|
||||
}
|
||||
|
||||
@Test
|
||||
void nativeGitAndLargeLfsRoundTripWithRefProtectionAndExplicitPublication() throws Exception {
|
||||
String url = "http://127.0.0.1:" + port + "/git/projects/" + project + ".git";
|
||||
String tree = "/api/workspace/projects/" + project + "/tree";
|
||||
String first = api("GET", tree, null, siteToken).path("commitSha").asText();
|
||||
api(
|
||||
"PUT",
|
||||
"/api/workspace/projects/" + project + "/publication",
|
||||
Map.of("commitSha", first),
|
||||
siteToken);
|
||||
git(
|
||||
directory,
|
||||
"-c",
|
||||
"protocol.version=0",
|
||||
"clone",
|
||||
"--single-branch",
|
||||
"--branch",
|
||||
"main",
|
||||
url,
|
||||
"one");
|
||||
Path one = directory.resolve("one");
|
||||
git(one, "config", "credential.useHttpPath", "true");
|
||||
git(one, "lfs", "install", "--local");
|
||||
git(one, "lfs", "track", "*.bin");
|
||||
byte[] bytes = new byte[51 * 1024 * 1024 + 17];
|
||||
new Random(71).nextBytes(bytes);
|
||||
Files.write(one.resolve("large.bin"), bytes);
|
||||
Files.createDirectories(one.resolve("子目录"));
|
||||
Files.writeString(one.resolve("子目录/说明.md"), "本地提交\n");
|
||||
git(one, "add", ".");
|
||||
git(one, "commit", "-m", "本地 Git 和 LFS 更新");
|
||||
String second = git(one, "rev-parse", "HEAD");
|
||||
git(one, "push", "origin", "main");
|
||||
assertThat(api("GET", tree, null, siteToken).path("commitSha").asText()).isEqualTo(second);
|
||||
assertThat(
|
||||
api("GET", "/api/community/projects/" + project + "/tree", null, null)
|
||||
.path("commitSha")
|
||||
.asText())
|
||||
.isEqualTo(first);
|
||||
assertThat(
|
||||
api(
|
||||
"GET",
|
||||
"/api/workspace/projects/" + project + "/commits",
|
||||
null,
|
||||
siteToken)
|
||||
.path("items")
|
||||
.get(0)
|
||||
.path("message")
|
||||
.asText())
|
||||
.contains("本地 Git 和 LFS 更新");
|
||||
git(directory, "-c", "protocol.version=2", "clone", "--no-checkout", url, "two");
|
||||
Path two = directory.resolve("two");
|
||||
git(two, "lfs", "install", "--local");
|
||||
git(two, "checkout", "main");
|
||||
assertThat(Arrays.equals(Files.readAllBytes(two.resolve("large.bin")), bytes))
|
||||
.as("LFS 实体逐字节一致")
|
||||
.isTrue();
|
||||
assertThat(Files.readString(two.resolve("子目录/说明.md"))).isEqualTo("本地提交\n");
|
||||
assertThat(command(one, "push", "origin", "HEAD:refs/heads/other").status()).isNotZero();
|
||||
assertThat(command(one, "push", "origin", "HEAD:refs/tags/test").status()).isNotZero();
|
||||
assertThat(command(one, "push", "origin", ":main").status()).isNotZero();
|
||||
git(one, "reset", "--hard", first);
|
||||
assertThat(command(one, "push", "--force", "origin", "main").status()).isNotZero();
|
||||
assertThat(api("GET", tree, null, siteToken).path("commitSha").asText()).isEqualTo(second);
|
||||
api(
|
||||
"PUT",
|
||||
"/api/workspace/projects/" + project + "/publication",
|
||||
Map.of("commitSha", second),
|
||||
siteToken);
|
||||
assertThat(
|
||||
api("GET", "/api/community/projects/" + project + "/tree", null, null)
|
||||
.path("commitSha")
|
||||
.asText())
|
||||
.isEqualTo(second);
|
||||
Files.writeString(two.resolve("pull.txt"), "下一次更新");
|
||||
git(two, "add", "pull.txt");
|
||||
git(two, "commit", "-m", "继续更新");
|
||||
git(two, "push", "origin", "main");
|
||||
git(one, "pull", "--ff-only");
|
||||
assertThat(Files.readString(one.resolve("pull.txt"))).isEqualTo("下一次更新");
|
||||
// 非 LFS 的大 pack + 极小 postBuffer 强制走 HTTP 分块及认证 probe。
|
||||
byte[] packData = new byte[2 * 1024 * 1024 + 7];
|
||||
new Random(19).nextBytes(packData);
|
||||
Files.write(one.resolve("native.dat"), packData);
|
||||
git(one, "add", "native.dat");
|
||||
git(one, "commit", "-m", "大 pack 更新");
|
||||
git(one, "-c", "http.postBuffer=1024", "push", "origin", "main");
|
||||
git(two, "pull", "--ff-only");
|
||||
assertThat(Arrays.equals(Files.readAllBytes(two.resolve("native.dat")), packData)).isTrue();
|
||||
git(two, "checkout", "-b", "local-only");
|
||||
Files.writeString(two.resolve("left.txt"), "left");
|
||||
git(two, "add", "left.txt");
|
||||
git(two, "commit", "-m", "本地分支提交");
|
||||
git(two, "checkout", "main");
|
||||
Files.writeString(two.resolve("right.txt"), "right");
|
||||
git(two, "add", "right.txt");
|
||||
git(two, "commit", "-m", "main 提交");
|
||||
git(two, "merge", "--no-ff", "local-only", "-m", "合并本地提交");
|
||||
git(two, "push", "origin", "main");
|
||||
Path signing = directory.resolve("signing-key");
|
||||
Process keygen =
|
||||
new ProcessBuilder(
|
||||
"ssh-keygen",
|
||||
"-q",
|
||||
"-t",
|
||||
"ed25519",
|
||||
"-N",
|
||||
"",
|
||||
"-f",
|
||||
signing.toString())
|
||||
.redirectErrorStream(true)
|
||||
.start();
|
||||
assertThat(keygen.waitFor(10, TimeUnit.SECONDS)).isTrue();
|
||||
assertThat(keygen.exitValue()).isZero();
|
||||
git(
|
||||
two,
|
||||
"-c",
|
||||
"gpg.format=ssh",
|
||||
"-c",
|
||||
"user.signingkey=" + signing,
|
||||
"commit",
|
||||
"--allow-empty",
|
||||
"-S",
|
||||
"-m",
|
||||
"签名提交");
|
||||
String signed = git(two, "rev-parse", "HEAD");
|
||||
git(two, "push", "origin", "main");
|
||||
assertThat(api("GET", tree, null, siteToken).path("commitSha").asText()).isEqualTo(signed);
|
||||
assertThat(git(two, "cat-file", "-p", "HEAD"))
|
||||
.contains("gpgsig -----BEGIN SSH SIGNATURE-----");
|
||||
String originalToken = gitToken;
|
||||
String other =
|
||||
api(
|
||||
"POST",
|
||||
"/api/workspace/projects",
|
||||
Map.of("name", "第二项目", "projectType", "GENERAL"),
|
||||
siteToken)
|
||||
.path("projectCode")
|
||||
.asText();
|
||||
gitToken =
|
||||
api(
|
||||
"POST",
|
||||
"/api/workspace/projects/" + other + "/git/tokens",
|
||||
Map.of("name", "第二项目凭据", "scope", "READ_WRITE"),
|
||||
siteToken)
|
||||
.path("token")
|
||||
.asText();
|
||||
String otherUrl = "http://127.0.0.1:" + port + "/git/projects/" + other + ".git";
|
||||
git(directory, "clone", otherUrl, "other");
|
||||
Path otherRepo = directory.resolve("other");
|
||||
git(otherRepo, "lfs", "install", "--local");
|
||||
git(otherRepo, "lfs", "track", "*.bin");
|
||||
Files.write(otherRepo.resolve("other.bin"), Arrays.copyOf(bytes, 1024));
|
||||
git(otherRepo, "add", ".");
|
||||
git(otherRepo, "commit", "-m", "第二项目 LFS");
|
||||
git(otherRepo, "push", "origin", "main");
|
||||
assertThat(command(one, "fetch", "origin").status()).isNotZero();
|
||||
gitToken = originalToken;
|
||||
git(one, "fetch", "origin");
|
||||
}
|
||||
}
|
||||
@@ -26,11 +26,12 @@ final class SchemaSupport {
|
||||
"JSON NOT NULL DEFAULT (JSON_ARRAY())",
|
||||
"CLOB NOT NULL DEFAULT '[]'")
|
||||
.replace(" CHARACTER SET utf8mb4 COLLATE utf8mb4_bin", "")
|
||||
.replace(" CHARACTER SET ascii COLLATE ascii_bin", "")
|
||||
.replaceAll(" COMMENT '[^']*'", "")
|
||||
.replaceAll("\\) ENGINE=InnoDB[^;]*;", ");");
|
||||
jdbc.execute(ddl);
|
||||
tables++;
|
||||
}
|
||||
if (tables != 6) throw new IllegalStateException("必须包含四张基础表、迁移任务表和仓库写入操作表");
|
||||
if (tables != 7) throw new IllegalStateException("必须包含七张业务与审计表");
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1263,7 +1263,7 @@ class WorkflowTest {
|
||||
"PROJECT",
|
||||
"PROJECT_COMMENT",
|
||||
"PROJECT_LIKE",
|
||||
"PROJECT_IMPORT_TASK", "REPOSITORY_WRITE_OPERATION");
|
||||
"PROJECT_IMPORT_TASK", "REPOSITORY_WRITE_OPERATION", "PROJECT_GIT_TOKEN");
|
||||
}
|
||||
|
||||
@Test
|
||||
|
||||
@@ -0,0 +1,49 @@
|
||||
package com.linkerbot.vibemake.bean;
|
||||
|
||||
import jakarta.validation.constraints.Max;
|
||||
import jakarta.validation.constraints.Min;
|
||||
import jakarta.validation.constraints.NotBlank;
|
||||
import jakarta.validation.constraints.NotNull;
|
||||
import jakarta.validation.constraints.Size;
|
||||
|
||||
import java.time.LocalDateTime;
|
||||
|
||||
public final class GitContracts {
|
||||
private GitContracts() {}
|
||||
|
||||
public enum Scope {
|
||||
READ_ONLY,
|
||||
READ_WRITE
|
||||
}
|
||||
|
||||
public record TokenInput(
|
||||
@NotBlank @Size(max = 64) String name,
|
||||
@NotNull Scope scope,
|
||||
@Min(1) @Max(365) Integer expiresInDays) {}
|
||||
|
||||
public record TokenView(
|
||||
String tokenCode,
|
||||
String name,
|
||||
Scope scope,
|
||||
String status,
|
||||
LocalDateTime createdAt,
|
||||
LocalDateTime expiresAt,
|
||||
LocalDateTime revokedAt,
|
||||
LocalDateTime lastUsedAt) {}
|
||||
|
||||
public record IssuedToken(TokenView credential, String token) {
|
||||
@Override
|
||||
public String toString() {
|
||||
return "IssuedToken[credential=" + credential + ", token=[redacted]]";
|
||||
}
|
||||
}
|
||||
|
||||
public record ConnectionView(
|
||||
String cloneUrl,
|
||||
String username,
|
||||
String defaultBranch,
|
||||
String headCommitSha,
|
||||
String publishedCommitSha,
|
||||
String writeStatus,
|
||||
boolean canPush) {}
|
||||
}
|
||||
@@ -0,0 +1,21 @@
|
||||
package com.linkerbot.vibemake.bean.entity;
|
||||
|
||||
import lombok.Getter;
|
||||
import lombok.Setter;
|
||||
|
||||
import java.time.LocalDateTime;
|
||||
|
||||
@Getter
|
||||
@Setter
|
||||
public class ProjectGitToken extends BaseEntity {
|
||||
private String tokenCode;
|
||||
private String projectCode;
|
||||
private Long ownerUserId;
|
||||
private String tokenName;
|
||||
private String tokenHash;
|
||||
private String permission;
|
||||
private Long sessionVersion;
|
||||
private LocalDateTime expiresAt;
|
||||
private LocalDateTime revokedAt;
|
||||
private LocalDateTime lastUsedAt;
|
||||
}
|
||||
+4
@@ -12,6 +12,10 @@ import java.time.LocalDateTime;
|
||||
public class RepositoryWriteOperation {
|
||||
@TableId(type = IdType.INPUT)
|
||||
private String operationCode;
|
||||
private String operationKind;
|
||||
private String targetCommitSha;
|
||||
private String actorUserCode;
|
||||
private String gitTokenCode;
|
||||
private String projectCode;
|
||||
private Long repositoryId;
|
||||
private String repositoryOwner;
|
||||
|
||||
+136
@@ -0,0 +1,136 @@
|
||||
package com.linkerbot.vibemake.security;
|
||||
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
import com.linkerbot.vibemake.common.BusinessException;
|
||||
import com.linkerbot.vibemake.service.git.*;
|
||||
import com.linkerbot.vibemake.service.state.ClientAddresses;
|
||||
|
||||
import jakarta.servlet.*;
|
||||
import jakarta.servlet.http.*;
|
||||
|
||||
import lombok.RequiredArgsConstructor;
|
||||
|
||||
import org.springframework.stereotype.Component;
|
||||
import org.springframework.web.filter.OncePerRequestFilter;
|
||||
|
||||
import java.io.IOException;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.util.*;
|
||||
import java.util.regex.*;
|
||||
|
||||
@Component
|
||||
@RequiredArgsConstructor
|
||||
public class GitAuthenticationFilter extends OncePerRequestFilter {
|
||||
private static final org.slf4j.Logger log =
|
||||
org.slf4j.LoggerFactory.getLogger(GitAuthenticationFilter.class);
|
||||
public static final String ACCESS = GitAuthenticationFilter.class.getName() + ".access";
|
||||
public static final String ACTION = GitAuthenticationFilter.class.getName() + ".action";
|
||||
private static final Pattern PATH =
|
||||
Pattern.compile("/git/projects/([A-Za-z0-9]{1,32})\\.git/(.+)");
|
||||
private final GitTokens tokens;
|
||||
private final LfsGateway lfs;
|
||||
private final GitLimits limits;
|
||||
private final ClientAddresses addresses;
|
||||
private final ObjectMapper json;
|
||||
|
||||
@Override
|
||||
protected void doFilterInternal(
|
||||
HttpServletRequest request, HttpServletResponse response, FilterChain chain)
|
||||
throws IOException, ServletException {
|
||||
response.setHeader("Cache-Control", "no-store");
|
||||
String ip = addresses.resolve(request);
|
||||
try {
|
||||
tokens.enabled();
|
||||
limits.authenticationAllowed(ip);
|
||||
Matcher matcher =
|
||||
PATH.matcher(
|
||||
request.getRequestURI().substring(request.getContextPath().length()));
|
||||
if (!matcher.matches()) throw BusinessException.notFound();
|
||||
String code = matcher.group(1);
|
||||
String endpoint = matcher.group(2);
|
||||
String authorization = request.getHeader("Authorization");
|
||||
GitAccess access;
|
||||
if (endpoint.startsWith("info/lfs/actions/")) {
|
||||
if (authorization == null || !authorization.startsWith("Bearer "))
|
||||
throw GitTokens.unauthorized();
|
||||
LfsGateway.Action action =
|
||||
lfs.authenticate(
|
||||
endpoint.substring("info/lfs/actions/".length()),
|
||||
code,
|
||||
authorization.substring(7),
|
||||
request.getMethod());
|
||||
access = action.access();
|
||||
request.setAttribute(ACTION, action);
|
||||
} else {
|
||||
if (authorization == null
|
||||
|| !authorization.startsWith("Basic ")
|
||||
|| authorization.length() > 2048) throw GitTokens.unauthorized();
|
||||
String decoded;
|
||||
try {
|
||||
decoded =
|
||||
new String(
|
||||
Base64.getDecoder().decode(authorization.substring(6)),
|
||||
StandardCharsets.UTF_8);
|
||||
} catch (IllegalArgumentException error) {
|
||||
throw GitTokens.unauthorized();
|
||||
}
|
||||
int separator = decoded.indexOf(':');
|
||||
if (separator < 1) throw GitTokens.unauthorized();
|
||||
access =
|
||||
tokens.authenticate(
|
||||
decoded.substring(0, separator),
|
||||
decoded.substring(separator + 1),
|
||||
code);
|
||||
}
|
||||
limits.request(access, endpoint.startsWith("info/lfs/"));
|
||||
request.setAttribute(ACCESS, access);
|
||||
} catch (BusinessException error) {
|
||||
if (request.getHeader("Authorization") != null)
|
||||
log.warn("Git 认证或限流拒绝 code={} status={}", error.code(), error.status());
|
||||
// Git 客户端的首次无凭据请求用于协商 Basic challenge,不计为密码认证失败。
|
||||
if (error.status() == 401 && request.getHeader("Authorization") != null)
|
||||
limits.authenticationFailed(ip);
|
||||
failure(request, response, json, error);
|
||||
return;
|
||||
} catch (RuntimeException error) {
|
||||
log.warn("Git 认证服务异常 errorType={}", error.getClass().getSimpleName());
|
||||
failure(
|
||||
request,
|
||||
response,
|
||||
json,
|
||||
new BusinessException(503, "GIT_UNAVAILABLE", "Git 鉴权服务暂不可用"));
|
||||
return;
|
||||
}
|
||||
chain.doFilter(request, response);
|
||||
}
|
||||
|
||||
public static void failure(
|
||||
HttpServletRequest request,
|
||||
HttpServletResponse response,
|
||||
ObjectMapper json,
|
||||
BusinessException error)
|
||||
throws IOException {
|
||||
if (response.isCommitted()) {
|
||||
response.getOutputStream().close();
|
||||
return;
|
||||
}
|
||||
response.resetBuffer();
|
||||
response.setHeader("Content-Length", null);
|
||||
response.setStatus(error.status());
|
||||
response.setHeader("Cache-Control", "no-store");
|
||||
if (error.status() == 401)
|
||||
response.setHeader(
|
||||
"WWW-Authenticate", "Basic realm=\"Vibemake Git\", charset=\"UTF-8\"");
|
||||
if (error.status() == 429) response.setHeader("Retry-After", "60");
|
||||
if (request.getRequestURI().contains("/info/lfs/")) {
|
||||
response.setContentType(LfsGateway.MEDIA + ";charset=UTF-8");
|
||||
json.writeValue(response.getOutputStream(), Map.of("message", error.getMessage()));
|
||||
} else {
|
||||
response.setContentType("text/plain;charset=UTF-8");
|
||||
response.getOutputStream()
|
||||
.write(
|
||||
("error: " + error.getMessage() + "\n")
|
||||
.getBytes(StandardCharsets.UTF_8));
|
||||
}
|
||||
}
|
||||
}
|
||||
+40
@@ -0,0 +1,40 @@
|
||||
package com.linkerbot.vibemake.security;
|
||||
|
||||
import org.springframework.boot.web.servlet.FilterRegistrationBean;
|
||||
import org.springframework.context.annotation.*;
|
||||
import org.springframework.core.annotation.Order;
|
||||
import org.springframework.security.config.annotation.web.builders.HttpSecurity;
|
||||
import org.springframework.security.config.http.SessionCreationPolicy;
|
||||
import org.springframework.security.web.SecurityFilterChain;
|
||||
import org.springframework.security.web.authentication.UsernamePasswordAuthenticationFilter;
|
||||
|
||||
@Configuration
|
||||
public class GitSecurityConfiguration {
|
||||
@Bean
|
||||
public FilterRegistrationBean<GitAuthenticationFilter> disableGitServletRegistration(
|
||||
GitAuthenticationFilter filter) {
|
||||
FilterRegistrationBean<GitAuthenticationFilter> registration =
|
||||
new FilterRegistrationBean<>(filter);
|
||||
registration.setEnabled(false);
|
||||
return registration;
|
||||
}
|
||||
|
||||
@Bean
|
||||
@Order(1)
|
||||
public SecurityFilterChain gitSecurityFilterChain(
|
||||
HttpSecurity http, GitAuthenticationFilter filter) throws Exception {
|
||||
return http.securityMatcher("/git/**")
|
||||
.csrf(csrf -> csrf.disable())
|
||||
.cors(cors -> cors.disable())
|
||||
.sessionManagement(
|
||||
session -> session.sessionCreationPolicy(SessionCreationPolicy.STATELESS))
|
||||
.requestCache(cache -> cache.disable())
|
||||
.formLogin(form -> form.disable())
|
||||
.httpBasic(basic -> basic.disable())
|
||||
.logout(logout -> logout.disable())
|
||||
// 专用过滤器已完成每请求鉴权,不创建可被普通 Bearer 链复用的登录状态。
|
||||
.authorizeHttpRequests(auth -> auth.anyRequest().permitAll())
|
||||
.addFilterBefore(filter, UsernamePasswordAuthenticationFilter.class)
|
||||
.build();
|
||||
}
|
||||
}
|
||||
+95
@@ -0,0 +1,95 @@
|
||||
package com.linkerbot.vibemake.service.config;
|
||||
|
||||
import jakarta.validation.constraints.AssertTrue;
|
||||
import jakarta.validation.constraints.Max;
|
||||
import jakarta.validation.constraints.Min;
|
||||
import jakarta.validation.constraints.Positive;
|
||||
|
||||
import lombok.Getter;
|
||||
import lombok.Setter;
|
||||
|
||||
import org.springframework.boot.context.properties.ConfigurationProperties;
|
||||
import org.springframework.stereotype.Component;
|
||||
import org.springframework.validation.annotation.Validated;
|
||||
|
||||
import java.net.URI;
|
||||
import java.time.Duration;
|
||||
|
||||
@Getter
|
||||
@Setter
|
||||
@Component
|
||||
@Validated
|
||||
@ConfigurationProperties("vibemake.git")
|
||||
public class GitGatewayProperties {
|
||||
private boolean enabled;
|
||||
private String publicBaseUrl = "";
|
||||
private String upstreamUsername = "";
|
||||
private String upstreamToken = "";
|
||||
private Duration timeout = Duration.ofMinutes(10);
|
||||
@Positive private long maxPackBytes = 5L * 1024 * 1024 * 1024;
|
||||
@Positive private long maxLfsObjectBytes = 1024L * 1024 * 1024;
|
||||
|
||||
@Min(1024)
|
||||
@Max(16777216)
|
||||
private int maxMetadataBytes = 8 * 1024 * 1024;
|
||||
|
||||
@Min(1)
|
||||
@Max(500)
|
||||
private int maxBatchObjects = 500;
|
||||
|
||||
@Min(1)
|
||||
@Max(32)
|
||||
private int readConcurrent = 4;
|
||||
|
||||
@Min(1)
|
||||
@Max(32)
|
||||
private int pushConcurrent = 2;
|
||||
|
||||
@Min(1)
|
||||
@Max(64)
|
||||
private int lfsConcurrent = 8;
|
||||
|
||||
@Min(1)
|
||||
@Max(10)
|
||||
private int maxActiveTokens = 10;
|
||||
|
||||
@Min(1)
|
||||
@Max(10000)
|
||||
private int gitRequestsPerMinute = 120;
|
||||
|
||||
@Min(1)
|
||||
@Max(10000)
|
||||
private int lfsRequestsPerMinute = 600;
|
||||
|
||||
@Min(1)
|
||||
@Max(1000)
|
||||
private int authFailuresPerMinute = 20;
|
||||
|
||||
@AssertTrue(message = "Git 网关启用时必须配置公网 HTTPS 基础地址(以 /git 结尾)和独立技术账号凭据")
|
||||
public boolean isConfigurationValid() {
|
||||
if (!enabled) return true;
|
||||
try {
|
||||
URI uri = URI.create(publicBaseUrl);
|
||||
return "https".equals(uri.getScheme())
|
||||
&& uri.getHost() != null
|
||||
&& "/git".equals(uri.getRawPath())
|
||||
&& uri.getRawUserInfo() == null
|
||||
&& uri.getRawQuery() == null
|
||||
&& uri.getRawFragment() == null
|
||||
&& upstreamUsername != null
|
||||
&& !upstreamUsername.isBlank()
|
||||
&& !upstreamUsername.contains(":")
|
||||
&& upstreamToken != null
|
||||
&& !upstreamToken.isBlank();
|
||||
} catch (IllegalArgumentException error) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
@AssertTrue(message = "Git 请求预算必须在 100 毫秒至 1 小时之间")
|
||||
public boolean isTimeoutValid() {
|
||||
return timeout != null
|
||||
&& timeout.compareTo(Duration.ofMillis(100)) >= 0
|
||||
&& timeout.compareTo(Duration.ofHours(1)) <= 0;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,5 @@
|
||||
package com.linkerbot.vibemake.service.git;
|
||||
|
||||
/** 只在请求内部使用;不包含凭据明文,不作为 API 响应返回。 */
|
||||
public record GitAccess(
|
||||
long userId, String userCode, String tokenCode, String projectCode, boolean writable) {}
|
||||
@@ -0,0 +1,294 @@
|
||||
package com.linkerbot.vibemake.service.git;
|
||||
|
||||
import com.linkerbot.vibemake.bean.GitContracts.ConnectionView;
|
||||
import com.linkerbot.vibemake.bean.entity.Project;
|
||||
import com.linkerbot.vibemake.common.BusinessException;
|
||||
import com.linkerbot.vibemake.service.ProjectService;
|
||||
import com.linkerbot.vibemake.service.UserService;
|
||||
import com.linkerbot.vibemake.service.config.GitGatewayProperties;
|
||||
import com.linkerbot.vibemake.service.gitea.GiteaClient;
|
||||
import com.linkerbot.vibemake.service.write.RepositoryWrites;
|
||||
|
||||
import jakarta.servlet.http.*;
|
||||
|
||||
import lombok.RequiredArgsConstructor;
|
||||
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
import org.springframework.stereotype.Service;
|
||||
|
||||
import java.io.*;
|
||||
import java.util.*;
|
||||
import java.util.zip.GZIPInputStream;
|
||||
|
||||
@Service
|
||||
@RequiredArgsConstructor
|
||||
public class GitGateway {
|
||||
private static final Logger log = LoggerFactory.getLogger(GitGateway.class);
|
||||
private final GitTokens tokens;
|
||||
private final GitTransport transport;
|
||||
private final GitGatewayProperties properties;
|
||||
private final ProjectService projects;
|
||||
private final UserService users;
|
||||
private final GiteaClient gitea;
|
||||
private final GitPolicy policy;
|
||||
private final RepositoryWrites writes;
|
||||
private final GitLimits limits;
|
||||
|
||||
public ConnectionView connection(long userId, String code) {
|
||||
tokens.enabled();
|
||||
Project project = projects.readyOwner(userId, code);
|
||||
boolean canPush =
|
||||
"IDLE".equals(project.getRepoWriteStatus()) && writes.active(code) == null;
|
||||
if (canPush) {
|
||||
try {
|
||||
policy.verify(project);
|
||||
} catch (BusinessException error) {
|
||||
canPush = false;
|
||||
}
|
||||
}
|
||||
return new ConnectionView(
|
||||
properties.getPublicBaseUrl() + "/projects/" + code + ".git",
|
||||
users.require(userId).getUserCode(),
|
||||
"main",
|
||||
gitea.head(projects.repository(project)),
|
||||
project.getPublishedCommitSha(),
|
||||
project.getRepoWriteStatus(),
|
||||
canPush);
|
||||
}
|
||||
|
||||
public void serve(
|
||||
GitAccess access,
|
||||
String endpoint,
|
||||
HttpServletRequest request,
|
||||
HttpServletResponse response)
|
||||
throws IOException {
|
||||
String service = "info/refs".equals(endpoint) ? request.getParameter("service") : endpoint;
|
||||
boolean advertisement = "info/refs".equals(endpoint);
|
||||
if (!("git-upload-pack".equals(service) || "git-receive-pack".equals(service))
|
||||
|| !(advertisement ? "GET" : "POST").equals(request.getMethod()))
|
||||
throw GitIo.invalid();
|
||||
if (request.getParameterMap().size() != (advertisement ? 1 : 0)) throw GitIo.invalid();
|
||||
boolean push = "git-receive-pack".equals(service);
|
||||
if (push) GitTokens.requireWrite(access);
|
||||
long started = System.nanoTime();
|
||||
java.util.concurrent.atomic.AtomicLong transferred =
|
||||
new java.util.concurrent.atomic.AtomicLong();
|
||||
log.info(
|
||||
"Git 请求开始 projectCode={} userCode={} tokenCode={} endpoint={}",
|
||||
access.projectCode(),
|
||||
access.userCode(),
|
||||
access.tokenCode(),
|
||||
endpoint);
|
||||
try (AutoCloseable permit = limits.acquire(push, false);
|
||||
GitTransport.Budget budget = transport.budget()) {
|
||||
budget.register(request.getInputStream());
|
||||
budget.onTimeout(
|
||||
() -> {
|
||||
if (response.isCommitted()) response.getOutputStream().close();
|
||||
});
|
||||
Project project = projects.readyOwner(access.userId(), access.projectCode());
|
||||
Map<String, String> headers = new HashMap<>();
|
||||
headers.put(
|
||||
"Accept",
|
||||
"application/x-" + service + (advertisement ? "-advertisement" : "-result"));
|
||||
if (!push && request.getHeader("Git-Protocol") != null) {
|
||||
if (!request.getHeader("Git-Protocol").matches("version=[012]"))
|
||||
throw GitIo.invalid();
|
||||
headers.put("Git-Protocol", request.getHeader("Git-Protocol"));
|
||||
}
|
||||
String path =
|
||||
transport.repositoryPath(project)
|
||||
+ "/"
|
||||
+ endpoint
|
||||
+ (advertisement ? "?service=" + service : "");
|
||||
if (push && !advertisement) {
|
||||
requireContentType(request, "application/x-git-receive-pack-request");
|
||||
InputStream input =
|
||||
GitIo.counted(
|
||||
decoded(request, properties.getMaxPackBytes(), budget),
|
||||
transferred);
|
||||
GitIo.PushCommand command = GitIo.command(input);
|
||||
if (command == null) {
|
||||
if (input.read() != -1) throw GitIo.invalid();
|
||||
response.setContentType("application/x-git-receive-pack-result");
|
||||
response.setContentLength(0);
|
||||
return;
|
||||
}
|
||||
java.util.concurrent.atomic.AtomicReference<Project> pushing =
|
||||
new java.util.concurrent.atomic.AtomicReference<>();
|
||||
byte[] result =
|
||||
writes.push(
|
||||
access,
|
||||
command,
|
||||
() -> {
|
||||
GitAccess current = tokens.recheck(access);
|
||||
GitTokens.requireWrite(current);
|
||||
Project fresh =
|
||||
projects.readyOwner(
|
||||
current.userId(), current.projectCode());
|
||||
if (!command.oldSha()
|
||||
.equals(gitea.head(projects.repository(fresh))))
|
||||
throw new BusinessException(
|
||||
409, "GIT_STALE_MAIN", "main 已变化,请先 pull、合并后再推送");
|
||||
policy.verify(fresh);
|
||||
budget.check();
|
||||
pushing.set(fresh);
|
||||
return fresh;
|
||||
},
|
||||
() -> {
|
||||
headers.put(
|
||||
"Content-Type",
|
||||
"application/x-git-receive-pack-request");
|
||||
InputStream body =
|
||||
new SequenceInputStream(
|
||||
new ByteArrayInputStream(command.prefix()),
|
||||
input);
|
||||
try (GitTransport.Response upstream =
|
||||
transport.exchange(
|
||||
transport.repositoryPath(pushing.get())
|
||||
+ "/git-receive-pack",
|
||||
"POST",
|
||||
body,
|
||||
-1,
|
||||
headers,
|
||||
budget)) {
|
||||
requireSuccess(
|
||||
upstream, "application/x-git-receive-pack-result");
|
||||
return GitIo.metadata(
|
||||
upstream.input(), properties.getMaxMetadataBytes());
|
||||
} catch (IOException error) {
|
||||
budget.check();
|
||||
throw GitIo.upstreamInvalid();
|
||||
}
|
||||
});
|
||||
response.setContentType("application/x-git-receive-pack-result");
|
||||
response.setContentLength(result.length);
|
||||
response.getOutputStream().write(result);
|
||||
log.info(
|
||||
"Git 推送响应完成 projectCode={} userCode={} tokenCode={} oldSha={} newSha={}"
|
||||
+ " bytes={} durationMs={}",
|
||||
access.projectCode(),
|
||||
access.userCode(),
|
||||
access.tokenCode(),
|
||||
command.oldSha(),
|
||||
command.newSha(),
|
||||
transferred.get(),
|
||||
(System.nanoTime() - started) / 1000000);
|
||||
} else {
|
||||
InputStream body = null;
|
||||
if (!advertisement) {
|
||||
requireContentType(request, "application/x-git-upload-pack-request");
|
||||
body = decoded(request, properties.getMaxMetadataBytes(), budget);
|
||||
headers.put("Content-Type", "application/x-git-upload-pack-request");
|
||||
}
|
||||
String type =
|
||||
"application/x-" + service + (advertisement ? "-advertisement" : "-result");
|
||||
try (GitTransport.Response upstream =
|
||||
transport.exchange(path, request.getMethod(), body, -1, headers, budget)) {
|
||||
requireSuccess(upstream, type);
|
||||
response.setContentType(type);
|
||||
if (push) {
|
||||
byte[] result =
|
||||
GitIo.receiveAdvertisement(
|
||||
GitIo.metadata(
|
||||
upstream.input(),
|
||||
properties.getMaxMetadataBytes()));
|
||||
response.setContentLength(result.length);
|
||||
response.getOutputStream().write(result);
|
||||
} else {
|
||||
long bytes =
|
||||
copy(
|
||||
upstream.input(),
|
||||
response,
|
||||
properties.getMaxPackBytes(),
|
||||
budget,
|
||||
transferred);
|
||||
log.info(
|
||||
"Git 读取完成 projectCode={} userCode={} tokenCode={} bytes={}"
|
||||
+ " durationMs={}",
|
||||
access.projectCode(),
|
||||
access.userCode(),
|
||||
access.tokenCode(),
|
||||
bytes,
|
||||
(System.nanoTime() - started) / 1000000);
|
||||
}
|
||||
}
|
||||
}
|
||||
} catch (BusinessException | IOException error) {
|
||||
log.warn(
|
||||
"Git 请求失败或取消 projectCode={} userCode={} tokenCode={} endpoint={} errorType={}"
|
||||
+ " bytes={} durationMs={}",
|
||||
access.projectCode(),
|
||||
access.userCode(),
|
||||
access.tokenCode(),
|
||||
endpoint,
|
||||
error.getClass().getSimpleName(),
|
||||
transferred.get(),
|
||||
(System.nanoTime() - started) / 1000000);
|
||||
throw error;
|
||||
} catch (Exception error) {
|
||||
throw GitIo.upstreamInvalid();
|
||||
}
|
||||
}
|
||||
|
||||
public static InputStream decoded(
|
||||
HttpServletRequest request, long maximum, GitTransport.Budget budget)
|
||||
throws IOException {
|
||||
if (request.getContentLengthLong() > maximum)
|
||||
throw new BusinessException(413, "GIT_TRANSFER_TOO_LARGE", "Git/LFS 请求过大");
|
||||
InputStream raw = GitIo.limited(request.getInputStream(), maximum);
|
||||
budget.register(raw);
|
||||
String encoding = request.getHeader("Content-Encoding");
|
||||
if (encoding != null && !"identity".equalsIgnoreCase(encoding)) {
|
||||
if (!"gzip".equalsIgnoreCase(encoding)) throw GitIo.invalid();
|
||||
raw = new GZIPInputStream(raw);
|
||||
}
|
||||
return GitIo.limited(raw, maximum);
|
||||
}
|
||||
|
||||
public static void requireContentType(HttpServletRequest request, String type) {
|
||||
if (request.getContentType() == null
|
||||
|| !request.getContentType().split(";", 2)[0].equalsIgnoreCase(type))
|
||||
throw GitIo.invalid();
|
||||
}
|
||||
|
||||
public static void requireSuccess(GitTransport.Response response, String type) {
|
||||
if (response.status() != 200
|
||||
|| !response.headers()
|
||||
.firstValue("Content-Type")
|
||||
.orElse("")
|
||||
.split(";", 2)[0]
|
||||
.equalsIgnoreCase(type)) throw GitIo.upstreamInvalid();
|
||||
}
|
||||
|
||||
public static long copy(
|
||||
InputStream input,
|
||||
HttpServletResponse response,
|
||||
long maximum,
|
||||
GitTransport.Budget budget)
|
||||
throws IOException {
|
||||
return copy(input, response, maximum, budget, new java.util.concurrent.atomic.AtomicLong());
|
||||
}
|
||||
|
||||
public static long copy(
|
||||
InputStream input,
|
||||
HttpServletResponse response,
|
||||
long maximum,
|
||||
GitTransport.Budget budget,
|
||||
java.util.concurrent.atomic.AtomicLong transferred)
|
||||
throws IOException {
|
||||
byte[] buffer = new byte[65536];
|
||||
long total = 0;
|
||||
int count;
|
||||
InputStream bounded = GitIo.limited(input, maximum);
|
||||
while ((count = bounded.read(buffer)) >= 0) {
|
||||
budget.check();
|
||||
response.getOutputStream().write(buffer, 0, count);
|
||||
total += count;
|
||||
transferred.addAndGet(count);
|
||||
}
|
||||
budget.check();
|
||||
return total;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,186 @@
|
||||
package com.linkerbot.vibemake.service.git;
|
||||
|
||||
import com.linkerbot.vibemake.common.BusinessException;
|
||||
|
||||
import java.io.*;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.util.Set;
|
||||
|
||||
/** Git pkt-line 的有界元数据处理;pack 正文始终以流处理。 */
|
||||
public final class GitIo {
|
||||
private GitIo() {}
|
||||
|
||||
public record PushCommand(String oldSha, String newSha, byte[] prefix) {}
|
||||
|
||||
private static final Set<String> PUSH_CAPS = Set.of("report-status", "ofs-delta", "quiet");
|
||||
|
||||
public static InputStream counted(
|
||||
InputStream input, java.util.concurrent.atomic.AtomicLong count) {
|
||||
return new FilterInputStream(input) {
|
||||
@Override
|
||||
public int read() throws IOException {
|
||||
byte[] one = new byte[1];
|
||||
return read(one, 0, 1) < 0 ? -1 : one[0] & 255;
|
||||
}
|
||||
|
||||
@Override
|
||||
public int read(byte[] bytes, int offset, int length) throws IOException {
|
||||
int size = super.read(bytes, offset, length);
|
||||
if (size > 0) count.addAndGet(size);
|
||||
return size;
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
public static InputStream limited(InputStream input, long maximum) {
|
||||
return new FilterInputStream(input) {
|
||||
private long count;
|
||||
|
||||
@Override
|
||||
public int read() throws IOException {
|
||||
byte[] bytes = new byte[1];
|
||||
return read(bytes, 0, 1) < 0 ? -1 : bytes[0] & 255;
|
||||
}
|
||||
|
||||
@Override
|
||||
public int read(byte[] bytes, int offset, int length) throws IOException {
|
||||
if (length == 0) return 0;
|
||||
long remaining = maximum - count;
|
||||
int size =
|
||||
super.read(
|
||||
bytes, offset, remaining >= length ? length : (int) remaining + 1);
|
||||
if (size > 0 && (count += size) > maximum)
|
||||
throw new BusinessException(413, "GIT_TRANSFER_TOO_LARGE", "Git/LFS 传输超过大小限制");
|
||||
return size;
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
public static byte[] metadata(InputStream input, int maximum) throws IOException {
|
||||
byte[] bytes = input.readNBytes(maximum + 1);
|
||||
if (bytes.length > maximum)
|
||||
throw new BusinessException(413, "GIT_METADATA_TOO_LARGE", "Git/LFS 元数据超过限制");
|
||||
return bytes;
|
||||
}
|
||||
|
||||
public static byte[] packet(InputStream input) throws IOException {
|
||||
byte[] header = input.readNBytes(4);
|
||||
if (header.length != 4) throw invalid();
|
||||
String hex = new String(header, StandardCharsets.US_ASCII);
|
||||
if (!hex.matches("[0-9a-fA-F]{4}")) throw invalid();
|
||||
int size = Integer.parseInt(hex, 16);
|
||||
if (size == 0) return null;
|
||||
if (size < 4 || size > 65520) throw invalid();
|
||||
byte[] data = input.readNBytes(size - 4);
|
||||
if (data.length != size - 4) throw invalid();
|
||||
return data;
|
||||
}
|
||||
|
||||
public static void packet(OutputStream output, byte[] data) throws IOException {
|
||||
output.write(
|
||||
String.format(java.util.Locale.ROOT, "%04x", data.length + 4)
|
||||
.getBytes(StandardCharsets.US_ASCII));
|
||||
output.write(data);
|
||||
}
|
||||
|
||||
private static boolean capability(String value) {
|
||||
return PUSH_CAPS.contains(value)
|
||||
|| value.matches("agent=[!-~]{1,128}")
|
||||
|| value.equals("object-format=sha1")
|
||||
|| value.equals("object-format=sha256");
|
||||
}
|
||||
|
||||
public static PushCommand command(InputStream input) throws IOException {
|
||||
byte[] first = packet(input);
|
||||
// Smart HTTP 在大 pack 分块上传前会单独 POST 一个 flush 进行认证探测。
|
||||
if (first == null) return null;
|
||||
if (first.length > 4096) throw invalid();
|
||||
String command = new String(first, StandardCharsets.UTF_8).stripTrailing();
|
||||
int nul = command.indexOf('\0');
|
||||
if (nul < 0) throw invalid();
|
||||
String[] fields = command.substring(0, nul).split(" ", -1);
|
||||
if (fields.length != 3 || !"refs/heads/main".equals(fields[2])) throw invalid();
|
||||
String oldSha = com.linkerbot.vibemake.common.FilePaths.sha(fields[0]);
|
||||
String newSha = com.linkerbot.vibemake.common.FilePaths.sha(fields[1]);
|
||||
if (oldSha.length() != newSha.length() || oldSha.matches("0+") || newSha.matches("0+"))
|
||||
throw invalid();
|
||||
// 原生 send-pack 会在 NUL 后的能力列表前保留一个空格。
|
||||
String[] caps = command.substring(nul + 1).strip().split(" ");
|
||||
boolean report = false;
|
||||
for (String cap : caps) {
|
||||
if (!capability(cap)) throw invalid();
|
||||
if (cap.equals("report-status")) report = true;
|
||||
}
|
||||
if (!report || packet(input) != null) throw invalid();
|
||||
ByteArrayOutputStream prefix = new ByteArrayOutputStream();
|
||||
packet(prefix, first);
|
||||
prefix.write("0000".getBytes(StandardCharsets.US_ASCII));
|
||||
return new PushCommand(oldSha, newSha, prefix.toByteArray());
|
||||
}
|
||||
|
||||
public static byte[] receiveAdvertisement(byte[] bytes) throws IOException {
|
||||
ByteArrayInputStream input = new ByteArrayInputStream(bytes);
|
||||
byte[] service = upstreamPacket(input);
|
||||
if (service == null
|
||||
|| !new String(service, StandardCharsets.US_ASCII)
|
||||
.equals("# service=git-receive-pack\n")
|
||||
|| upstreamPacket(input) != null) throw upstreamInvalid();
|
||||
String main = null;
|
||||
String caps = null;
|
||||
byte[] entry;
|
||||
while ((entry = upstreamPacket(input)) != null) {
|
||||
String line = new String(entry, StandardCharsets.UTF_8).stripTrailing();
|
||||
int nul = line.indexOf('\0');
|
||||
if (nul >= 0) {
|
||||
caps = line.substring(nul + 1);
|
||||
line = line.substring(0, nul);
|
||||
}
|
||||
if (line.matches("[0-9a-fA-F]{40,64} refs/heads/main")) main = line;
|
||||
}
|
||||
if (main == null || caps == null || input.available() != 0) throw upstreamInvalid();
|
||||
StringBuilder filtered = new StringBuilder();
|
||||
for (String cap : caps.split(" ")) if (capability(cap)) filtered.append(cap).append(' ');
|
||||
if (!java.util.Arrays.asList(caps.split(" ")).contains("report-status"))
|
||||
throw upstreamInvalid();
|
||||
ByteArrayOutputStream output = new ByteArrayOutputStream();
|
||||
packet(output, service);
|
||||
output.write("0000".getBytes(StandardCharsets.US_ASCII));
|
||||
packet(
|
||||
output,
|
||||
(main + "\0" + filtered.toString().trim() + "\n").getBytes(StandardCharsets.UTF_8));
|
||||
output.write("0000".getBytes(StandardCharsets.US_ASCII));
|
||||
return output.toByteArray();
|
||||
}
|
||||
|
||||
public static boolean pushSucceeded(byte[] bytes) throws IOException {
|
||||
ByteArrayInputStream input = new ByteArrayInputStream(bytes);
|
||||
byte[] unpack = upstreamPacket(input);
|
||||
if (unpack == null) throw upstreamInvalid();
|
||||
String unpackStatus = new String(unpack, StandardCharsets.UTF_8).stripTrailing();
|
||||
byte[] ref = upstreamPacket(input);
|
||||
if (ref == null) throw upstreamInvalid();
|
||||
String refStatus = new String(ref, StandardCharsets.UTF_8).stripTrailing();
|
||||
if (!unpackStatus.startsWith("unpack ")
|
||||
|| !(refStatus.equals("ok refs/heads/main")
|
||||
|| refStatus.startsWith("ng refs/heads/main "))
|
||||
|| upstreamPacket(input) != null
|
||||
|| input.available() != 0) throw upstreamInvalid();
|
||||
return unpackStatus.equals("unpack ok") && refStatus.equals("ok refs/heads/main");
|
||||
}
|
||||
|
||||
private static byte[] upstreamPacket(InputStream input) throws IOException {
|
||||
try {
|
||||
return packet(input);
|
||||
} catch (BusinessException error) {
|
||||
throw upstreamInvalid();
|
||||
}
|
||||
}
|
||||
|
||||
public static BusinessException invalid() {
|
||||
return new BusinessException(400, "GIT_COMMAND_REJECTED", "仅支持一次 main 更新,禁止删除、其他引用及扩展推送选项");
|
||||
}
|
||||
|
||||
public static BusinessException upstreamInvalid() {
|
||||
return new BusinessException(502, "GIT_UPSTREAM_ERROR", "Git 上游响应不完整或格式异常");
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,62 @@
|
||||
package com.linkerbot.vibemake.service.git;
|
||||
|
||||
import com.linkerbot.vibemake.common.BusinessException;
|
||||
import com.linkerbot.vibemake.common.state.TransientStateStore;
|
||||
import com.linkerbot.vibemake.service.config.GitGatewayProperties;
|
||||
|
||||
import org.springframework.stereotype.Component;
|
||||
|
||||
import java.time.Duration;
|
||||
import java.util.concurrent.Semaphore;
|
||||
|
||||
@Component
|
||||
public class GitLimits {
|
||||
private final TransientStateStore state;
|
||||
private final GitGatewayProperties properties;
|
||||
private final Semaphore reads;
|
||||
private final Semaphore pushes;
|
||||
private final Semaphore lfs;
|
||||
|
||||
public GitLimits(TransientStateStore state, GitGatewayProperties properties) {
|
||||
this.state = state;
|
||||
this.properties = properties;
|
||||
reads = new Semaphore(properties.getReadConcurrent());
|
||||
pushes = new Semaphore(properties.getPushConcurrent());
|
||||
lfs = new Semaphore(properties.getLfsConcurrent());
|
||||
}
|
||||
|
||||
public void authenticationAllowed(String ip) {
|
||||
String count =
|
||||
state.get("git:auth:" + com.linkerbot.vibemake.common.SecurityHashes.sha256(ip));
|
||||
if (count != null && Long.parseLong(count) >= properties.getAuthFailuresPerMinute())
|
||||
throw busy();
|
||||
}
|
||||
|
||||
public void authenticationFailed(String ip) {
|
||||
state.increment(
|
||||
"git:auth:" + com.linkerbot.vibemake.common.SecurityHashes.sha256(ip),
|
||||
Duration.ofMinutes(1),
|
||||
false);
|
||||
}
|
||||
|
||||
public void request(GitAccess access, boolean isLfs) {
|
||||
int maximum =
|
||||
isLfs ? properties.getLfsRequestsPerMinute() : properties.getGitRequestsPerMinute();
|
||||
if (state.increment(
|
||||
"git:rate:" + (isLfs ? "lfs:" : "git:") + access.userCode(),
|
||||
Duration.ofMinutes(1),
|
||||
false)
|
||||
.count()
|
||||
> maximum) throw busy();
|
||||
}
|
||||
|
||||
public AutoCloseable acquire(boolean push, boolean isLfs) {
|
||||
Semaphore pool = isLfs ? lfs : push ? pushes : reads;
|
||||
if (!pool.tryAcquire()) throw busy();
|
||||
return pool::release;
|
||||
}
|
||||
|
||||
private BusinessException busy() {
|
||||
return new BusinessException(429, "GIT_BUSY", "Git/LFS 请求过于频繁或服务繁忙,请稍后重试");
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,79 @@
|
||||
package com.linkerbot.vibemake.service.git;
|
||||
|
||||
import com.fasterxml.jackson.databind.JsonNode;
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
import com.linkerbot.vibemake.bean.entity.Project;
|
||||
import com.linkerbot.vibemake.common.BusinessException;
|
||||
import com.linkerbot.vibemake.service.ProjectService;
|
||||
import com.linkerbot.vibemake.service.config.GitGatewayProperties;
|
||||
import com.linkerbot.vibemake.service.gitea.GiteaClient;
|
||||
|
||||
import lombok.RequiredArgsConstructor;
|
||||
|
||||
import org.springframework.stereotype.Service;
|
||||
|
||||
import java.io.IOException;
|
||||
import java.util.Map;
|
||||
|
||||
@Service
|
||||
@RequiredArgsConstructor
|
||||
public class GitPolicy {
|
||||
private final GiteaClient gitea;
|
||||
private final ProjectService projects;
|
||||
private final GitTransport transport;
|
||||
private final GitGatewayProperties properties;
|
||||
private final ObjectMapper json;
|
||||
|
||||
public void ensure(Project project) {
|
||||
identity(project);
|
||||
gitea.ensureGitProtection(projects.repository(project), properties.getUpstreamUsername());
|
||||
}
|
||||
|
||||
public void provision(Project project) {
|
||||
identity(project);
|
||||
gitea.ensureGitAccess(projects.repository(project), properties.getUpstreamUsername());
|
||||
}
|
||||
|
||||
public void verify(Project project) {
|
||||
identity(project);
|
||||
gitea.verifyGitProtection(projects.repository(project), properties.getUpstreamUsername());
|
||||
}
|
||||
|
||||
private void identity(Project project) {
|
||||
try (GitTransport.Budget budget = transport.budget();
|
||||
GitTransport.Response response =
|
||||
transport.exchange(
|
||||
"/api/v1/user",
|
||||
"GET",
|
||||
null,
|
||||
-1,
|
||||
Map.of("Accept", "application/json"),
|
||||
budget)) {
|
||||
if (response.status() != 200) throw invalid();
|
||||
JsonNode user =
|
||||
json.readTree(
|
||||
GitIo.metadata(response.input(), properties.getMaxMetadataBytes()));
|
||||
if (user == null
|
||||
|| !properties.getUpstreamUsername().equals(user.path("login").asText())
|
||||
|| !user.has("is_admin")
|
||||
|| user.path("is_admin").asBoolean()) throw invalid();
|
||||
GiteaClient.RemoteRepository remote = gitea.inspect(project.getGiteaRepoName());
|
||||
if (!java.util.Objects.equals(project.getGiteaRepoId(), remote.id())
|
||||
|| !project.getGiteaOwner().equals(remote.owner())
|
||||
|| !project.getGiteaRepoName().equals(remote.name())
|
||||
|| !remote.privateRepo()
|
||||
|| remote.mirror()
|
||||
|| remote.empty()
|
||||
|| !"main".equals(remote.defaultBranch()))
|
||||
throw new BusinessException(
|
||||
503, "GIT_POLICY_UNAVAILABLE", "Git 仓库身份或 main 配置不兼容,请联系管理员");
|
||||
budget.check();
|
||||
} catch (IOException error) {
|
||||
throw invalid();
|
||||
}
|
||||
}
|
||||
|
||||
private BusinessException invalid() {
|
||||
return new BusinessException(503, "GIT_POLICY_UNAVAILABLE", "Git 必须配置身份匹配的独立非管理员技术账号");
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,222 @@
|
||||
package com.linkerbot.vibemake.service.git;
|
||||
|
||||
import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper;
|
||||
import com.baomidou.mybatisplus.core.conditions.update.LambdaUpdateWrapper;
|
||||
import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
|
||||
import com.linkerbot.vibemake.bean.GitContracts.*;
|
||||
import com.linkerbot.vibemake.bean.entity.Project;
|
||||
import com.linkerbot.vibemake.bean.entity.ProjectGitToken;
|
||||
import com.linkerbot.vibemake.bean.entity.User;
|
||||
import com.linkerbot.vibemake.common.*;
|
||||
import com.linkerbot.vibemake.service.ProjectService;
|
||||
import com.linkerbot.vibemake.service.UserService;
|
||||
import com.linkerbot.vibemake.service.config.GitGatewayProperties;
|
||||
import com.linkerbot.vibemake.service.mapper.ProjectGitTokenMapper;
|
||||
import com.linkerbot.vibemake.service.support.ProjectLocks;
|
||||
import com.linkerbot.vibemake.service.support.ProjectSupport;
|
||||
|
||||
import lombok.RequiredArgsConstructor;
|
||||
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
import org.springframework.stereotype.Service;
|
||||
|
||||
import java.security.SecureRandom;
|
||||
import java.time.LocalDateTime;
|
||||
import java.util.Base64;
|
||||
|
||||
@Service
|
||||
@RequiredArgsConstructor
|
||||
public class GitTokens {
|
||||
private static final Logger log = LoggerFactory.getLogger(GitTokens.class);
|
||||
private final ProjectGitTokenMapper tokens;
|
||||
private final ProjectService projects;
|
||||
private final UserService users;
|
||||
private final ProjectLocks locks;
|
||||
private final GitPolicy policy;
|
||||
private final GitGatewayProperties properties;
|
||||
private final SecureRandom random = new SecureRandom();
|
||||
|
||||
public void enabled() {
|
||||
if (!properties.isEnabled())
|
||||
throw new BusinessException(503, "GIT_UNAVAILABLE", "Git 接入尚未启用");
|
||||
}
|
||||
|
||||
public IssuedToken issue(long userId, String code, TokenInput input) {
|
||||
enabled();
|
||||
return locks.serial(
|
||||
code,
|
||||
() -> {
|
||||
Project project = projects.readyOwner(userId, code);
|
||||
User user = users.require(userId);
|
||||
requireRole(user);
|
||||
LocalDateTime now = ProjectSupport.now();
|
||||
long active =
|
||||
tokens.selectCount(
|
||||
new LambdaQueryWrapper<ProjectGitToken>()
|
||||
.eq(ProjectGitToken::getProjectCode, code)
|
||||
.eq(ProjectGitToken::getOwnerUserId, userId)
|
||||
.eq(
|
||||
ProjectGitToken::getSessionVersion,
|
||||
user.getSessionVersion())
|
||||
.isNull(ProjectGitToken::getRevokedAt)
|
||||
.gt(ProjectGitToken::getExpiresAt, now));
|
||||
if (active >= properties.getMaxActiveTokens())
|
||||
throw BusinessException.conflict("有效 Git 凭据已达上限,请先撤销旧凭据");
|
||||
policy.provision(project);
|
||||
if (input.scope() == Scope.READ_WRITE) policy.ensure(project);
|
||||
byte[] bytes = new byte[32];
|
||||
random.nextBytes(bytes);
|
||||
String secret =
|
||||
"vbg_" + Base64.getUrlEncoder().withoutPadding().encodeToString(bytes);
|
||||
ProjectGitToken token = new ProjectGitToken();
|
||||
token.setTokenCode(Codes.next());
|
||||
token.setProjectCode(code);
|
||||
token.setOwnerUserId(userId);
|
||||
token.setTokenName(input.name().trim());
|
||||
token.setTokenHash(SecurityHashes.sha256(secret));
|
||||
token.setPermission(input.scope().name());
|
||||
token.setSessionVersion(user.getSessionVersion());
|
||||
token.setExpiresAt(
|
||||
now.plusDays(
|
||||
input.expiresInDays() == null ? 90 : input.expiresInDays()));
|
||||
token.setCreatedAt(now);
|
||||
token.setUpdatedAt(now);
|
||||
tokens.insert(token);
|
||||
log.info(
|
||||
"Git 凭据创建 projectCode={} userCode={} tokenCode={} permission={}",
|
||||
code,
|
||||
user.getUserCode(),
|
||||
token.getTokenCode(),
|
||||
token.getPermission());
|
||||
return new IssuedToken(view(token, user.getSessionVersion()), secret);
|
||||
});
|
||||
}
|
||||
|
||||
public PageResponse<TokenView> list(long userId, String code, long page, long size) {
|
||||
projects.owner(userId, code);
|
||||
if (page < 1 || page > 100000 || size < 1 || size > 100)
|
||||
throw BusinessException.badRequest("分页参数不合法");
|
||||
long version = users.require(userId).getSessionVersion();
|
||||
Page<ProjectGitToken> result =
|
||||
tokens.selectPage(
|
||||
new Page<>(page, size),
|
||||
new LambdaQueryWrapper<ProjectGitToken>()
|
||||
.eq(ProjectGitToken::getProjectCode, code)
|
||||
.eq(ProjectGitToken::getOwnerUserId, userId)
|
||||
.orderByDesc(ProjectGitToken::getCreatedAt)
|
||||
.orderByDesc(ProjectGitToken::getTokenCode));
|
||||
return new PageResponse<>(
|
||||
result.getRecords().stream().map(token -> view(token, version)).toList(),
|
||||
page,
|
||||
size,
|
||||
result.getTotal());
|
||||
}
|
||||
|
||||
public void revoke(long userId, String code, String tokenCode) {
|
||||
projects.owner(userId, code);
|
||||
ProjectGitToken token =
|
||||
tokens.selectOne(
|
||||
new LambdaQueryWrapper<ProjectGitToken>()
|
||||
.eq(ProjectGitToken::getTokenCode, tokenCode)
|
||||
.eq(ProjectGitToken::getProjectCode, code)
|
||||
.eq(ProjectGitToken::getOwnerUserId, userId));
|
||||
if (token == null) throw BusinessException.notFound();
|
||||
LocalDateTime now = ProjectSupport.now();
|
||||
tokens.update(
|
||||
null,
|
||||
new LambdaUpdateWrapper<ProjectGitToken>()
|
||||
.eq(ProjectGitToken::getTokenCode, tokenCode)
|
||||
.isNull(ProjectGitToken::getRevokedAt)
|
||||
.set(ProjectGitToken::getRevokedAt, now)
|
||||
.set(ProjectGitToken::getUpdatedAt, now));
|
||||
log.info("Git 凭据撤销 projectCode={} tokenCode={}", code, tokenCode);
|
||||
}
|
||||
|
||||
public GitAccess authenticate(String username, String secret, String code) {
|
||||
enabled();
|
||||
if (secret == null || !secret.matches("vbg_[A-Za-z0-9_-]{43}")) throw unauthorized();
|
||||
ProjectGitToken token =
|
||||
tokens.selectOne(
|
||||
new LambdaQueryWrapper<ProjectGitToken>()
|
||||
.eq(ProjectGitToken::getTokenHash, SecurityHashes.sha256(secret)));
|
||||
GitAccess access = validate(token, code);
|
||||
if (!access.userCode().equals(username)) throw unauthorized();
|
||||
touch(token);
|
||||
return access;
|
||||
}
|
||||
|
||||
public GitAccess recheck(GitAccess access) {
|
||||
enabled();
|
||||
ProjectGitToken token =
|
||||
tokens.selectOne(
|
||||
new LambdaQueryWrapper<ProjectGitToken>()
|
||||
.eq(ProjectGitToken::getTokenCode, access.tokenCode()));
|
||||
GitAccess current = validate(token, access.projectCode());
|
||||
if (current.userId() != access.userId()) throw unauthorized();
|
||||
return current;
|
||||
}
|
||||
|
||||
private GitAccess validate(ProjectGitToken token, String code) {
|
||||
if (token == null
|
||||
|| token.getRevokedAt() != null
|
||||
|| !token.getExpiresAt().isAfter(ProjectSupport.now())) throw unauthorized();
|
||||
if (!token.getProjectCode().equals(code)) throw BusinessException.notFound();
|
||||
User user = users.require(token.getOwnerUserId());
|
||||
requireRole(user);
|
||||
if (!user.getSessionVersion().equals(token.getSessionVersion())) throw unauthorized();
|
||||
projects.readyOwner(user.getId(), code);
|
||||
if (!"READ_ONLY".equals(token.getPermission())
|
||||
&& !"READ_WRITE".equals(token.getPermission())) throw unauthorized();
|
||||
return new GitAccess(
|
||||
user.getId(),
|
||||
user.getUserCode(),
|
||||
token.getTokenCode(),
|
||||
code,
|
||||
"READ_WRITE".equals(token.getPermission()));
|
||||
}
|
||||
|
||||
private void touch(ProjectGitToken token) {
|
||||
LocalDateTime now = ProjectSupport.now();
|
||||
tokens.update(
|
||||
null,
|
||||
new LambdaUpdateWrapper<ProjectGitToken>()
|
||||
.eq(ProjectGitToken::getTokenCode, token.getTokenCode())
|
||||
.and(
|
||||
query ->
|
||||
query.isNull(ProjectGitToken::getLastUsedAt)
|
||||
.or()
|
||||
.lt(
|
||||
ProjectGitToken::getLastUsedAt,
|
||||
now.minusMinutes(5)))
|
||||
.set(ProjectGitToken::getLastUsedAt, now));
|
||||
}
|
||||
|
||||
private void requireRole(User user) {
|
||||
if (!"USER".equals(user.getRole()) && !"ADMIN".equals(user.getRole())) throw unauthorized();
|
||||
}
|
||||
|
||||
private TokenView view(ProjectGitToken token, long version) {
|
||||
String status =
|
||||
token.getRevokedAt() != null || token.getSessionVersion() != version
|
||||
? "REVOKED"
|
||||
: token.getExpiresAt().isAfter(ProjectSupport.now()) ? "ACTIVE" : "EXPIRED";
|
||||
return new TokenView(
|
||||
token.getTokenCode(),
|
||||
token.getTokenName(),
|
||||
Scope.valueOf(token.getPermission()),
|
||||
status,
|
||||
token.getCreatedAt(),
|
||||
token.getExpiresAt(),
|
||||
token.getRevokedAt(),
|
||||
token.getLastUsedAt());
|
||||
}
|
||||
|
||||
public static BusinessException unauthorized() {
|
||||
return new BusinessException(401, "GIT_UNAUTHORIZED", "请使用网站用户编码和有效的项目 Git Token");
|
||||
}
|
||||
|
||||
public static void requireWrite(GitAccess access) {
|
||||
if (!access.writable()) throw new BusinessException(403, "GIT_READ_ONLY", "该 Git 凭据仅允许读取");
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,242 @@
|
||||
package com.linkerbot.vibemake.service.git;
|
||||
|
||||
import com.linkerbot.vibemake.common.BusinessException;
|
||||
import com.linkerbot.vibemake.service.config.GitGatewayProperties;
|
||||
import com.linkerbot.vibemake.service.config.VibemakeProperties;
|
||||
|
||||
import jakarta.annotation.PreDestroy;
|
||||
|
||||
import org.springframework.stereotype.Component;
|
||||
|
||||
import java.io.*;
|
||||
import java.net.URI;
|
||||
import java.net.URLEncoder;
|
||||
import java.net.http.*;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.util.*;
|
||||
import java.util.concurrent.*;
|
||||
import java.util.concurrent.atomic.AtomicBoolean;
|
||||
|
||||
/** 仅访问配置的 Gitea;请求正文与响应正文都不落盘、不完整缓冲。 */
|
||||
@Component
|
||||
public class GitTransport implements AutoCloseable {
|
||||
private final GitGatewayProperties properties;
|
||||
private final VibemakeProperties config;
|
||||
private final HttpClient http;
|
||||
private final ScheduledThreadPoolExecutor timers =
|
||||
new ScheduledThreadPoolExecutor(
|
||||
2, Thread.ofPlatform().daemon().name("git-budget-", 0).factory());
|
||||
|
||||
public GitTransport(GitGatewayProperties properties, VibemakeProperties config) {
|
||||
timers.setRemoveOnCancelPolicy(true);
|
||||
this.properties = properties;
|
||||
this.config = config;
|
||||
this.http =
|
||||
HttpClient.newBuilder()
|
||||
.connectTimeout(config.getGitea().getConnectTimeout())
|
||||
.followRedirects(HttpClient.Redirect.NEVER)
|
||||
.build();
|
||||
}
|
||||
|
||||
public final class Budget implements AutoCloseable {
|
||||
private final long deadline = System.nanoTime() + properties.getTimeout().toNanos();
|
||||
private final Set<AutoCloseable> resources = ConcurrentHashMap.newKeySet();
|
||||
private final Set<AutoCloseable> timeoutResources = ConcurrentHashMap.newKeySet();
|
||||
private final AtomicBoolean expired = new AtomicBoolean();
|
||||
private final ScheduledFuture<?> timer;
|
||||
|
||||
private Budget() {
|
||||
timer =
|
||||
timers.schedule(
|
||||
() -> {
|
||||
expired.set(true);
|
||||
for (AutoCloseable resource : resources)
|
||||
Thread.startVirtualThread(() -> closeQuietly(resource));
|
||||
for (AutoCloseable resource : timeoutResources)
|
||||
Thread.startVirtualThread(() -> closeQuietly(resource));
|
||||
},
|
||||
properties.getTimeout().toNanos(),
|
||||
TimeUnit.NANOSECONDS);
|
||||
}
|
||||
|
||||
public void check() {
|
||||
if (expired.get() || System.nanoTime() - deadline >= 0) throw timeout();
|
||||
}
|
||||
|
||||
public void register(AutoCloseable resource) {
|
||||
resources.add(resource);
|
||||
if (expired.get()) {
|
||||
closeQuietly(resource);
|
||||
throw timeout();
|
||||
}
|
||||
}
|
||||
|
||||
public void onTimeout(AutoCloseable resource) {
|
||||
timeoutResources.add(resource);
|
||||
if (expired.get()) {
|
||||
closeQuietly(resource);
|
||||
throw timeout();
|
||||
}
|
||||
}
|
||||
|
||||
public java.time.Duration remaining() {
|
||||
check();
|
||||
return java.time.Duration.ofNanos(Math.max(1, deadline - System.nanoTime()));
|
||||
}
|
||||
|
||||
@Override
|
||||
public void close() {
|
||||
timer.cancel(false);
|
||||
resources.forEach(GitTransport::closeQuietly);
|
||||
resources.clear();
|
||||
timeoutResources.clear();
|
||||
}
|
||||
}
|
||||
|
||||
public record Response(int status, HttpHeaders headers, InputStream input)
|
||||
implements AutoCloseable {
|
||||
@Override
|
||||
public void close() {
|
||||
closeQuietly(input);
|
||||
}
|
||||
}
|
||||
|
||||
public Budget budget() {
|
||||
return new Budget();
|
||||
}
|
||||
|
||||
private static void closeQuietly(AutoCloseable resource) {
|
||||
try {
|
||||
resource.close();
|
||||
} catch (Exception ignored) {
|
||||
}
|
||||
}
|
||||
|
||||
public URI base() {
|
||||
URI base = URI.create(config.getGitea().getBaseUrl().replaceAll("/+$", ""));
|
||||
if (!("http".equals(base.getScheme()) || "https".equals(base.getScheme()))
|
||||
|| base.getHost() == null
|
||||
|| base.getRawUserInfo() != null
|
||||
|| base.getRawQuery() != null
|
||||
|| base.getRawFragment() != null)
|
||||
throw new BusinessException(503, "GIT_UNAVAILABLE", "Git 上游地址配置不合法");
|
||||
return base;
|
||||
}
|
||||
|
||||
public static String encode(String value) {
|
||||
return URLEncoder.encode(value, StandardCharsets.UTF_8).replace("+", "%20");
|
||||
}
|
||||
|
||||
public String repositoryPath(com.linkerbot.vibemake.bean.entity.Project project) {
|
||||
return "/"
|
||||
+ encode(project.getGiteaOwner())
|
||||
+ "/"
|
||||
+ encode(project.getGiteaRepoName())
|
||||
+ ".git";
|
||||
}
|
||||
|
||||
public Response exchange(
|
||||
String path,
|
||||
String method,
|
||||
InputStream body,
|
||||
long length,
|
||||
Map<String, String> headers,
|
||||
Budget budget) {
|
||||
return exchange(path, method, body, length, headers, budget, null);
|
||||
}
|
||||
|
||||
public Response exchange(
|
||||
String path,
|
||||
String method,
|
||||
InputStream body,
|
||||
long length,
|
||||
Map<String, String> headers,
|
||||
Budget budget,
|
||||
Map<String, String> actionHeaders) {
|
||||
budget.check();
|
||||
if (!path.startsWith("/")
|
||||
|| path.startsWith("//")
|
||||
|| path.contains("\\")
|
||||
|| path.contains("#")
|
||||
|| path.contains("/../")
|
||||
|| path.contains("/./")) throw GitIo.upstreamInvalid();
|
||||
URI destination = URI.create(base().toString() + path);
|
||||
String basic =
|
||||
Base64.getEncoder()
|
||||
.encodeToString(
|
||||
(properties.getUpstreamUsername()
|
||||
+ ":"
|
||||
+ properties.getUpstreamToken())
|
||||
.getBytes(StandardCharsets.UTF_8));
|
||||
HttpRequest.Builder builder =
|
||||
HttpRequest.newBuilder(destination)
|
||||
.timeout(budget.remaining())
|
||||
.header("Authorization", "Basic " + basic);
|
||||
headers.forEach(builder::setHeader);
|
||||
if (actionHeaders != null) actionHeaders.forEach(builder::setHeader);
|
||||
HttpRequest.BodyPublisher publisher = HttpRequest.BodyPublishers.noBody();
|
||||
if (body != null) {
|
||||
budget.register(body);
|
||||
publisher = HttpRequest.BodyPublishers.ofInputStream(() -> body);
|
||||
if (length > 0) publisher = HttpRequest.BodyPublishers.fromPublisher(publisher, length);
|
||||
}
|
||||
CompletableFuture<HttpResponse<InputStream>> future =
|
||||
http.sendAsync(
|
||||
builder.method(method, publisher).build(),
|
||||
HttpResponse.BodyHandlers.ofInputStream());
|
||||
budget.register(() -> future.cancel(true));
|
||||
try {
|
||||
HttpResponse<InputStream> response =
|
||||
future.get(budget.remaining().toNanos(), TimeUnit.NANOSECONDS);
|
||||
budget.register(response.body());
|
||||
budget.check();
|
||||
if (response.statusCode() >= 300 && response.statusCode() < 400)
|
||||
throw GitIo.upstreamInvalid();
|
||||
return new Response(
|
||||
response.statusCode(), response.headers(), checked(response.body(), budget));
|
||||
} catch (InterruptedException error) {
|
||||
Thread.currentThread().interrupt();
|
||||
throw timeout();
|
||||
} catch (TimeoutException error) {
|
||||
future.cancel(true);
|
||||
throw timeout();
|
||||
} catch (ExecutionException | CancellationException error) {
|
||||
budget.check();
|
||||
throw GitIo.upstreamInvalid();
|
||||
}
|
||||
}
|
||||
|
||||
private InputStream checked(InputStream input, Budget budget) {
|
||||
return new FilterInputStream(input) {
|
||||
@Override
|
||||
public int read() throws IOException {
|
||||
byte[] b = new byte[1];
|
||||
return read(b, 0, 1) < 0 ? -1 : b[0] & 255;
|
||||
}
|
||||
|
||||
@Override
|
||||
public int read(byte[] bytes, int offset, int length) throws IOException {
|
||||
budget.check();
|
||||
try {
|
||||
int size = super.read(bytes, offset, length);
|
||||
budget.check();
|
||||
return size;
|
||||
} catch (IOException error) {
|
||||
budget.check();
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
public static BusinessException timeout() {
|
||||
return new BusinessException(504, "GIT_TIMEOUT", "Git/LFS 请求超时;涉及推送请先刷新工作区核对结果");
|
||||
}
|
||||
|
||||
@PreDestroy
|
||||
@Override
|
||||
public void close() {
|
||||
timers.shutdownNow();
|
||||
http.shutdownNow();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,438 @@
|
||||
package com.linkerbot.vibemake.service.git;
|
||||
|
||||
import com.fasterxml.jackson.databind.*;
|
||||
import com.fasterxml.jackson.databind.node.*;
|
||||
import com.linkerbot.vibemake.bean.entity.Project;
|
||||
import com.linkerbot.vibemake.common.*;
|
||||
import com.linkerbot.vibemake.common.state.TransientStateStore;
|
||||
import com.linkerbot.vibemake.service.ProjectService;
|
||||
import com.linkerbot.vibemake.service.config.GitGatewayProperties;
|
||||
|
||||
import jakarta.servlet.http.*;
|
||||
|
||||
import lombok.RequiredArgsConstructor;
|
||||
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
import org.springframework.stereotype.Service;
|
||||
|
||||
import java.io.*;
|
||||
import java.net.URI;
|
||||
import java.security.*;
|
||||
import java.time.*;
|
||||
import java.util.*;
|
||||
|
||||
@Service
|
||||
@RequiredArgsConstructor
|
||||
public class LfsGateway {
|
||||
public static final String MEDIA = "application/vnd.git-lfs+json";
|
||||
private static final Logger log = LoggerFactory.getLogger(LfsGateway.class);
|
||||
private final GitTokens tokens;
|
||||
private final GitTransport transport;
|
||||
private final GitGatewayProperties properties;
|
||||
private final ProjectService projects;
|
||||
private final TransientStateStore state;
|
||||
private final ObjectMapper json;
|
||||
private final GitLimits limits;
|
||||
private final SecureRandom random = new SecureRandom();
|
||||
|
||||
public record Action(
|
||||
GitAccess access,
|
||||
String hash,
|
||||
String oid,
|
||||
long size,
|
||||
String method,
|
||||
String path,
|
||||
Map<String, String> headers,
|
||||
Instant expiresAt) {}
|
||||
|
||||
public Action authenticate(String code, String projectCode, String bearer, String method) {
|
||||
if (!code.matches("[a-f0-9]{32}") || bearer == null || !bearer.matches("[A-Za-z0-9_-]{43}"))
|
||||
throw GitTokens.unauthorized();
|
||||
String stored = state.get("git:lfs-action:" + code);
|
||||
try {
|
||||
Action action = stored == null ? null : json.readValue(stored, Action.class);
|
||||
if (action == null
|
||||
|| !MessageDigest.isEqual(
|
||||
action.hash().getBytes(java.nio.charset.StandardCharsets.US_ASCII),
|
||||
SecurityHashes.sha256(bearer)
|
||||
.getBytes(java.nio.charset.StandardCharsets.US_ASCII))
|
||||
|| !action.expiresAt().isAfter(Instant.now())
|
||||
|| !action.access().projectCode().equals(projectCode)
|
||||
|| !action.method().equals(method)) throw GitTokens.unauthorized();
|
||||
GitAccess access = tokens.recheck(action.access());
|
||||
if (!"GET".equals(method)) GitTokens.requireWrite(access);
|
||||
return action;
|
||||
} catch (com.fasterxml.jackson.core.JsonProcessingException error) {
|
||||
throw GitTokens.unauthorized();
|
||||
}
|
||||
}
|
||||
|
||||
public void serve(
|
||||
GitAccess access,
|
||||
String endpoint,
|
||||
HttpServletRequest request,
|
||||
HttpServletResponse response)
|
||||
throws IOException {
|
||||
try (AutoCloseable permit = limits.acquire(false, true);
|
||||
GitTransport.Budget budget = transport.budget()) {
|
||||
budget.register(request.getInputStream());
|
||||
budget.onTimeout(
|
||||
() -> {
|
||||
if (response.isCommitted()) response.getOutputStream().close();
|
||||
});
|
||||
if ("locks/verify".equals(endpoint) && "POST".equals(request.getMethod())) {
|
||||
GitIo.metadata(
|
||||
GitGateway.decoded(request, properties.getMaxMetadataBytes(), budget),
|
||||
properties.getMaxMetadataBytes());
|
||||
ObjectNode emptyLocks = json.createObjectNode();
|
||||
emptyLocks.set("ours", json.createArrayNode());
|
||||
emptyLocks.set("theirs", json.createArrayNode());
|
||||
write(response, emptyLocks);
|
||||
return;
|
||||
}
|
||||
if (!"objects/batch".equals(endpoint) || !"POST".equals(request.getMethod()))
|
||||
throw new BusinessException(501, "LFS_LOCKS_UNSUPPORTED", "暂不支持 LFS 文件锁或该协议接口");
|
||||
GitGateway.requireContentType(request, MEDIA);
|
||||
JsonNode input = clientJson(request, budget);
|
||||
if (input == null || !input.isObject()) throw invalid();
|
||||
String operation = input.path("operation").asText();
|
||||
if (!("upload".equals(operation) || "download".equals(operation))) throw invalid();
|
||||
if ("upload".equals(operation)) GitTokens.requireWrite(access);
|
||||
JsonNode objects = input.path("objects");
|
||||
if (!objects.isArray()
|
||||
|| objects.isEmpty()
|
||||
|| objects.size() > properties.getMaxBatchObjects()) throw invalid();
|
||||
Map<String, Long> requested = new HashMap<>();
|
||||
for (JsonNode object : objects) {
|
||||
String oid = object.path("oid").asText();
|
||||
long size = object.path("size").asLong(-1);
|
||||
if (!oid.matches("[a-f0-9]{64}")
|
||||
|| !object.path("size").isIntegralNumber()
|
||||
|| !object.path("size").canConvertToLong()
|
||||
|| size < 0) throw invalid();
|
||||
if (size > properties.getMaxLfsObjectBytes())
|
||||
throw new BusinessException(413, "LFS_TOO_LARGE", "LFS 对象超过大小限制");
|
||||
if (requested.put(oid, size) != null) throw invalid();
|
||||
}
|
||||
// 明确协商 Basic Transfer;不向上游传入客户端自定义 transfer、ref 或 action。
|
||||
ObjectNode sanitized = json.createObjectNode().put("operation", operation);
|
||||
sanitized.set("objects", objects);
|
||||
sanitized.putArray("transfers").add("basic");
|
||||
Project project = projects.readyOwner(access.userId(), access.projectCode());
|
||||
byte[] body = json.writeValueAsBytes(sanitized);
|
||||
try (GitTransport.Response upstream =
|
||||
transport.exchange(
|
||||
transport.repositoryPath(project) + "/info/lfs/objects/batch",
|
||||
"POST",
|
||||
new ByteArrayInputStream(body),
|
||||
body.length,
|
||||
Map.of("Accept", MEDIA, "Content-Type", MEDIA),
|
||||
budget)) {
|
||||
GitGateway.requireSuccess(upstream, MEDIA);
|
||||
JsonNode result =
|
||||
json.readTree(
|
||||
GitIo.metadata(upstream.input(), properties.getMaxMetadataBytes()));
|
||||
if (result == null
|
||||
|| !result.path("objects").isArray()
|
||||
|| result.path("objects").size() != requested.size()
|
||||
|| !"basic".equals(result.path("transfer").asText("basic")))
|
||||
throw GitIo.upstreamInvalid();
|
||||
ObjectNode output = json.createObjectNode().put("transfer", "basic");
|
||||
ArrayNode results = output.putArray("objects");
|
||||
Set<String> seen = new HashSet<>();
|
||||
for (JsonNode object : result.path("objects")) {
|
||||
String oid = object.path("oid").asText();
|
||||
Long size = requested.get(oid);
|
||||
if (size == null
|
||||
|| !seen.add(oid)
|
||||
|| !object.path("size").isIntegralNumber()
|
||||
|| !object.path("size").canConvertToLong()
|
||||
|| object.path("size").asLong(-1) != size)
|
||||
throw GitIo.upstreamInvalid();
|
||||
ObjectNode target =
|
||||
results.addObject()
|
||||
.put("oid", oid)
|
||||
.put("size", size)
|
||||
.put("authenticated", true);
|
||||
if (object.has("error")) {
|
||||
// 上游错误正文可能含内部地址,仅返回固定文本与错误码。
|
||||
target.putObject("error")
|
||||
.put("code", object.path("error").path("code").asInt(502))
|
||||
.put("message", "LFS 对象不可用");
|
||||
continue;
|
||||
}
|
||||
JsonNode actions = object.path("actions");
|
||||
if (actions.isObject()) {
|
||||
ObjectNode rewritten = target.putObject("actions");
|
||||
Iterator<String> names = actions.fieldNames();
|
||||
while (names.hasNext()) {
|
||||
String name = names.next();
|
||||
if (!(name.equals(operation)
|
||||
|| name.equals("verify") && operation.equals("upload")))
|
||||
throw GitIo.upstreamInvalid();
|
||||
rewritten.set(
|
||||
name,
|
||||
issue(access, project, oid, size, name, actions.get(name)));
|
||||
}
|
||||
}
|
||||
}
|
||||
write(response, output);
|
||||
log.info(
|
||||
"LFS batch 完成 projectCode={} userCode={} tokenCode={} operation={}"
|
||||
+ " objects={}",
|
||||
access.projectCode(),
|
||||
access.userCode(),
|
||||
access.tokenCode(),
|
||||
operation,
|
||||
requested.size());
|
||||
}
|
||||
} catch (BusinessException | IOException error) {
|
||||
throw error;
|
||||
} catch (Exception error) {
|
||||
throw GitIo.upstreamInvalid();
|
||||
}
|
||||
}
|
||||
|
||||
private ObjectNode issue(
|
||||
GitAccess access, Project project, String oid, long size, String name, JsonNode source)
|
||||
throws IOException {
|
||||
String expectedPath =
|
||||
transport.base().getRawPath()
|
||||
+ transport.repositoryPath(project)
|
||||
+ "/info/lfs/"
|
||||
+ (name.equals("verify")
|
||||
? "verify"
|
||||
: "objects/" + oid + (name.equals("upload") ? "/" + size : ""));
|
||||
URI uri;
|
||||
try {
|
||||
uri = URI.create(source.path("href").asText());
|
||||
} catch (IllegalArgumentException error) {
|
||||
throw GitIo.upstreamInvalid();
|
||||
}
|
||||
URI base = transport.base();
|
||||
if (!Objects.equals(base.getScheme(), uri.getScheme())
|
||||
|| !Objects.equals(base.getHost(), uri.getHost())
|
||||
|| base.getPort() != uri.getPort()
|
||||
|| uri.getRawUserInfo() != null
|
||||
|| uri.getRawQuery() != null
|
||||
|| uri.getRawFragment() != null
|
||||
|| !expectedPath.equals(uri.getRawPath())) throw GitIo.upstreamInvalid();
|
||||
Map<String, String> headers = new HashMap<>();
|
||||
JsonNode sourceHeaders = source.path("header");
|
||||
if (sourceHeaders.isObject()) {
|
||||
Iterator<Map.Entry<String, JsonNode>> fields = sourceHeaders.fields();
|
||||
while (fields.hasNext()) {
|
||||
Map.Entry<String, JsonNode> field = fields.next();
|
||||
// Gitea 原生上传 action 会给出此 hop-by-hop 提示。网关使用自己的
|
||||
// BodyPublisher 决定传输方式,不能把上游的 Transfer-Encoding 直接设置到 JDK 客户端。
|
||||
if (name.equals("upload")
|
||||
&& field.getKey().equalsIgnoreCase("Transfer-Encoding")
|
||||
&& field.getValue().isTextual()
|
||||
&& "chunked".equalsIgnoreCase(field.getValue().asText())) continue;
|
||||
if (!Set.of("authorization", "accept", "content-type")
|
||||
.contains(field.getKey().toLowerCase(Locale.ROOT))
|
||||
|| !field.getValue().isTextual()
|
||||
|| field.getValue().asText().length() > 8192
|
||||
|| field.getValue().asText().contains("\n")
|
||||
|| field.getValue().asText().contains("\r")) throw GitIo.upstreamInvalid();
|
||||
headers.put(field.getKey(), field.getValue().asText());
|
||||
}
|
||||
}
|
||||
long ttl = 900;
|
||||
if (source.has("expires_in")) ttl = Math.min(ttl, source.path("expires_in").asLong(0));
|
||||
if (source.has("expires_at")) {
|
||||
try {
|
||||
ttl =
|
||||
Math.min(
|
||||
ttl,
|
||||
Duration.between(
|
||||
Instant.now(),
|
||||
Instant.parse(source.path("expires_at").asText()))
|
||||
.getSeconds());
|
||||
} catch (java.time.format.DateTimeParseException error) {
|
||||
throw GitIo.upstreamInvalid();
|
||||
}
|
||||
}
|
||||
if (ttl < 1) throw GitIo.upstreamInvalid();
|
||||
byte[] bytes = new byte[32];
|
||||
random.nextBytes(bytes);
|
||||
String secret = Base64.getUrlEncoder().withoutPadding().encodeToString(bytes);
|
||||
String code = Codes.next();
|
||||
String method = name.equals("download") ? "GET" : name.equals("upload") ? "PUT" : "POST";
|
||||
String path =
|
||||
transport.repositoryPath(project)
|
||||
+ "/info/lfs/"
|
||||
+ (name.equals("verify")
|
||||
? "verify"
|
||||
: "objects/" + oid + (name.equals("upload") ? "/" + size : ""));
|
||||
Action action =
|
||||
new Action(
|
||||
access,
|
||||
SecurityHashes.sha256(secret),
|
||||
oid,
|
||||
size,
|
||||
method,
|
||||
path,
|
||||
headers,
|
||||
Instant.now().plusSeconds(ttl));
|
||||
if (!state.putIfAbsent(
|
||||
"git:lfs-action:" + code, json.writeValueAsString(action), Duration.ofSeconds(ttl)))
|
||||
throw GitIo.upstreamInvalid();
|
||||
ObjectNode result =
|
||||
json.createObjectNode()
|
||||
.put(
|
||||
"href",
|
||||
properties.getPublicBaseUrl()
|
||||
+ "/projects/"
|
||||
+ access.projectCode()
|
||||
+ ".git/info/lfs/actions/"
|
||||
+ code)
|
||||
.put("expires_in", ttl);
|
||||
result.putObject("header").put("Authorization", "Bearer " + secret);
|
||||
return result;
|
||||
}
|
||||
|
||||
public void action(Action action, HttpServletRequest request, HttpServletResponse response)
|
||||
throws IOException {
|
||||
long started = System.nanoTime();
|
||||
java.util.concurrent.atomic.AtomicLong transferred =
|
||||
new java.util.concurrent.atomic.AtomicLong();
|
||||
Project project =
|
||||
projects.readyOwner(action.access().userId(), action.access().projectCode());
|
||||
String expectedPath =
|
||||
transport.repositoryPath(project)
|
||||
+ "/info/lfs/"
|
||||
+ (action.method().equals("POST")
|
||||
? "verify"
|
||||
: "objects/"
|
||||
+ action.oid()
|
||||
+ (action.method().equals("PUT")
|
||||
? "/" + action.size()
|
||||
: ""));
|
||||
if (!expectedPath.equals(action.path())) throw GitTokens.unauthorized();
|
||||
try (AutoCloseable permit = limits.acquire(false, true);
|
||||
GitTransport.Budget budget = transport.budget()) {
|
||||
budget.register(request.getInputStream());
|
||||
budget.onTimeout(
|
||||
() -> {
|
||||
if (response.isCommitted()) response.getOutputStream().close();
|
||||
});
|
||||
InputStream body = null;
|
||||
long length = -1;
|
||||
String type = "application/octet-stream";
|
||||
MessageDigest digest = MessageDigest.getInstance("SHA-256");
|
||||
if (action.method().equals("PUT")) {
|
||||
if (request.getContentLengthLong() >= 0
|
||||
&& request.getContentLengthLong() != action.size()) throw invalid();
|
||||
if (request.getHeader("Content-Encoding") != null) throw invalid();
|
||||
body =
|
||||
new DigestInputStream(
|
||||
GitIo.counted(
|
||||
GitIo.limited(request.getInputStream(), action.size()),
|
||||
transferred),
|
||||
digest);
|
||||
length = action.size();
|
||||
} else if (action.method().equals("POST")) {
|
||||
JsonNode verification = clientJson(request, budget);
|
||||
if (verification == null
|
||||
|| !action.oid().equals(verification.path("oid").asText())
|
||||
|| !verification.path("size").isIntegralNumber()
|
||||
|| !verification.path("size").canConvertToLong()
|
||||
|| verification.path("size").asLong(-1) != action.size()) throw invalid();
|
||||
byte[] bytes =
|
||||
json.writeValueAsBytes(Map.of("oid", action.oid(), "size", action.size()));
|
||||
body = new ByteArrayInputStream(bytes);
|
||||
length = bytes.length;
|
||||
transferred.set(bytes.length);
|
||||
type = MEDIA;
|
||||
}
|
||||
try (GitTransport.Response upstream =
|
||||
transport.exchange(
|
||||
action.path(),
|
||||
action.method(),
|
||||
body,
|
||||
length,
|
||||
Map.of("Accept", MEDIA, "Content-Type", type),
|
||||
budget,
|
||||
action.headers())) {
|
||||
if (upstream.status() < 200 || upstream.status() >= 300)
|
||||
throw GitIo.upstreamInvalid();
|
||||
if (action.method().equals("GET")) {
|
||||
long size = upstream.headers().firstValueAsLong("Content-Length").orElse(-1);
|
||||
if (size >= 0 && size != action.size()) throw GitIo.upstreamInvalid();
|
||||
response.setContentType("application/octet-stream");
|
||||
response.setContentLengthLong(action.size());
|
||||
long copied =
|
||||
GitGateway.copy(
|
||||
new DigestInputStream(upstream.input(), digest),
|
||||
response,
|
||||
action.size(),
|
||||
budget,
|
||||
transferred);
|
||||
if (copied != action.size()
|
||||
|| !HexFormat.of().formatHex(digest.digest()).equals(action.oid()))
|
||||
throw GitIo.upstreamInvalid();
|
||||
} else {
|
||||
GitIo.metadata(upstream.input(), properties.getMaxMetadataBytes());
|
||||
if (action.method().equals("PUT")
|
||||
&& !HexFormat.of().formatHex(digest.digest()).equals(action.oid()))
|
||||
throw invalid();
|
||||
response.setStatus(200);
|
||||
response.setContentType(MEDIA);
|
||||
response.setContentLength(0);
|
||||
}
|
||||
log.info(
|
||||
"LFS action 完成 projectCode={} userCode={} tokenCode={} method={} oid={}"
|
||||
+ " bytes={} durationMs={}",
|
||||
action.access().projectCode(),
|
||||
action.access().userCode(),
|
||||
action.access().tokenCode(),
|
||||
action.method(),
|
||||
action.oid(),
|
||||
transferred.get(),
|
||||
(System.nanoTime() - started) / 1000000);
|
||||
}
|
||||
} catch (BusinessException | IOException error) {
|
||||
log.warn(
|
||||
"LFS action 失败或取消 projectCode={} tokenCode={} method={} oid={} bytes={}"
|
||||
+ " errorType={}",
|
||||
action.access().projectCode(),
|
||||
action.access().tokenCode(),
|
||||
action.method(),
|
||||
action.oid(),
|
||||
transferred.get(),
|
||||
error.getClass().getSimpleName());
|
||||
throw error;
|
||||
} catch (Exception error) {
|
||||
throw GitIo.upstreamInvalid();
|
||||
}
|
||||
}
|
||||
|
||||
private JsonNode clientJson(HttpServletRequest request, GitTransport.Budget budget)
|
||||
throws IOException {
|
||||
GitGateway.requireContentType(request, MEDIA);
|
||||
byte[] bytes =
|
||||
GitIo.metadata(
|
||||
GitGateway.decoded(request, properties.getMaxMetadataBytes(), budget),
|
||||
properties.getMaxMetadataBytes());
|
||||
try {
|
||||
return json.reader()
|
||||
.with(DeserializationFeature.FAIL_ON_TRAILING_TOKENS)
|
||||
.readTree(bytes);
|
||||
} catch (com.fasterxml.jackson.core.JsonProcessingException error) {
|
||||
throw invalid();
|
||||
}
|
||||
}
|
||||
|
||||
private BusinessException invalid() {
|
||||
return BusinessException.badRequest("LFS 请求的操作、OID、大小或 JSON 不合法");
|
||||
}
|
||||
|
||||
private void write(HttpServletResponse response, JsonNode value) throws IOException {
|
||||
response.setContentType(MEDIA);
|
||||
byte[] bytes = json.writeValueAsBytes(value);
|
||||
response.setContentLength(bytes.length);
|
||||
response.getOutputStream().write(bytes);
|
||||
}
|
||||
}
|
||||
@@ -10,8 +10,19 @@ import java.util.List;
|
||||
import java.time.Duration;
|
||||
|
||||
public interface GiteaClient {
|
||||
default void ensureGitAccess(Repository repository, String username) {
|
||||
throw new UnsupportedOperationException("未实现 Git 技术账号仓库授权");
|
||||
}
|
||||
default String authenticatedUsername() { throw new UnsupportedOperationException("未实现服务账号查询"); }
|
||||
|
||||
default void ensureGitProtection(Repository repository, String username) {
|
||||
throw new UnsupportedOperationException("未实现 Git main 保护初始化");
|
||||
}
|
||||
|
||||
default void verifyGitProtection(Repository repository, String username) {
|
||||
throw new UnsupportedOperationException("未实现 Git main 保护核验");
|
||||
}
|
||||
|
||||
record Repository(long id, String owner, String name) {}
|
||||
|
||||
record StoredFile(String path, String blobSha, byte[] bytes) {}
|
||||
|
||||
+45
@@ -49,6 +49,51 @@ public class RestGiteaClient implements GiteaClient {
|
||||
return "/repos/" + encode(repo.owner()) + "/" + encode(repo.name());
|
||||
}
|
||||
|
||||
@Override
|
||||
public void ensureGitAccess(Repository repo, String username) {
|
||||
String path = repoPath(repo) + "/collaborators/" + encode(username);
|
||||
String permission = "none";
|
||||
try { permission = parse(request("GET", path + "/permission", null).body(), false).path("permission").asText(); }
|
||||
catch (GiteaFailure error) { if (error.status() != 404) throw error; }
|
||||
if ("admin".equals(permission) || "owner".equals(permission)) throw gitPolicyFailure();
|
||||
if (!"write".equals(permission)) request("PUT", path, Map.of("permission", "write"));
|
||||
JsonNode current = parse(request("GET", path + "/permission", null).body(), false);
|
||||
if (!"write".equals(current.path("permission").asText())) throw gitPolicyFailure();
|
||||
}
|
||||
|
||||
@Override
|
||||
public void ensureGitProtection(Repository repo, String username) {
|
||||
JsonNode rules = parse(request("GET", repoPath(repo) + "/branch_protections", null).body(), false);
|
||||
if (rules.isArray() && rules.isEmpty()) {
|
||||
request("POST", repoPath(repo) + "/branch_protections", Map.of(
|
||||
"branch_name", "main", "rule_name", "main", "enable_push", true, "enable_force_push", false));
|
||||
}
|
||||
verifyGitProtection(repo, username);
|
||||
}
|
||||
|
||||
@Override
|
||||
public void verifyGitProtection(Repository repo, String username) {
|
||||
JsonNode permission = parse(request("GET", repoPath(repo) + "/collaborators/"
|
||||
+ encode(username) + "/permission", null).body(), false);
|
||||
if (!"write".equals(permission.path("permission").asText())) throw gitPolicyFailure();
|
||||
JsonNode rules = parse(request("GET", repoPath(repo) + "/branch_protections", null).body(), false);
|
||||
if (!rules.isArray() || rules.size() != 1) throw gitPolicyFailure();
|
||||
JsonNode rule = rules.get(0);
|
||||
String name = rule.path("rule_name").asText(rule.path("branch_name").asText());
|
||||
if (!"main".equals(name) || !rule.path("enable_push").asBoolean()
|
||||
|| !rule.has("enable_force_push") || rule.path("enable_force_push").asBoolean()) throw gitPolicyFailure();
|
||||
if (rule.path("enable_push_whitelist").asBoolean()) {
|
||||
boolean allowed = false;
|
||||
for (JsonNode member : rule.path("push_whitelist_usernames"))
|
||||
if (username.equals(member.asText())) allowed = true;
|
||||
if (!allowed) throw gitPolicyFailure();
|
||||
}
|
||||
}
|
||||
|
||||
private BusinessException gitPolicyFailure() {
|
||||
return new BusinessException(503, "GIT_POLICY_UNAVAILABLE", "main 保护规则或 Git 技术账号权限不兼容,请联系管理员");
|
||||
}
|
||||
|
||||
private String filePath(String path) {
|
||||
return Arrays.stream(path.split("/"))
|
||||
.map(RestGiteaClient::encode)
|
||||
|
||||
+6
@@ -78,6 +78,11 @@ public class ProjectLocks implements AutoCloseable {
|
||||
}
|
||||
|
||||
public <T> T write(String code, Supplier<T> action) {
|
||||
return write(code, () -> {}, action);
|
||||
}
|
||||
|
||||
/** prepare 在写保护检查后、任何远端写入前执行,用于原子持久化协议写入凭据。 */
|
||||
public <T> T write(String code, Runnable prepare, Supplier<T> action) {
|
||||
return serial(
|
||||
code,
|
||||
() -> {
|
||||
@@ -96,6 +101,7 @@ public class ProjectLocks implements AutoCloseable {
|
||||
markUncertain(code);
|
||||
throw blocked();
|
||||
}
|
||||
prepare.run();
|
||||
state(code, "WRITING");
|
||||
writing.get().add(code);
|
||||
try {
|
||||
|
||||
+294
-93
@@ -6,21 +6,26 @@ import com.linkerbot.vibemake.bean.entity.Project;
|
||||
import com.linkerbot.vibemake.bean.entity.RepositoryWriteOperation;
|
||||
import com.linkerbot.vibemake.common.*;
|
||||
import com.linkerbot.vibemake.service.ProjectService;
|
||||
import com.linkerbot.vibemake.service.git.GitAccess;
|
||||
import com.linkerbot.vibemake.service.git.GitIo;
|
||||
import com.linkerbot.vibemake.service.gitea.*;
|
||||
import com.linkerbot.vibemake.service.mapper.ProjectMapper;
|
||||
import com.linkerbot.vibemake.service.mapper.RepositoryWriteOperationMapper;
|
||||
import com.linkerbot.vibemake.service.support.ProjectActors;
|
||||
import com.linkerbot.vibemake.service.support.ProjectLocks;
|
||||
import com.linkerbot.vibemake.service.support.ProjectSupport;
|
||||
|
||||
import lombok.RequiredArgsConstructor;
|
||||
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
import org.springframework.stereotype.Service;
|
||||
import org.springframework.transaction.support.TransactionTemplate;
|
||||
import java.time.LocalDateTime;
|
||||
|
||||
import java.util.List;
|
||||
import java.util.Objects;
|
||||
import java.util.function.Function;
|
||||
import java.util.function.Supplier;
|
||||
|
||||
/** 数据库凭据先于远端写入落盘。未知结果只读取核对,绝不自动重发写请求。 */
|
||||
@Service
|
||||
@@ -37,61 +42,87 @@ public class RepositoryWrites {
|
||||
|
||||
public record Request(String code, String digest) {}
|
||||
|
||||
public CommitView execute(long userId, String projectCode, String key, String digest,
|
||||
Function<Request, CommitView> action) {
|
||||
String operationCode = key == null ? Codes.next() : key.replace("-", "").toLowerCase(java.util.Locale.ROOT);
|
||||
public CommitView execute(
|
||||
long userId,
|
||||
String projectCode,
|
||||
String key,
|
||||
String digest,
|
||||
Function<Request, CommitView> action) {
|
||||
String operationCode =
|
||||
key == null
|
||||
? Codes.next()
|
||||
: key.replace("-", "").toLowerCase(java.util.Locale.ROOT);
|
||||
if (!operationCode.matches("[a-f0-9]{32}"))
|
||||
throw BusinessException.badRequest("操作编码必须为 UUID 或 32 位十六进制编码");
|
||||
return locks.serial(projectCode, () -> {
|
||||
projects.readyOwner(userId, projectCode); // 重放也必须重新鉴权。
|
||||
RepositoryWriteOperation previous = operations.selectById(operationCode);
|
||||
if (previous != null) {
|
||||
if (!projectCode.equals(previous.getProjectCode()) || !digest.equals(previous.getRequestDigest()))
|
||||
throw new BusinessException(409, "IDEMPOTENCY_KEY_REUSED", "操作编码已用于不同请求,请勿复用");
|
||||
return replay(previous);
|
||||
}
|
||||
RepositoryWriteOperation pending = active(projectCode);
|
||||
if (pending != null)
|
||||
throw pending(pending);
|
||||
try {
|
||||
CommitView result = action.apply(new Request(operationCode, digest));
|
||||
RepositoryWriteOperation operation = operations.selectById(operationCode);
|
||||
if (operation == null) throw new IllegalStateException("仓库写入缺少持久化凭据");
|
||||
complete(operation, result.commitSha(), "SUCCEEDED");
|
||||
return result;
|
||||
} catch (RuntimeException error) {
|
||||
RepositoryWriteOperation operation = operations.selectById(operationCode);
|
||||
if (operation == null) throw error; // 参数校验、读取失败发生在任何远端写入之前。
|
||||
if (!projectCode.equals(operation.getProjectCode()) || !digest.equals(operation.getRequestDigest()))
|
||||
throw new BusinessException(409, "IDEMPOTENCY_KEY_REUSED", "操作编码已用于不同请求,请勿复用");
|
||||
boolean definite = error instanceof GiteaFailure remote && !remote.uncertain();
|
||||
if (definite) {
|
||||
BusinessException business = (BusinessException) error;
|
||||
operation.setStatus("FAILED");
|
||||
operation.setFailureStatus(business.status());
|
||||
operation.setLastErrorCode(business.code());
|
||||
save(operation);
|
||||
throw error;
|
||||
}
|
||||
locks.markUncertain(projectCode);
|
||||
operation.setLastErrorCode(error instanceof BusinessException business
|
||||
? business.code() : "WRITE_RESULT_UNKNOWN");
|
||||
save(operation);
|
||||
log.warn("仓库写入转后台核对 projectCode={} operationCode={} errorCode={} errorType={}",
|
||||
projectCode, operationCode, operation.getLastErrorCode(), error.getClass().getSimpleName());
|
||||
if (!(error instanceof BusinessException)) SafeExceptionLog.write(log, "仓库提交结果未知", error);
|
||||
throw error;
|
||||
}
|
||||
});
|
||||
return locks.serial(
|
||||
projectCode,
|
||||
() -> {
|
||||
projects.readyOwner(userId, projectCode); // 重放也必须重新鉴权。
|
||||
RepositoryWriteOperation previous = operations.selectById(operationCode);
|
||||
if (previous != null) {
|
||||
if (!projectCode.equals(previous.getProjectCode())
|
||||
|| !digest.equals(previous.getRequestDigest()))
|
||||
throw new BusinessException(
|
||||
409, "IDEMPOTENCY_KEY_REUSED", "操作编码已用于不同请求,请勿复用");
|
||||
return replay(previous);
|
||||
}
|
||||
RepositoryWriteOperation pending = active(projectCode);
|
||||
if (pending != null) throw pending(pending);
|
||||
try {
|
||||
CommitView result = action.apply(new Request(operationCode, digest));
|
||||
RepositoryWriteOperation operation = operations.selectById(operationCode);
|
||||
if (operation == null) throw new IllegalStateException("仓库写入缺少持久化凭据");
|
||||
complete(operation, result.commitSha(), "SUCCEEDED");
|
||||
return result;
|
||||
} catch (RuntimeException error) {
|
||||
RepositoryWriteOperation operation = operations.selectById(operationCode);
|
||||
if (operation == null) throw error; // 参数校验、读取失败发生在任何远端写入之前。
|
||||
if (!projectCode.equals(operation.getProjectCode())
|
||||
|| !digest.equals(operation.getRequestDigest()))
|
||||
throw new BusinessException(
|
||||
409, "IDEMPOTENCY_KEY_REUSED", "操作编码已用于不同请求,请勿复用");
|
||||
boolean definite =
|
||||
error instanceof GiteaFailure remote && !remote.uncertain();
|
||||
if (definite) {
|
||||
BusinessException business = (BusinessException) error;
|
||||
operation.setStatus("FAILED");
|
||||
operation.setFailureStatus(business.status());
|
||||
operation.setLastErrorCode(business.code());
|
||||
save(operation);
|
||||
throw error;
|
||||
}
|
||||
locks.markUncertain(projectCode);
|
||||
operation.setLastErrorCode(
|
||||
error instanceof BusinessException business
|
||||
? business.code()
|
||||
: "WRITE_RESULT_UNKNOWN");
|
||||
save(operation);
|
||||
log.warn(
|
||||
"仓库写入转后台核对 projectCode={} operationCode={} errorCode={}"
|
||||
+ " errorType={}",
|
||||
projectCode,
|
||||
operationCode,
|
||||
operation.getLastErrorCode(),
|
||||
error.getClass().getSimpleName());
|
||||
if (!(error instanceof BusinessException))
|
||||
SafeExceptionLog.write(log, "仓库提交结果未知", error);
|
||||
throw error;
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
/** 必须在 ProjectLocks.write 内调用;此处是唯一一次发起远端写请求的位置。 */
|
||||
public CommitView commit(Request request, Project project, String baseSha,
|
||||
List<GiteaClient.Change> changes, String message,
|
||||
ProjectActors.Author author) {
|
||||
public CommitView commit(
|
||||
Request request,
|
||||
Project project,
|
||||
String baseSha,
|
||||
List<GiteaClient.Change> changes,
|
||||
String message,
|
||||
ProjectActors.Author author) {
|
||||
GiteaClient.Repository repository = projects.repository(project);
|
||||
RepositoryWriteOperation operation = new RepositoryWriteOperation();
|
||||
operation.setOperationCode(request.code());
|
||||
operation.setOperationKind("WEB_COMMIT");
|
||||
operation.setProjectCode(project.getProjectCode());
|
||||
operation.setRepositoryId(repository.id());
|
||||
operation.setRepositoryOwner(repository.owner());
|
||||
@@ -109,11 +140,20 @@ public class RepositoryWrites {
|
||||
} catch (org.springframework.dao.DuplicateKeyException collision) {
|
||||
throw new BusinessException(409, "IDEMPOTENCY_KEY_REUSED", "操作编码已用于其他请求,请勿复用");
|
||||
}
|
||||
log.info("仓库写入开始 projectCode={} operationCode={} baseCommitSha={} fileCount={}",
|
||||
project.getProjectCode(), request.code(), baseSha, changes.size());
|
||||
log.info(
|
||||
"仓库写入开始 projectCode={} operationCode={} baseCommitSha={} fileCount={}",
|
||||
project.getProjectCode(),
|
||||
request.code(),
|
||||
baseSha,
|
||||
changes.size());
|
||||
try {
|
||||
CommitView result = gitea.commit(repository, changes,
|
||||
message + WriteReceipt.suffix(request.code(), request.digest()), author.nickname(), author.code());
|
||||
CommitView result =
|
||||
gitea.commit(
|
||||
repository,
|
||||
changes,
|
||||
message + WriteReceipt.suffix(request.code(), request.digest()),
|
||||
author.nickname(),
|
||||
author.code());
|
||||
return new CommitView(result.commitSha(), message);
|
||||
} catch (GiteaFailure error) {
|
||||
if (!error.uncertain()) throw error;
|
||||
@@ -126,23 +166,141 @@ public class RepositoryWrites {
|
||||
|
||||
private CommitView replay(RepositoryWriteOperation operation) {
|
||||
return switch (operation.getStatus()) {
|
||||
case "SUCCEEDED" -> new CommitView(operation.getResultCommitSha(), operation.getCommitMessage());
|
||||
case "FAILED" -> throw new BusinessException(operation.getFailureStatus(),
|
||||
operation.getLastErrorCode(), "该操作已被仓库拒绝,请刷新后使用新的操作编码提交");
|
||||
case "SUCCEEDED" ->
|
||||
new CommitView(operation.getResultCommitSha(), operation.getCommitMessage());
|
||||
case "FAILED" ->
|
||||
throw new BusinessException(
|
||||
operation.getFailureStatus(),
|
||||
operation.getLastErrorCode(),
|
||||
"该操作已被仓库拒绝,请刷新后使用新的操作编码提交");
|
||||
default -> throw pending(operation);
|
||||
};
|
||||
}
|
||||
|
||||
private BusinessException pending(RepositoryWriteOperation operation) {
|
||||
return new BusinessException(409, "REMOTE_RESULT_UNKNOWN", "仓库提交结果正在后台核对,请稍后刷新工作区,暂勿重复提交",
|
||||
return new BusinessException(
|
||||
409,
|
||||
"REMOTE_RESULT_UNKNOWN",
|
||||
"仓库提交结果正在后台核对,请稍后刷新工作区,暂勿重复提交",
|
||||
java.util.Map.of("projectCode", operation.getProjectCode()));
|
||||
}
|
||||
|
||||
public RepositoryWriteOperation active(String code) {
|
||||
return operations.selectOne(new LambdaQueryWrapper<RepositoryWriteOperation>()
|
||||
.eq(RepositoryWriteOperation::getProjectCode, code)
|
||||
.in(RepositoryWriteOperation::getStatus, ACTIVE)
|
||||
.orderByDesc(RepositoryWriteOperation::getCreatedAt).last("LIMIT 1"));
|
||||
return operations.selectOne(
|
||||
new LambdaQueryWrapper<RepositoryWriteOperation>()
|
||||
.eq(RepositoryWriteOperation::getProjectCode, code)
|
||||
.in(RepositoryWriteOperation::getStatus, ACTIVE)
|
||||
.orderByDesc(RepositoryWriteOperation::getCreatedAt)
|
||||
.last("LIMIT 1"));
|
||||
}
|
||||
|
||||
/** 实时校验、写入以及命名锁释放全部纳入同一结果核对边界。 */
|
||||
public byte[] push(
|
||||
GitAccess access,
|
||||
GitIo.PushCommand command,
|
||||
Supplier<Project> validate,
|
||||
Supplier<byte[]> transfer) {
|
||||
java.util.concurrent.atomic.AtomicReference<RepositoryWriteOperation> registered =
|
||||
new java.util.concurrent.atomic.AtomicReference<>();
|
||||
try {
|
||||
return locks.serial(
|
||||
access.projectCode(),
|
||||
() -> pushLocked(validate.get(), access, command, transfer, registered::set));
|
||||
} catch (RuntimeException error) {
|
||||
RepositoryWriteOperation operation = registered.get();
|
||||
if (operation != null && operations.selectById(operation.getOperationCode()) != null) {
|
||||
locks.markUncertain(access.projectCode());
|
||||
operation.setStatus("PROCESSING");
|
||||
operation.setLastErrorCode(
|
||||
error instanceof BusinessException business
|
||||
? business.code()
|
||||
: "GIT_RESULT_UNKNOWN");
|
||||
save(operation);
|
||||
log.warn(
|
||||
"Git 推送待核对 projectCode={} operationCode={} oldSha={} newSha={}"
|
||||
+ " errorCode={}",
|
||||
access.projectCode(),
|
||||
operation.getOperationCode(),
|
||||
command.oldSha(),
|
||||
command.newSha(),
|
||||
operation.getLastErrorCode());
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
/** 调用者已在共享项目锁中完成实时鉴权、old SHA 和保护规则校验。 */
|
||||
private byte[] pushLocked(
|
||||
Project project,
|
||||
GitAccess access,
|
||||
GitIo.PushCommand command,
|
||||
Supplier<byte[]> transfer,
|
||||
java.util.function.Consumer<RepositoryWriteOperation> registered) {
|
||||
RepositoryWriteOperation pending = active(project.getProjectCode());
|
||||
if (pending != null) throw pending(pending);
|
||||
GiteaClient.Repository repository = projects.repository(project);
|
||||
RepositoryWriteOperation operation = new RepositoryWriteOperation();
|
||||
operation.setOperationCode(Codes.next());
|
||||
operation.setOperationKind("GIT_PUSH");
|
||||
operation.setProjectCode(project.getProjectCode());
|
||||
operation.setRepositoryId(repository.id());
|
||||
operation.setRepositoryOwner(repository.owner());
|
||||
operation.setRepositoryName(repository.name());
|
||||
operation.setBaseCommitSha(command.oldSha());
|
||||
operation.setTargetCommitSha(command.newSha());
|
||||
operation.setActorUserCode(access.userCode());
|
||||
operation.setGitTokenCode(access.tokenCode());
|
||||
operation.setRequestDigest(
|
||||
SecurityHashes.sha256(
|
||||
repository.id()
|
||||
+ ":refs/heads/main:"
|
||||
+ command.oldSha()
|
||||
+ ":"
|
||||
+ command.newSha()));
|
||||
operation.setCommitMessage("Git push main");
|
||||
operation.setStatus("PROCESSING");
|
||||
operation.setCheckCount(0);
|
||||
operation.setCreatedAt(ProjectSupport.now());
|
||||
operation.setUpdatedAt(operation.getCreatedAt());
|
||||
operation.setNextCheckAt(operation.getCreatedAt().plusSeconds(15));
|
||||
registered.accept(operation);
|
||||
return locks.write(
|
||||
project.getProjectCode(),
|
||||
() ->
|
||||
transactions.executeWithoutResult(
|
||||
transaction -> {
|
||||
operations.insert(operation);
|
||||
project.setRepoWriteStatus("WRITING");
|
||||
if (projectMapper.updateById(project) != 1)
|
||||
throw new IllegalStateException("项目写保护保存失败");
|
||||
}),
|
||||
() -> {
|
||||
log.info(
|
||||
"Git 推送开始 projectCode={} userCode={} tokenCode={} operationCode={}"
|
||||
+ " oldSha={} newSha={}",
|
||||
project.getProjectCode(),
|
||||
access.userCode(),
|
||||
access.tokenCode(),
|
||||
operation.getOperationCode(),
|
||||
command.oldSha(),
|
||||
command.newSha());
|
||||
byte[] result = transfer.get();
|
||||
boolean success;
|
||||
try {
|
||||
success = GitIo.pushSucceeded(result);
|
||||
} catch (java.io.IOException error) {
|
||||
throw GitIo.upstreamInvalid();
|
||||
}
|
||||
if (success && !command.newSha().equals(gitea.head(repository)))
|
||||
throw new BusinessException(503, "REMOTE_RESULT_UNKNOWN", "推送结果尚未核对");
|
||||
operation.setLastErrorCode(success ? null : "GIT_PUSH_REJECTED");
|
||||
operation.setFailureStatus(success ? null : 409);
|
||||
complete(
|
||||
operation,
|
||||
success ? command.newSha() : null,
|
||||
success ? "SUCCEEDED" : "FAILED");
|
||||
return result;
|
||||
});
|
||||
}
|
||||
|
||||
private boolean slowCheckDue(RepositoryWriteOperation operation) {
|
||||
@@ -151,20 +309,29 @@ public class RepositoryWrites {
|
||||
|
||||
/** 重启后直接扫描数据库,无内存任务队列;多实例由相同项目命名锁串行核对。 */
|
||||
public void reconcileDue() {
|
||||
List<RepositoryWriteOperation> due = operations.selectList(new LambdaQueryWrapper<RepositoryWriteOperation>()
|
||||
.in(RepositoryWriteOperation::getStatus, ACTIVE)
|
||||
.le(RepositoryWriteOperation::getNextCheckAt, ProjectSupport.now())
|
||||
.orderByAsc(RepositoryWriteOperation::getNextCheckAt).last("LIMIT 20"));
|
||||
List<RepositoryWriteOperation> due =
|
||||
operations.selectList(
|
||||
new LambdaQueryWrapper<RepositoryWriteOperation>()
|
||||
.in(RepositoryWriteOperation::getStatus, ACTIVE)
|
||||
.le(RepositoryWriteOperation::getNextCheckAt, ProjectSupport.now())
|
||||
.orderByAsc(RepositoryWriteOperation::getNextCheckAt)
|
||||
.last("LIMIT 20"));
|
||||
for (RepositoryWriteOperation candidate : due) {
|
||||
try {
|
||||
locks.serial(candidate.getProjectCode(), () -> {
|
||||
reconcile(candidate.getOperationCode());
|
||||
return null;
|
||||
});
|
||||
locks.serial(
|
||||
candidate.getProjectCode(),
|
||||
() -> {
|
||||
reconcile(candidate.getOperationCode());
|
||||
return null;
|
||||
});
|
||||
} catch (RuntimeException error) {
|
||||
log.warn("仓库后台核对未完成 projectCode={} operationCode={} errorCode={} errorType={}",
|
||||
candidate.getProjectCode(), candidate.getOperationCode(),
|
||||
error instanceof BusinessException business ? business.code() : "CHECK_FAILED",
|
||||
log.warn(
|
||||
"仓库后台核对未完成 projectCode={} operationCode={} errorCode={} errorType={}",
|
||||
candidate.getProjectCode(),
|
||||
candidate.getOperationCode(),
|
||||
error instanceof BusinessException business
|
||||
? business.code()
|
||||
: "CHECK_FAILED",
|
||||
error.getClass().getSimpleName());
|
||||
}
|
||||
}
|
||||
@@ -172,10 +339,13 @@ public class RepositoryWrites {
|
||||
|
||||
private void reconcile(String operationCode) {
|
||||
RepositoryWriteOperation operation = operations.selectById(operationCode);
|
||||
if (operation == null || !ACTIVE.contains(operation.getStatus())
|
||||
if (operation == null
|
||||
|| !ACTIVE.contains(operation.getStatus())
|
||||
|| operation.getNextCheckAt().isAfter(ProjectSupport.now())) return;
|
||||
Project project = projectMapper.selectOne(new LambdaQueryWrapper<Project>()
|
||||
.eq(Project::getProjectCode, operation.getProjectCode()));
|
||||
Project project =
|
||||
projectMapper.selectOne(
|
||||
new LambdaQueryWrapper<Project>()
|
||||
.eq(Project::getProjectCode, operation.getProjectCode()));
|
||||
String receipt = project == null ? null : receipt(operation, project);
|
||||
operation.setCheckCount(operation.getCheckCount() + 1);
|
||||
if (receipt != null) {
|
||||
@@ -186,9 +356,13 @@ public class RepositoryWrites {
|
||||
operation.setStatus(overdue ? "NEEDS_REVIEW" : "PROCESSING");
|
||||
operation.setNextCheckAt(ProjectSupport.now().plusSeconds(overdue ? 60 : 15));
|
||||
save(operation);
|
||||
log.info("仓库写入继续核对 projectCode={} operationCode={} status={} checkCount={} errorCode={}",
|
||||
operation.getProjectCode(), operationCode, operation.getStatus(),
|
||||
operation.getCheckCount(), operation.getLastErrorCode());
|
||||
log.info(
|
||||
"仓库写入继续核对 projectCode={} operationCode={} status={} checkCount={} errorCode={}",
|
||||
operation.getProjectCode(),
|
||||
operationCode,
|
||||
operation.getStatus(),
|
||||
operation.getCheckCount(),
|
||||
operation.getLastErrorCode());
|
||||
}
|
||||
|
||||
private String receipt(RepositoryWriteOperation operation, Project project) {
|
||||
@@ -202,34 +376,61 @@ public class RepositoryWrites {
|
||||
if (remote.id() != operation.getRepositoryId()
|
||||
|| !remote.owner().equals(operation.getRepositoryOwner())
|
||||
|| !remote.name().equals(operation.getRepositoryName())
|
||||
|| !remote.privateRepo() || remote.mirror() || remote.empty()
|
||||
|| !remote.privateRepo()
|
||||
|| remote.mirror()
|
||||
|| remote.empty()
|
||||
|| !"main".equals(remote.defaultBranch()))
|
||||
throw new GiteaFailure(409, "WRITE_REPOSITORY_MISMATCH", "远端仓库身份发生变化", false);
|
||||
String sha = gitea.findWriteReceipt(remote.repository(), operation.getBaseCommitSha(),
|
||||
operation.getOperationCode(), operation.getRequestDigest());
|
||||
String sha;
|
||||
if ("GIT_PUSH".equals(operation.getOperationKind())) {
|
||||
String head = gitea.head(remote.repository());
|
||||
sha = Objects.equals(head, operation.getTargetCommitSha()) ? head : null;
|
||||
// main 仍为旧 SHA 不能证明接收任务已经结束,继续保持写保护。
|
||||
} else {
|
||||
sha =
|
||||
gitea.findWriteReceipt(
|
||||
remote.repository(),
|
||||
operation.getBaseCommitSha(),
|
||||
operation.getOperationCode(),
|
||||
operation.getRequestDigest());
|
||||
}
|
||||
if (sha == null) operation.setLastErrorCode("WRITE_RECEIPT_NOT_FOUND");
|
||||
return sha;
|
||||
} catch (RuntimeException error) {
|
||||
operation.setLastErrorCode(error instanceof BusinessException business
|
||||
? business.code() : "WRITE_RECEIPT_READ_FAILED");
|
||||
log.warn("仓库写入凭据读取未确认 projectCode={} operationCode={} errorCode={} errorType={}",
|
||||
operation.getProjectCode(), operation.getOperationCode(), operation.getLastErrorCode(),
|
||||
operation.setLastErrorCode(
|
||||
error instanceof BusinessException business
|
||||
? business.code()
|
||||
: "WRITE_RECEIPT_READ_FAILED");
|
||||
log.warn(
|
||||
"仓库写入凭据读取未确认 projectCode={} operationCode={} errorCode={} errorType={}",
|
||||
operation.getProjectCode(),
|
||||
operation.getOperationCode(),
|
||||
operation.getLastErrorCode(),
|
||||
error.getClass().getSimpleName());
|
||||
if (!(error instanceof BusinessException)) SafeExceptionLog.write(log, "仓库凭据核对异常", error);
|
||||
if (!(error instanceof BusinessException))
|
||||
SafeExceptionLog.write(log, "仓库凭据核对异常", error);
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
private void complete(RepositoryWriteOperation operation, String sha, String status) {
|
||||
// 项目恢复与操作终态在同一事务提交;任务获得锁后重读终态,旧任务不会清除新操作的保护。
|
||||
transactions.executeWithoutResult(transaction -> {
|
||||
operation.setStatus(status);
|
||||
operation.setResultCommitSha(sha);
|
||||
save(operation);
|
||||
locks.confirmKnownResult(operation.getProjectCode());
|
||||
});
|
||||
log.info("仓库写入结果已落盘 projectCode={} operationCode={} status={} commitSha={}",
|
||||
operation.getProjectCode(), operation.getOperationCode(), status, sha);
|
||||
transactions.executeWithoutResult(
|
||||
transaction -> {
|
||||
operation.setStatus(status);
|
||||
operation.setResultCommitSha(sha);
|
||||
save(operation);
|
||||
// 持锁连接释放异常可能使上一条终态重新进入核对;不能解除另一条操作的保护。
|
||||
if (active(operation.getProjectCode()) == null)
|
||||
locks.confirmKnownResult(operation.getProjectCode());
|
||||
else locks.markUncertain(operation.getProjectCode());
|
||||
});
|
||||
log.info(
|
||||
"仓库写入结果已落盘 projectCode={} operationCode={} status={} commitSha={}",
|
||||
operation.getProjectCode(),
|
||||
operation.getOperationCode(),
|
||||
status,
|
||||
sha);
|
||||
}
|
||||
|
||||
private void save(RepositoryWriteOperation operation) {
|
||||
|
||||
@@ -0,0 +1,99 @@
|
||||
package com.linkerbot.vibemake.service.git;
|
||||
|
||||
import static org.assertj.core.api.Assertions.*;
|
||||
|
||||
import com.linkerbot.vibemake.common.BusinessException;
|
||||
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
import java.io.*;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.util.List;
|
||||
|
||||
class GitIoTest {
|
||||
private byte[] pkt(String... lines) throws Exception {
|
||||
ByteArrayOutputStream out = new ByteArrayOutputStream();
|
||||
for (String line : lines)
|
||||
if (line == null) out.write("0000".getBytes());
|
||||
else GitIo.packet(out, line.getBytes(StandardCharsets.UTF_8));
|
||||
return out.toByteArray();
|
||||
}
|
||||
|
||||
@Test
|
||||
void commandLeavesPackUnreadAndRejectsAllOtherRefsAndExtensions() throws Exception {
|
||||
String command =
|
||||
"a".repeat(40)
|
||||
+ " "
|
||||
+ "b".repeat(40)
|
||||
+ " refs/heads/main\0 report-status ofs-delta agent=git/test";
|
||||
ByteArrayOutputStream body = new ByteArrayOutputStream();
|
||||
body.write(pkt(command, null));
|
||||
body.write("PACK".getBytes());
|
||||
ByteArrayInputStream input = new ByteArrayInputStream(body.toByteArray());
|
||||
assertThat(GitIo.command(input).newSha()).isEqualTo("b".repeat(40));
|
||||
assertThat(input.readAllBytes()).isEqualTo("PACK".getBytes());
|
||||
for (String bad :
|
||||
List.of(
|
||||
command.replace("main", "other"),
|
||||
command.replace("refs/heads/main", "refs/tags/v1"),
|
||||
command.replace("b".repeat(40), "0".repeat(40)),
|
||||
command + " push-options",
|
||||
command + " side-band-64k",
|
||||
command + " report-status-v2")) {
|
||||
assertThatThrownBy(() -> GitIo.command(new ByteArrayInputStream(pkt(bad, null))))
|
||||
.isInstanceOf(BusinessException.class);
|
||||
}
|
||||
assertThatThrownBy(
|
||||
() -> GitIo.command(new ByteArrayInputStream(pkt(command, command, null))))
|
||||
.isInstanceOf(BusinessException.class);
|
||||
}
|
||||
|
||||
@Test
|
||||
void advertisementNegotiatesOnlyAuditedReportStatusAndMain() throws Exception {
|
||||
byte[] advertisement =
|
||||
pkt(
|
||||
"# service=git-receive-pack\n",
|
||||
null,
|
||||
"a".repeat(40)
|
||||
+ " refs/heads/main\0report-status report-status-v2 delete-refs"
|
||||
+ " side-band-64k ofs-delta push-options\n",
|
||||
"b".repeat(40) + " refs/tags/v1\n",
|
||||
null);
|
||||
String result =
|
||||
new String(GitIo.receiveAdvertisement(advertisement), StandardCharsets.UTF_8);
|
||||
assertThat(result)
|
||||
.contains("report-status", "ofs-delta", "refs/heads/main")
|
||||
.doesNotContain(
|
||||
"side-band",
|
||||
"delete-refs",
|
||||
"push-options",
|
||||
"report-status-v2",
|
||||
"refs/tags");
|
||||
}
|
||||
|
||||
@Test
|
||||
void statusMustBeGitSuccessNotMerelyHttp200() throws Exception {
|
||||
assertThat(GitIo.pushSucceeded(pkt("unpack ok\n", "ok refs/heads/main\n", null))).isTrue();
|
||||
assertThat(
|
||||
GitIo.pushSucceeded(
|
||||
pkt("unpack ok\n", "ng refs/heads/main non-fast-forward\n", null)))
|
||||
.isFalse();
|
||||
assertThatThrownBy(() -> GitIo.pushSucceeded("HTTP 200".getBytes()))
|
||||
.isInstanceOf(BusinessException.class);
|
||||
assertThatThrownBy(() -> GitIo.pushSucceeded(pkt("unpack ok\n", "ok refs/heads/main\n")))
|
||||
.isInstanceOf(BusinessException.class);
|
||||
}
|
||||
|
||||
@Test
|
||||
void actualBytesAndMetadataAreBoundedWithoutReadAll() throws Exception {
|
||||
assertThatThrownBy(
|
||||
() ->
|
||||
GitIo.limited(new ByteArrayInputStream(new byte[1025]), 1024)
|
||||
.transferTo(OutputStream.nullOutputStream()))
|
||||
.isInstanceOf(BusinessException.class)
|
||||
.extracting("status")
|
||||
.isEqualTo(413);
|
||||
assertThatThrownBy(() -> GitIo.metadata(new ByteArrayInputStream(new byte[1025]), 1024))
|
||||
.isInstanceOf(BusinessException.class);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,46 @@
|
||||
package com.linkerbot.vibemake.service.git;
|
||||
|
||||
import static org.assertj.core.api.Assertions.*;
|
||||
|
||||
import com.linkerbot.vibemake.common.BusinessException;
|
||||
import com.linkerbot.vibemake.service.config.GitGatewayProperties;
|
||||
import com.linkerbot.vibemake.testing.InMemoryStateStore;
|
||||
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
class GitLimitsTest {
|
||||
@Test
|
||||
void poolsAreIndependentAndReleaseAfterCancellationAndUserAndIpAreSeparate() throws Exception {
|
||||
GitGatewayProperties config = new GitGatewayProperties();
|
||||
config.setReadConcurrent(1);
|
||||
config.setPushConcurrent(1);
|
||||
config.setLfsConcurrent(1);
|
||||
config.setGitRequestsPerMinute(1);
|
||||
config.setAuthFailuresPerMinute(1);
|
||||
GitLimits limits = new GitLimits(new InMemoryStateStore(), config);
|
||||
try (AutoCloseable read = limits.acquire(false, false);
|
||||
AutoCloseable push = limits.acquire(true, false);
|
||||
AutoCloseable lfs = limits.acquire(false, true)) {
|
||||
assertThatThrownBy(() -> limits.acquire(false, false))
|
||||
.isInstanceOf(BusinessException.class);
|
||||
assertThatThrownBy(() -> limits.acquire(true, false))
|
||||
.isInstanceOf(BusinessException.class);
|
||||
assertThatThrownBy(() -> limits.acquire(false, true))
|
||||
.isInstanceOf(BusinessException.class);
|
||||
}
|
||||
try (AutoCloseable read = limits.acquire(false, false)) {
|
||||
assertThat(read).isNotNull();
|
||||
}
|
||||
GitAccess first = new GitAccess(1, "user-one", "token-one", "project", true);
|
||||
GitAccess second = new GitAccess(2, "user-two", "token-two", "project", true);
|
||||
limits.request(first, false);
|
||||
limits.request(second, false);
|
||||
limits.request(first, true);
|
||||
assertThatThrownBy(() -> limits.request(first, false))
|
||||
.isInstanceOf(BusinessException.class);
|
||||
limits.authenticationFailed("127.0.0.1");
|
||||
limits.authenticationAllowed("127.0.0.2");
|
||||
assertThatThrownBy(() -> limits.authenticationAllowed("127.0.0.1"))
|
||||
.isInstanceOf(BusinessException.class);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,70 @@
|
||||
package com.linkerbot.vibemake.service.git;
|
||||
|
||||
import static org.assertj.core.api.Assertions.*;
|
||||
import static org.mockito.Mockito.*;
|
||||
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
import com.linkerbot.vibemake.bean.entity.Project;
|
||||
import com.linkerbot.vibemake.common.BusinessException;
|
||||
import com.linkerbot.vibemake.service.ProjectService;
|
||||
import com.linkerbot.vibemake.service.config.*;
|
||||
import com.linkerbot.vibemake.service.gitea.GiteaClient;
|
||||
import com.sun.net.httpserver.HttpServer;
|
||||
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
import java.net.InetSocketAddress;
|
||||
import java.util.concurrent.atomic.AtomicBoolean;
|
||||
|
||||
class GitPolicyTest {
|
||||
@Test
|
||||
void independentlyVerifiesBotIdentityAndRepositoryIdentityBeforeGrantingAnyAccess()
|
||||
throws Exception {
|
||||
AtomicBoolean admin = new AtomicBoolean(false);
|
||||
HttpServer server = HttpServer.create(new InetSocketAddress("127.0.0.1", 0), 0);
|
||||
server.createContext(
|
||||
"/api/v1/user",
|
||||
exchange -> {
|
||||
byte[] bytes =
|
||||
("{\"login\":\"bot\",\"is_admin\":" + admin.get() + "}").getBytes();
|
||||
exchange.sendResponseHeaders(200, bytes.length);
|
||||
exchange.getResponseBody().write(bytes);
|
||||
exchange.close();
|
||||
});
|
||||
server.start();
|
||||
GitGatewayProperties config = new GitGatewayProperties();
|
||||
config.setUpstreamUsername("bot");
|
||||
config.setUpstreamToken("fixture");
|
||||
VibemakeProperties general = new VibemakeProperties();
|
||||
general.getGitea().setBaseUrl("http://127.0.0.1:" + server.getAddress().getPort());
|
||||
GiteaClient remote = mock(GiteaClient.class);
|
||||
ProjectService projects = mock(ProjectService.class);
|
||||
Project project = new Project();
|
||||
project.setGiteaRepoId(9L);
|
||||
project.setGiteaOwner("owner");
|
||||
project.setGiteaRepoName("repo");
|
||||
GiteaClient.Repository repo = new GiteaClient.Repository(9, "owner", "repo");
|
||||
when(projects.repository(project)).thenReturn(repo);
|
||||
when(remote.inspect("repo"))
|
||||
.thenReturn(
|
||||
new GiteaClient.RemoteRepository(
|
||||
9, "owner", "repo", true, false, false, null, "main"));
|
||||
try (GitTransport transport = new GitTransport(config, general)) {
|
||||
GitPolicy policy =
|
||||
new GitPolicy(remote, projects, transport, config, new ObjectMapper());
|
||||
policy.provision(project);
|
||||
verify(remote).ensureGitAccess(repo, "bot");
|
||||
admin.set(true);
|
||||
assertThatThrownBy(() -> policy.verify(project)).isInstanceOf(BusinessException.class);
|
||||
admin.set(false);
|
||||
when(remote.inspect("repo"))
|
||||
.thenReturn(
|
||||
new GiteaClient.RemoteRepository(
|
||||
10, "owner", "repo", true, false, false, null, "main"));
|
||||
assertThatThrownBy(() -> policy.verify(project)).isInstanceOf(BusinessException.class);
|
||||
verify(remote, never()).verifyGitProtection(repo, "bot");
|
||||
} finally {
|
||||
server.stop(0);
|
||||
}
|
||||
}
|
||||
}
|
||||
+72
@@ -0,0 +1,72 @@
|
||||
package com.linkerbot.vibemake.service.git;
|
||||
|
||||
import static org.assertj.core.api.Assertions.*;
|
||||
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
import com.linkerbot.vibemake.common.BusinessException;
|
||||
import com.linkerbot.vibemake.service.config.VibemakeProperties;
|
||||
import com.linkerbot.vibemake.service.gitea.*;
|
||||
import com.sun.net.httpserver.HttpServer;
|
||||
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
import java.net.InetSocketAddress;
|
||||
import java.util.concurrent.atomic.*;
|
||||
|
||||
class GitProtectionTest {
|
||||
@Test
|
||||
void strictExistingRulesAreNeverOverwrittenAndAdminOrForcePushAreRejected() throws Exception {
|
||||
AtomicReference<String> rule = new AtomicReference<>("[]");
|
||||
AtomicReference<String> permission = new AtomicReference<>("write");
|
||||
AtomicInteger writes = new AtomicInteger();
|
||||
HttpServer server = HttpServer.create(new InetSocketAddress("127.0.0.1", 0), 0);
|
||||
server.createContext(
|
||||
"/",
|
||||
exchange -> {
|
||||
String response;
|
||||
if (exchange.getRequestURI().getPath().endsWith("/permission"))
|
||||
response = "{\"permission\":\"" + permission.get() + "\"}";
|
||||
else {
|
||||
if (exchange.getRequestMethod().equals("POST")) {
|
||||
writes.incrementAndGet();
|
||||
rule.set(
|
||||
"[{\"rule_name\":\"main\",\"enable_push\":true,\"enable_force_push\":false}]");
|
||||
}
|
||||
response = rule.get();
|
||||
}
|
||||
byte[] bytes = response.getBytes();
|
||||
exchange.sendResponseHeaders(200, bytes.length);
|
||||
exchange.getResponseBody().write(bytes);
|
||||
exchange.close();
|
||||
});
|
||||
server.start();
|
||||
try {
|
||||
VibemakeProperties config = new VibemakeProperties();
|
||||
config.getGitea().setBaseUrl("http://127.0.0.1:" + server.getAddress().getPort());
|
||||
config.getGitea().setToken("fixture");
|
||||
GiteaClient client = new RestGiteaClient(config, new ObjectMapper());
|
||||
GiteaClient.Repository repo = new GiteaClient.Repository(1, "fixture", "repo");
|
||||
client.ensureGitProtection(repo, "bot");
|
||||
assertThat(writes.get()).isEqualTo(1);
|
||||
client.ensureGitProtection(repo, "bot");
|
||||
assertThat(writes.get()).isEqualTo(1);
|
||||
permission.set("admin");
|
||||
assertThatThrownBy(() -> client.verifyGitProtection(repo, "bot"))
|
||||
.isInstanceOf(BusinessException.class);
|
||||
permission.set("write");
|
||||
for (String bad :
|
||||
new String[] {
|
||||
"[{\"rule_name\":\"main\",\"enable_push\":true,\"enable_force_push\":true}]",
|
||||
"[{\"rule_name\":\"*\",\"enable_push\":true,\"enable_force_push\":false}]",
|
||||
"[{\"rule_name\":\"main\",\"enable_push\":false,\"enable_force_push\":false}]"
|
||||
}) {
|
||||
rule.set(bad);
|
||||
assertThatThrownBy(() -> client.ensureGitProtection(repo, "bot"))
|
||||
.isInstanceOf(BusinessException.class);
|
||||
}
|
||||
assertThat(writes.get()).isEqualTo(1);
|
||||
} finally {
|
||||
server.stop(0);
|
||||
}
|
||||
}
|
||||
}
|
||||
+72
@@ -0,0 +1,72 @@
|
||||
package com.linkerbot.vibemake.service.git;
|
||||
|
||||
import static org.assertj.core.api.Assertions.*;
|
||||
|
||||
import com.linkerbot.vibemake.common.BusinessException;
|
||||
import com.linkerbot.vibemake.service.config.*;
|
||||
import com.sun.net.httpserver.HttpServer;
|
||||
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
import java.io.*;
|
||||
import java.net.InetSocketAddress;
|
||||
import java.time.Duration;
|
||||
import java.util.Map;
|
||||
import java.util.concurrent.*;
|
||||
|
||||
class GitTransportTest {
|
||||
@Test
|
||||
void budgetCoversStalledResponseBodyAndNeverFollowsRedirects() throws Exception {
|
||||
HttpServer server = HttpServer.create(new InetSocketAddress("127.0.0.1", 0), 0);
|
||||
server.setExecutor(Executors.newVirtualThreadPerTaskExecutor());
|
||||
CountDownLatch release = new CountDownLatch(1);
|
||||
server.createContext(
|
||||
"/stall",
|
||||
exchange -> {
|
||||
exchange.sendResponseHeaders(200, 0);
|
||||
exchange.getResponseBody().write(1);
|
||||
exchange.getResponseBody().flush();
|
||||
try {
|
||||
release.await(5, TimeUnit.SECONDS);
|
||||
} catch (InterruptedException error) {
|
||||
Thread.currentThread().interrupt();
|
||||
}
|
||||
exchange.close();
|
||||
});
|
||||
server.createContext(
|
||||
"/redirect",
|
||||
exchange -> {
|
||||
exchange.getResponseHeaders().set("Location", "http://untrusted.invalid/");
|
||||
exchange.sendResponseHeaders(302, -1);
|
||||
exchange.close();
|
||||
});
|
||||
server.start();
|
||||
GitGatewayProperties properties = new GitGatewayProperties();
|
||||
properties.setTimeout(Duration.ofMillis(300));
|
||||
VibemakeProperties config = new VibemakeProperties();
|
||||
config.getGitea().setBaseUrl("http://127.0.0.1:" + server.getAddress().getPort());
|
||||
try (GitTransport transport = new GitTransport(properties, config)) {
|
||||
try (GitTransport.Budget budget = transport.budget();
|
||||
GitTransport.Response result =
|
||||
transport.exchange("/stall", "GET", null, -1, Map.of(), budget)) {
|
||||
long started = System.nanoTime();
|
||||
assertThatThrownBy(() -> result.input().transferTo(OutputStream.nullOutputStream()))
|
||||
.isInstanceOf(BusinessException.class)
|
||||
.extracting("code")
|
||||
.isEqualTo("GIT_TIMEOUT");
|
||||
assertThat(Duration.ofNanos(System.nanoTime() - started))
|
||||
.isLessThan(Duration.ofSeconds(3));
|
||||
}
|
||||
try (GitTransport.Budget budget = transport.budget()) {
|
||||
assertThatThrownBy(
|
||||
() ->
|
||||
transport.exchange(
|
||||
"/redirect", "GET", null, -1, Map.of(), budget))
|
||||
.isInstanceOf(BusinessException.class);
|
||||
}
|
||||
} finally {
|
||||
release.countDown();
|
||||
server.stop(0);
|
||||
}
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user