后台管理adminapi模块

This commit is contained in:
lxp
2026-10-20 17:25:18 +08:00
parent ee5c665d6d
commit 2b12321831
64 changed files with 5218 additions and 345 deletions
+12
View File
@@ -44,3 +44,15 @@ FORK_MAX_CONCURRENT=2
FORK_MAX_TEMPORARY_BYTES=5368709120
# 可选:独立临时磁盘目录,默认系统临时目录下 community-fork。
# FORK_TEMPORARY_DIRECTORY=/var/tmp/community-fork
# 管理端:共用上述 DB_* / GITEA_* / OSS_*,独立端口、跨域来源及本机登录会话。
ADMIN_SERVER_PORT=8994
# 多个来源用英文逗号分隔;默认不开放跨域。本地可使用前端开发代理。
ADMIN_ALLOWED_ORIGINS=
ADMIN_IMPORT_WORKER_ENABLED=true
ADMIN_IMPORT_CONCURRENCY=2
ADMIN_MIGRATION_TIMEOUT=30m
# 仅真实 MySQL 隔离验证时需要;未设置时脚本使用 DB_*,需具有创建临时库权限。
# ADMIN_MYSQL_TEST_URL=
# ADMIN_MYSQL_TEST_USERNAME=
# ADMIN_MYSQL_TEST_PASSWORD=
+21 -4
View File
@@ -1,6 +1,6 @@
# Community
Spring Boot 社区后端,工程分层参考 cadworld。使用 Java 21、Spring Boot 3.5.7、MyBatis-Plus 3.5.11、MySQL 和 Gitea API v1。业务库名为 community,只有 sys_user、project、project_comment、project_like 四张表。
Spring Boot 社区后端,工程分层参考 cadworld。使用 Java 21、Spring Boot 3.5.7、MyBatis-Plus 3.5.11、MySQL 和 Gitea API v1。业务库名为 community,包含 sys_user、project、project_comment、project_like,以及管理端的 project_import_task。
平台管理用户、项目归属、类型、发布、评论、点赞和 Fork 来源;Gitea 管理文件、目录、Commit 与历史。仓库统一存放在配置的 Gitea 组织下,为私有仓库。
@@ -15,20 +15,21 @@ Spring Boot 社区后端,工程分层参考 cadworld。使用 Java 21、Spring
| community-service | 业务 Service 接口在 service 根包,实现类在 service.impl;包含 mapper、config、gitea、ai、support |
| community-security | 平台登录会话、验证码、身份上下文、认证过滤器和安全配置 |
| community-common | 通用响应、分页、异常处理、requestId 和基础工具 |
| community-adminapi | 独立管理端应用,用户角色鉴权、用户与全站项目管理、GitHub 在线迁移 |
| community-import | 固定 Excel 的 GitHub 项目运维导入命令,不提供 HTTP 接口 |
UserService、ProjectService、FileService、InteractionService、CoverUploadService、UrdfService 采用接口加实现类,Controller、安全模块和服务之间通过接口注入。GiteaClient 与 RestGiteaClient 定义仓库调用边界;项目锁、UTC 时间和分页检查归入 support。安全模块内的 TokenService、CaptchaService 保留具体类,首期沿用本机缓存。
## 功能与边界
支持邮箱加图片验证码登录、我的项目列表、在线文本编辑、新建文件和目录、上传及多文件提交、手动发布和撤回、已发布版本浏览、快照 Fork、平铺评论和点赞,以及可配置开启的 Maker AI(聊天问答、查找、创建空项目、Fork)。工作区是用户项目列表,没有 workspace 表。没有 repository_operation 表、草稿保存、类型表、标签、收藏、成员协作或管理后台。
支持邮箱加图片验证码登录、我的项目列表、在线文本编辑、新建文件和目录、上传及多文件提交、手动发布和撤回、已发布版本浏览、快照 Fork、平铺评论和点赞,以及可配置开启的 Maker AI(聊天问答、查找、创建空项目、Fork)。工作区是用户项目列表,没有 workspace 表。没有 repository_operation 表、草稿保存、类型表、标签、收藏、成员协作;管理后台接口由 community-adminapi 提供。
**当前登录不验证邮箱归属,知道别人的邮箱也可能登录其账号。** 验证码和登录会话使用本机缓存,单实例部署,重启后重新登录。所有仓库写入统一经过平台后端。
**普通用户登录不验证邮箱归属,知道别人的邮箱也可能登录其账号;ADMIN 和 OFFICIAL 禁止通过该入口登录,后台管理员必须验证密码。** 验证码和登录会话使用本机缓存,单实例部署,重启后重新登录。所有仓库写入统一经过平台后端。
## 启动
1. 安装 Java 21、Maven 3.9+,准备 MySQL 8.0.13 及以上和现有 Gitea。
2. 手动执行 [四表建表 SQL](docs/database.md),应用不会自动建表或修改数据库。
2. 新建数据库手动执行 [基础四表 SQL](docs/database.md),并创建 [迁移任务表](docs/admin-database.md)。已有数据库执行管理端文档的增量 SQL;应用不会自动迁库。
3. 在 Gitea 创建私有组织,将技术账号设为组织所有者,生成具有 write:repository、write:organization、read:user 权限的 Token。
4. 将 .env.example 复制为 .env,填写数据库连接与 Gitea 地址、组织、Token;默认使用 test 环境,.env 被 Git 忽略,不提交真实凭据。
5. 执行:
@@ -252,3 +253,19 @@ bash scripts/import-github-projects.sh --excel "$IMPORT_EXCEL" --execute --owner
报告逐项包含 Sheet、行号、短编码、阶段、错误码和兼容性提醒。退出码 0 表示本次选定工作完成,1 表示命令、配置或持久化等致命错误,2 表示存在预检未通过、失败或待核对项目。在线仓库或图片失效会减少实际导入量;报告中的候选数仍与静态筛选对账。
本次只完成仓库迁移、关联和发布。在线浏览和下载仍有单文件限制,LFS 实体、子模块及符号链接的在线浏览能力未扩展;Git 快照 Fork 已支持大文件、LFS 实体、符号链接及子模块引用,依赖仓库仍不递归复制。当前发布快照检测到的限制会写入报告。子模块只保留引用,不递归迁移独立仓库,也不下载网盘或其他外链资源。保留所有分支不代表社区提供分支切换界面。
## 管理端
独立端口默认 `8994`,共用现有 `sys_user`,通过 `USER/ADMIN/OFFICIAL` 角色鉴权,不新增管理员或会话表。后台管理员以邮箱和 BCrypt 密码登录;会话使用本机缓存,默认 8 小时,重启后重新登录。第一版社区端和后台各单实例,Token 不互通。
启用前先手工执行 [管理端 SQL](docs/admin-database.md),停止并排空旧写入进程,更新社区端、后台和 Excel 导入工具,最后启用后台。全部仓库写入端使用相同数据库的项目命名锁;不可让仍使用本地锁的旧版应用并行写入。
```bash
bash scripts/run-admin-local.sh
# 将一个已有普通用户初始化为管理员,交互输入密码,不传明文密码参数。
bash scripts/run-admin-local.sh --bootstrap-admin-user-code='<已有用户编码>'
# 真实 MySQL 验证:创建并删除随机临时库,不改现有业务数据。
bash scripts/test-admin-mysql.sh
```
前端对接见 [管理后台接口文档](docs/admin-api.md),服务初始化、配置与故障处理见 [管理端部署说明](docs/admin-deployment.md)。数据库 SQL 不会由启动脚本自动执行;真实仓库迁移和正式发布需在部署后由管理员通过接口触发,普通测试不会访问真实 GitHub、Gitea 或 OSS。
+39
View File
@@ -0,0 +1,39 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>com.linkerbot</groupId>
<artifactId>community</artifactId>
<version>0.0.1-SNAPSHOT</version>
</parent>
<artifactId>community-adminapi</artifactId>
<dependencies>
<dependency>
<groupId>com.linkerbot</groupId>
<artifactId>community-security</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.springdoc</groupId>
<artifactId>springdoc-openapi-starter-webmvc-ui</artifactId>
</dependency>
<dependency>
<groupId>com.mysql</groupId>
<artifactId>mysql-connector-j</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>com.h2database</groupId>
<artifactId>h2</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
@@ -0,0 +1,26 @@
package com.linkerbot.community.adminapi;
import com.linkerbot.community.security.TokenService;
import org.mybatis.spring.annotation.MapperScan;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.autoconfigure.security.servlet.UserDetailsServiceAutoConfiguration;
import org.springframework.context.annotation.Import;
import org.springframework.scheduling.annotation.EnableScheduling;
@EnableScheduling
@MapperScan("com.linkerbot.community.service.mapper")
@Import(TokenService.class)
@SpringBootApplication(
scanBasePackages = {
"com.linkerbot.community.adminapi",
"com.linkerbot.community.service",
"com.linkerbot.community.common"
},
exclude = UserDetailsServiceAutoConfiguration.class)
public class AdminApplication {
public static void main(String[] args) {
SpringApplication.run(AdminApplication.class, args);
}
}
@@ -0,0 +1,39 @@
package com.linkerbot.community.adminapi.config;
import com.linkerbot.community.adminapi.security.AdminAuthService;
import lombok.RequiredArgsConstructor;
import org.springframework.boot.*;
import org.springframework.context.ConfigurableApplicationContext;
import org.springframework.stereotype.Component;
import java.io.Console;
import java.util.Arrays;
@Component
@RequiredArgsConstructor
public class AdminBootstrap implements ApplicationRunner {
private final AdminAuthService auth;
private final ConfigurableApplicationContext context;
@Override
public void run(ApplicationArguments arguments) {
if (!arguments.containsOption("bootstrap-admin-user-code")) return;
Console console = System.console();
if (console == null) throw new IllegalStateException("初始化管理员需要交互终端,请直接运行 jar");
char[] password = console.readPassword("请输入管理员密码(至少12字符):");
char[] confirmation = console.readPassword("请再次输入密码:");
try {
if (password == null || confirmation == null || !Arrays.equals(password, confirmation))
throw new IllegalArgumentException("两次密码不一致");
auth.bootstrap(
arguments.getOptionValues("bootstrap-admin-user-code").getFirst(),
new String(password));
} finally {
if (password != null) Arrays.fill(password, '\0');
if (confirmation != null) Arrays.fill(confirmation, '\0');
}
SpringApplication.exit(context);
}
}
@@ -0,0 +1,25 @@
package com.linkerbot.community.adminapi.config;
import io.swagger.v3.oas.models.Components;
import io.swagger.v3.oas.models.OpenAPI;
import io.swagger.v3.oas.models.info.Info;
import io.swagger.v3.oas.models.security.*;
import org.springframework.context.annotation.*;
@Configuration
public class AdminOpenApiConfiguration {
@Bean
public OpenAPI adminOpenApi() {
return new OpenAPI()
.info(new Info().title("Community 管理端接口").version("v1"))
.components(
new Components()
.addSecuritySchemes(
"bearerAuth",
new SecurityScheme()
.type(SecurityScheme.Type.HTTP)
.scheme("bearer")))
.addSecurityItem(new SecurityRequirement().addList("bearerAuth"));
}
}
@@ -0,0 +1,99 @@
package com.linkerbot.community.adminapi.config;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.linkerbot.community.adminapi.security.AdminTokenFilter;
import com.linkerbot.community.common.*;
import com.linkerbot.community.service.config.AdminProperties;
import org.springframework.boot.web.servlet.FilterRegistrationBean;
import org.springframework.context.annotation.*;
import org.springframework.http.HttpMethod;
import org.springframework.security.config.annotation.web.builders.HttpSecurity;
import org.springframework.security.config.http.SessionCreationPolicy;
import org.springframework.security.web.SecurityFilterChain;
import org.springframework.security.web.authentication.UsernamePasswordAuthenticationFilter;
import org.springframework.web.cors.*;
import java.util.List;
@Configuration
public class AdminSecurityConfiguration {
@Bean
public FilterRegistrationBean<AdminTokenFilter> disableAdminFilterRegistration(
AdminTokenFilter filter) {
FilterRegistrationBean<AdminTokenFilter> registration =
new FilterRegistrationBean<>(filter);
registration.setEnabled(false);
return registration;
}
@Bean
public SecurityFilterChain adminSecurity(
HttpSecurity http,
AdminTokenFilter filter,
ObjectMapper json,
AdminProperties properties)
throws Exception {
CorsConfiguration cors = new CorsConfiguration();
cors.setAllowedOrigins(properties.getAllowedOrigins());
cors.setAllowedMethods(List.of("GET", "HEAD", "POST", "PUT", "PATCH", "DELETE", "OPTIONS"));
cors.setAllowedHeaders(List.of("Authorization", "Content-Type", "Accept"));
cors.setExposedHeaders(List.of("X-Request-Id", "Content-Disposition"));
cors.setAllowCredentials(false);
cors.setMaxAge(3600L);
UrlBasedCorsConfigurationSource source = new UrlBasedCorsConfigurationSource();
source.registerCorsConfiguration("/admin-api/**", cors);
http.cors(config -> config.configurationSource(source))
.csrf(config -> config.disable())
.sessionManagement(
config -> config.sessionCreationPolicy(SessionCreationPolicy.STATELESS))
.formLogin(config -> config.disable())
.httpBasic(config -> config.disable())
.logout(config -> config.disable())
.authorizeHttpRequests(
config ->
config.requestMatchers(HttpMethod.OPTIONS, "/**")
.permitAll()
.requestMatchers(
"/admin-api/auth/login",
"/error",
"/v3/api-docs/**",
"/swagger-ui/**",
"/swagger-ui.html")
.permitAll()
.requestMatchers("/admin-api/**")
.hasRole("ADMIN")
.anyRequest()
.denyAll())
.exceptionHandling(
config ->
config.authenticationEntryPoint(
(request, response, error) -> {
response.setStatus(401);
response.setContentType(
"application/json;charset=UTF-8");
json.writeValue(
response.getOutputStream(),
ApiResponse.failure(
new BusinessException(
401,
"UNAUTHORIZED",
"请登录后台")));
})
.accessDeniedHandler(
(request, response, error) -> {
response.setStatus(403);
response.setContentType(
"application/json;charset=UTF-8");
json.writeValue(
response.getOutputStream(),
ApiResponse.failure(
new BusinessException(
403,
"FORBIDDEN",
"无后台访问权限")));
}))
.addFilterBefore(filter, UsernamePasswordAuthenticationFilter.class);
return http.build();
}
}
@@ -0,0 +1,83 @@
package com.linkerbot.community.adminapi.config;
import com.linkerbot.community.service.admin.AdminImportService;
import com.linkerbot.community.service.config.AdminProperties;
import jakarta.annotation.PreDestroy;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.boot.ApplicationArguments;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Component;
import java.util.concurrent.*;
@Component
public class ImportWorker {
private static final Logger log = LoggerFactory.getLogger(ImportWorker.class);
private final AdminImportService imports;
private final boolean enabled;
private final ExecutorService pool;
private final Semaphore permits;
private boolean recovered;
public ImportWorker(
AdminImportService imports,
AdminProperties properties,
ApplicationArguments arguments) {
this.imports = imports;
this.enabled =
properties.isWorkerEnabled()
&& !arguments.containsOption("bootstrap-admin-user-code");
this.permits = new Semaphore(properties.getImportConcurrency());
this.pool =
new ThreadPoolExecutor(
properties.getImportConcurrency(),
properties.getImportConcurrency(),
0,
TimeUnit.MILLISECONDS,
new ArrayBlockingQueue<>(properties.getImportConcurrency()),
Thread.ofPlatform().name("admin-import-", 0).factory());
}
@Scheduled(fixedDelay = 1000, initialDelay = 1000)
public void dispatch() {
if (!enabled || pool.isShutdown()) return;
try {
if (!recovered) {
imports.recover();
recovered = true;
}
if (permits.availablePermits() == 0) return;
for (String code : imports.queued()) {
if (!permits.tryAcquire()) break;
try {
pool.execute(
() -> {
try {
imports.run(code);
} catch (RuntimeException error) {
log.error(
"迁移工作线程异常 taskCode={} errorType={}",
code,
error.getClass().getSimpleName());
} finally {
permits.release();
}
});
} catch (RejectedExecutionException error) {
permits.release();
}
}
} catch (RuntimeException error) {
log.error("迁移任务调度失败 errorType={}", error.getClass().getSimpleName());
}
}
@PreDestroy
public void close() throws InterruptedException {
pool.shutdown();
if (!pool.awaitTermination(5, TimeUnit.SECONDS)) pool.shutdownNow();
}
}
@@ -0,0 +1,32 @@
package com.linkerbot.community.adminapi.config;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Profile;
import org.springframework.core.env.Environment;
import java.util.List;
@Configuration(proxyBeanMethods = false)
@Profile("prod")
public class ProductionConfiguration {
public ProductionConfiguration(Environment environment) {
for (String property :
List.of(
"spring.datasource.url",
"spring.datasource.username",
"spring.datasource.password",
"community.gitea.base-url",
"community.gitea.token")) {
String value;
try {
value = environment.getRequiredProperty(property);
} catch (IllegalArgumentException | IllegalStateException error) {
// 仅提示配置项名称,避免异常链携带配置值。
throw new IllegalStateException("生产环境必须配置 " + property);
}
if (value.isBlank()) {
throw new IllegalStateException("生产环境必须配置 " + property);
}
}
}
}
@@ -0,0 +1,48 @@
package com.linkerbot.community.adminapi.controller;
import com.linkerbot.community.adminapi.security.*;
import com.linkerbot.community.bean.AdminContracts.*;
import com.linkerbot.community.common.ApiResponse;
import io.swagger.v3.oas.annotations.security.SecurityRequirements;
import jakarta.servlet.http.HttpServletRequest;
import jakarta.validation.Valid;
import lombok.RequiredArgsConstructor;
import org.springframework.http.*;
import org.springframework.web.bind.annotation.*;
@RestController
@RequiredArgsConstructor
@RequestMapping("/admin-api/auth")
public class AdminAuthController {
private final AdminAuthService auth;
@PostMapping("/login")
@SecurityRequirements
public ResponseEntity<ApiResponse<LoginView>> login(
@Valid @RequestBody LoginInput input, HttpServletRequest request) {
return ResponseEntity.ok()
.cacheControl(CacheControl.noStore())
.body(ApiResponse.success(auth.login(input, request.getRemoteAddr())));
}
@GetMapping("/me")
public ApiResponse<AdminView> me() {
return ApiResponse.success(auth.view(CurrentAdmin.required()));
}
@PostMapping("/logout")
public ApiResponse<Void> logout(@RequestHeader("Authorization") String header) {
auth.logout(header.substring(7));
return ApiResponse.success(null);
}
@PutMapping("/password")
public ApiResponse<Void> password(@Valid @RequestBody PasswordInput input) {
auth.password(CurrentAdmin.required(), input);
return ApiResponse.success(null);
}
}
@@ -0,0 +1,48 @@
package com.linkerbot.community.adminapi.controller;
import com.linkerbot.community.bean.AdminContracts.*;
import com.linkerbot.community.common.*;
import com.linkerbot.community.service.admin.AdminImportService;
import jakarta.validation.Valid;
import lombok.RequiredArgsConstructor;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.*;
@RestController
@RequiredArgsConstructor
@RequestMapping("/admin-api/project-imports")
public class AdminImportController {
private final AdminImportService service;
@PostMapping("/preview")
public ApiResponse<ImportPreview> preview(@Valid @RequestBody PreviewInput input) {
return ApiResponse.success(service.preview(input.repositoryUrl()));
}
@PostMapping
public ResponseEntity<ApiResponse<TaskView>> create(@Valid @RequestBody ImportInput input) {
return ResponseEntity.accepted().body(ApiResponse.success(service.create(input)));
}
@GetMapping
public ApiResponse<PageResponse<TaskView>> list(
@RequestParam(defaultValue = "1") long page,
@RequestParam(defaultValue = "20") long size,
@RequestParam(required = false) String status,
@RequestParam(required = false) String projectCode) {
return ApiResponse.success(service.list(page, size, status, projectCode));
}
@GetMapping("/{taskCode}")
public ApiResponse<TaskView> detail(@PathVariable String taskCode) {
return ApiResponse.success(service.view(service.task(taskCode)));
}
@PostMapping("/{taskCode}/retry")
public ResponseEntity<ApiResponse<TaskView>> retry(@PathVariable String taskCode) {
return ResponseEntity.accepted().body(ApiResponse.success(service.retry(taskCode)));
}
}
@@ -0,0 +1,181 @@
package com.linkerbot.community.adminapi.controller;
import com.linkerbot.community.bean.AdminContracts.*;
import com.linkerbot.community.bean.Contracts.*;
import com.linkerbot.community.common.*;
import com.linkerbot.community.service.*;
import com.linkerbot.community.service.admin.AdminManagementService;
import com.linkerbot.community.service.gitea.GiteaClient;
import jakarta.validation.Valid;
import lombok.RequiredArgsConstructor;
import org.springframework.http.*;
import org.springframework.web.bind.annotation.*;
import org.springframework.web.multipart.MultipartFile;
import java.nio.charset.StandardCharsets;
import java.util.List;
@RestController
@RequiredArgsConstructor
@RequestMapping("/admin-api/projects")
public class AdminProjectController {
private final AdminManagementService management;
private final ProjectService projects;
private final FileService files;
private final UrdfService urdf;
@GetMapping
public ApiResponse<PageResponse<ManagedProjectView>> list(
@RequestParam(defaultValue = "1") long page,
@RequestParam(defaultValue = "20") long size,
@RequestParam(required = false) String ownerUserCode,
@RequestParam(required = false) String type,
@RequestParam(required = false) String keyword,
@RequestParam(required = false) String publishStatus,
@RequestParam(required = false) String repoStatus,
@RequestParam(required = false) String moderationStatus) {
return ApiResponse.success(
management.projects(
page,
size,
ownerUserCode,
type,
keyword,
publishStatus,
repoStatus,
moderationStatus));
}
@GetMapping("/types")
public ApiResponse<List<TypeView>> types() {
return ApiResponse.success(projects.types());
}
@PostMapping
public ApiResponse<ProjectView> create(@Valid @RequestBody CreateProjectInput input) {
return ApiResponse.success(management.create(input));
}
@GetMapping("/{projectCode}")
public ApiResponse<ManagedProjectView> detail(@PathVariable String projectCode) {
return ApiResponse.success(management.detail(projectCode));
}
@PatchMapping("/{projectCode}")
public ApiResponse<ProjectView> update(
@PathVariable String projectCode, @Valid @RequestBody ProjectInput input) {
return ApiResponse.success(projects.update(0, projectCode, input));
}
@DeleteMapping("/{projectCode}")
public ApiResponse<Void> delete(@PathVariable String projectCode) {
projects.delete(0, projectCode);
return ApiResponse.success(null);
}
@GetMapping("/{projectCode}/tree")
public ApiResponse<TreeView> tree(
@PathVariable String projectCode,
@RequestParam(defaultValue = "") String path,
@RequestParam(required = false) String commitSha) {
return ApiResponse.success(files.ownerTree(0, projectCode, path, commitSha));
}
@GetMapping("/{projectCode}/files")
public ResponseEntity<?> file(
@PathVariable String projectCode,
@RequestParam String path,
@RequestParam(required = false) String commitSha,
@RequestParam(defaultValue = "false") boolean download) {
if (!download)
return ResponseEntity.ok(
ApiResponse.success(files.ownerFileView(0, projectCode, path, commitSha)));
GiteaClient.StoredFile file = files.ownerFile(0, projectCode, path, commitSha);
return ResponseEntity.ok()
.contentType(MediaType.APPLICATION_OCTET_STREAM)
.header(
HttpHeaders.CONTENT_DISPOSITION,
ContentDisposition.attachment()
.filename(
file.path().substring(file.path().lastIndexOf('/') + 1),
StandardCharsets.UTF_8)
.build()
.toString())
.body(file.bytes());
}
@GetMapping("/{projectCode}/commits")
public ApiResponse<PageResponse<HistoryEntry>> history(
@PathVariable String projectCode,
@RequestParam(defaultValue = "1") long page,
@RequestParam(defaultValue = "20") long size) {
return ApiResponse.success(files.history(0, projectCode, page, size));
}
@PostMapping("/{projectCode}/commits")
public ApiResponse<CommitView> commit(
@PathVariable String projectCode, @Valid @RequestBody CommitInput input) {
return ApiResponse.success(files.commit(0, projectCode, input));
}
@PostMapping(
value = "/{projectCode}/uploads/commit",
consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
public ApiResponse<CommitView> upload(
@PathVariable String projectCode,
@RequestParam String expectedCommitSha,
@RequestParam String message,
@RequestParam(defaultValue = "") String directory,
@RequestParam("files") List<MultipartFile> uploaded,
@RequestParam(required = false) List<String> paths) {
return ApiResponse.success(
files.upload(
0, projectCode, expectedCommitSha, message, directory, uploaded, paths));
}
@PutMapping("/{projectCode}/publication")
public ApiResponse<ProjectView> publish(
@PathVariable String projectCode, @Valid @RequestBody PublicationInput input) {
return ApiResponse.success(projects.publish(0, projectCode, input.commitSha()));
}
@DeleteMapping("/{projectCode}/publication")
public ApiResponse<Void> withdraw(@PathVariable String projectCode) {
projects.withdraw(0, projectCode);
return ApiResponse.success(null);
}
@PutMapping("/{projectCode}/moderation")
public ApiResponse<Void> block(
@PathVariable String projectCode, @Valid @RequestBody ReasonInput input) {
management.moderate(projectCode, true, input.reason());
return ApiResponse.success(null);
}
@DeleteMapping("/{projectCode}/moderation")
public ApiResponse<Void> unblock(@PathVariable String projectCode) {
management.moderate(projectCode, false, null);
return ApiResponse.success(null);
}
@PostMapping("/{projectCode}/urdf/signature")
public ApiResponse<UrdfUploadSignature> urdfSignature(
@PathVariable String projectCode, @Valid @RequestBody UrdfUploadInput input) {
return ApiResponse.success(urdf.signature(0, projectCode, input));
}
@PutMapping("/{projectCode}/urdf")
public ApiResponse<ProjectView> urdfConfirm(
@PathVariable String projectCode, @Valid @RequestBody UrdfConfirmInput input) {
return ApiResponse.success(urdf.confirm(0, projectCode, input));
}
@DeleteMapping("/{projectCode}/urdf")
public ApiResponse<Void> urdfClear(@PathVariable String projectCode) {
urdf.clear(0, projectCode);
return ApiResponse.success(null);
}
}
@@ -0,0 +1,27 @@
package com.linkerbot.community.adminapi.controller;
import com.linkerbot.community.bean.Contracts.*;
import com.linkerbot.community.common.ApiResponse;
import com.linkerbot.community.service.CoverUploadService;
import jakarta.validation.Valid;
import lombok.RequiredArgsConstructor;
import org.springframework.http.*;
import org.springframework.web.bind.annotation.*;
@RestController
@RequiredArgsConstructor
@RequestMapping("/admin-api/uploads")
public class AdminUploadController {
private final CoverUploadService covers;
@PostMapping("/project-covers/signature")
public ResponseEntity<ApiResponse<CoverUploadSignature>> signature(
@Valid @RequestBody CoverUploadInput input) {
return ResponseEntity.ok()
.cacheControl(CacheControl.noStore())
.body(ApiResponse.success(covers.signature(0, input)));
}
}
@@ -0,0 +1,58 @@
package com.linkerbot.community.adminapi.controller;
import com.linkerbot.community.bean.AdminContracts.*;
import com.linkerbot.community.common.*;
import com.linkerbot.community.service.admin.AdminManagementService;
import jakarta.validation.Valid;
import lombok.RequiredArgsConstructor;
import org.springframework.web.bind.annotation.*;
@RestController
@RequiredArgsConstructor
@RequestMapping("/admin-api/users")
public class AdminUserController {
private final AdminManagementService service;
@GetMapping
public ApiResponse<PageResponse<UserView>> list(
@RequestParam(defaultValue = "1") long page,
@RequestParam(defaultValue = "20") long size,
@RequestParam(required = false) String userCode,
@RequestParam(required = false) String nickname,
@RequestParam(required = false) String email,
@RequestParam(required = false) String role,
@RequestParam(required = false) String status) {
return ApiResponse.success(
service.users(page, size, userCode, nickname, email, role, status));
}
@GetMapping("/{userCode}")
public ApiResponse<UserView> detail(@PathVariable String userCode) {
return ApiResponse.success(service.userView(service.user(userCode)));
}
@PatchMapping("/{userCode}")
public ApiResponse<UserView> nickname(
@PathVariable String userCode, @Valid @RequestBody NicknameInput input) {
return ApiResponse.success(service.nickname(userCode, input.nickname()));
}
@PutMapping("/{userCode}/status")
public ApiResponse<Void> status(
@PathVariable String userCode, @Valid @RequestBody StatusInput input) {
service.status(userCode, input.status());
return ApiResponse.success(null);
}
@GetMapping("/{userCode}/projects")
public ApiResponse<PageResponse<ManagedProjectView>> projects(
@PathVariable String userCode,
@RequestParam(defaultValue = "1") long page,
@RequestParam(defaultValue = "20") long size) {
return ApiResponse.success(
service.projects(page, size, userCode, null, null, null, null, null));
}
}
@@ -0,0 +1,142 @@
package com.linkerbot.community.adminapi.security;
import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper;
import com.baomidou.mybatisplus.core.conditions.update.LambdaUpdateWrapper;
import com.github.benmanes.caffeine.cache.Cache;
import com.github.benmanes.caffeine.cache.Caffeine;
import com.linkerbot.community.bean.AdminContracts.*;
import com.linkerbot.community.bean.entity.User;
import com.linkerbot.community.common.BusinessException;
import com.linkerbot.community.security.TokenService;
import com.linkerbot.community.service.config.CommunityProperties;
import com.linkerbot.community.service.mapper.UserMapper;
import com.linkerbot.community.service.support.ProjectSupport;
import lombok.RequiredArgsConstructor;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.security.crypto.bcrypt.BCryptPasswordEncoder;
import org.springframework.stereotype.Service;
import java.nio.charset.StandardCharsets;
import java.time.Duration;
import java.util.Locale;
@Service
@RequiredArgsConstructor
public class AdminAuthService {
private static final Logger log = LoggerFactory.getLogger(AdminAuthService.class);
private final UserMapper users;
private final TokenService tokens;
private final CommunityProperties properties;
private final BCryptPasswordEncoder encoder = new BCryptPasswordEncoder(12);
private final String dummyHash = encoder.encode("invalid-admin-password");
private final Cache<String, Integer> failures =
Caffeine.newBuilder()
.maximumSize(10000)
.expireAfterWrite(Duration.ofMinutes(15))
.build();
public LoginView login(LoginInput input, String remoteAddress) {
String email = input.email().trim().toLowerCase(Locale.ROOT);
checkLimit("ip:" + remoteAddress, 30);
checkLimit("account:" + email, 5);
User user = users.selectOne(new LambdaQueryWrapper<User>().eq(User::getEmail, email));
String hash =
user == null || user.getPasswordHash() == null ? dummyHash : user.getPasswordHash();
boolean matches =
input.password().getBytes(StandardCharsets.UTF_8).length <= 72
&& encoder.matches(input.password(), hash);
if (!matches
|| user == null
|| !"ACTIVE".equals(user.getStatus())
|| !"ADMIN".equals(user.getRole())) {
failure("ip:" + remoteAddress);
failure("account:" + email);
log.warn("后台登录失败 code=ADMIN_LOGIN_FAILED");
throw new BusinessException(401, "ADMIN_LOGIN_FAILED", "账号或密码错误");
}
failures.invalidate("account:" + email);
log.info("后台登录 userCode={}", user.getUserCode());
return new LoginView(
tokens.issue(user.getId(), user.getSessionVersion()),
"Bearer",
properties.getTokenTtl().toSeconds(),
view(user));
}
private void checkLimit(String key, int limit) {
Integer count = failures.getIfPresent(key);
if (count != null && count >= limit)
throw new BusinessException(429, "ADMIN_LOGIN_LIMITED", "尝试次数过多,请15分钟后重试");
}
private void failure(String key) {
failures.asMap().compute(key, (ignored, count) -> count == null ? 1 : count + 1);
}
public User authenticate(String token) {
TokenService.Session session = tokens.session(token);
if (session == null) throw new BusinessException(401, "UNAUTHORIZED", "请登录后台");
User user = users.selectById(session.userId());
if (user == null
|| !"ACTIVE".equals(user.getStatus())
|| user.getSessionVersion() != session.version())
throw new BusinessException(401, "UNAUTHORIZED", "登录已失效");
if (!"ADMIN".equals(user.getRole()))
throw new BusinessException(403, "FORBIDDEN", "无后台访问权限");
return user;
}
public AdminView view(User user) {
return new AdminView(
user.getUserCode(), user.getEmail(), user.getNickname(), user.getRole());
}
public void logout(String token) {
tokens.revoke(token);
}
public void password(User user, PasswordInput input) {
validatePassword(input.newPassword());
if (input.oldPassword().getBytes(StandardCharsets.UTF_8).length > 72
|| !encoder.matches(input.oldPassword(), user.getPasswordHash()))
throw new BusinessException(400, "PASSWORD_INVALID", "原密码不正确");
int updated =
users.update(
null,
new LambdaUpdateWrapper<User>()
.eq(User::getId, user.getId())
.eq(User::getSessionVersion, user.getSessionVersion())
.eq(User::getRole, "ADMIN")
.eq(User::getStatus, "ACTIVE")
.set(User::getPasswordHash, encoder.encode(input.newPassword()))
.setSql("session_version = session_version + 1")
.set(User::getUpdatedAt, ProjectSupport.now()));
if (updated != 1) throw BusinessException.conflict("账号状态已变化,请重新登录");
log.info("后台修改密码 userCode={}", user.getUserCode());
}
public void bootstrap(String userCode, String password) {
validatePassword(password);
int updated =
users.update(
null,
new LambdaUpdateWrapper<User>()
.eq(User::getUserCode, userCode)
.eq(User::getRole, "USER")
.eq(User::getStatus, "ACTIVE")
.set(User::getRole, "ADMIN")
.set(User::getPasswordHash, encoder.encode(password))
.setSql("session_version = session_version + 1")
.set(User::getUpdatedAt, ProjectSupport.now()));
if (updated != 1) throw BusinessException.conflict("只能从已有正常普通用户初始化管理员");
log.info("初始化管理员 userCode={}", userCode);
}
private void validatePassword(String password) {
if (password.length() < 12 || password.getBytes(StandardCharsets.UTF_8).length > 72)
throw BusinessException.badRequest("密码至少12字符,UTF-8编码最多72字节");
}
}
@@ -0,0 +1,83 @@
package com.linkerbot.community.adminapi.security;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.linkerbot.community.bean.entity.User;
import com.linkerbot.community.common.*;
import com.linkerbot.community.service.support.ProjectActors;
import jakarta.servlet.FilterChain;
import jakarta.servlet.ServletException;
import jakarta.servlet.http.*;
import lombok.RequiredArgsConstructor;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.security.authentication.UsernamePasswordAuthenticationToken;
import org.springframework.security.core.authority.SimpleGrantedAuthority;
import org.springframework.security.core.context.SecurityContextHolder;
import org.springframework.stereotype.Component;
import org.springframework.web.filter.OncePerRequestFilter;
import java.io.IOException;
import java.util.List;
@Component
@RequiredArgsConstructor
public class AdminTokenFilter extends OncePerRequestFilter {
private static final Logger log = LoggerFactory.getLogger(AdminTokenFilter.class);
private final AdminAuthService auth;
private final ObjectMapper json;
@Override
protected void doFilterInternal(
HttpServletRequest request, HttpServletResponse response, FilterChain chain)
throws ServletException, IOException {
String header = request.getHeader("Authorization");
if (header == null || !header.startsWith("Bearer ")) {
chain.doFilter(request, response);
return;
}
User user;
try {
user = auth.authenticate(header.substring(7));
} catch (BusinessException error) {
response.setStatus(error.status());
response.setContentType("application/json;charset=UTF-8");
json.writeValue(response.getOutputStream(), ApiResponse.failure(error));
return;
}
SecurityContextHolder.getContext()
.setAuthentication(
new UsernamePasswordAuthenticationToken(
user, null, List.of(new SimpleGrantedAuthority("ROLE_ADMIN"))));
long started = System.nanoTime();
try (ProjectActors.Scope scope =
ProjectActors.enter(
new ProjectActors.AdminActor(user.getUserCode(), user.getNickname()))) {
chain.doFilter(request, response);
} finally {
if (!List.of("GET", "HEAD", "OPTIONS").contains(request.getMethod())) {
String targetCode = user.getUserCode();
for (String segment : request.getRequestURI().split("/"))
if (segment.matches("[a-z0-9]{12,32}")) targetCode = segment;
Object pattern =
request.getAttribute(
org.springframework.web.servlet.HandlerMapping
.BEST_MATCHING_PATTERN_ATTRIBUTE);
String resource = pattern instanceof String route ? route : "unknown";
log.info(
"后台操作 userCode={} targetCode={} action={}:{} result={} code=HTTP_{}"
+ " durationMs={}",
user.getUserCode(),
targetCode,
request.getMethod(),
resource,
response.getStatus() < 400 ? "SUCCESS" : "FAILED",
response.getStatus(),
(System.nanoTime() - started) / 1_000_000);
}
SecurityContextHolder.clearContext();
}
}
}
@@ -0,0 +1,18 @@
package com.linkerbot.community.adminapi.security;
import com.linkerbot.community.bean.entity.User;
import com.linkerbot.community.common.BusinessException;
import org.springframework.security.core.context.SecurityContextHolder;
public final class CurrentAdmin {
private CurrentAdmin() {}
public static User required() {
org.springframework.security.core.Authentication authentication =
SecurityContextHolder.getContext().getAuthentication();
if (authentication == null || !(authentication.getPrincipal() instanceof User user))
throw new BusinessException(401, "UNAUTHORIZED", "请登录后台");
return user;
}
}
@@ -0,0 +1,26 @@
spring:
datasource:
url: ${DB_URL}
username: ${DB_USERNAME}
password: ${DB_PASSWORD}
driver-class-name: com.mysql.cj.jdbc.Driver
hikari:
pool-name: CommunityAdminHikariCP
minimum-idle: 1
maximum-pool-size: 20
connection-timeout: 5000
community:
cors:
allowed-origins:
- https://vibemake.robotquan.com
oss:
root-directory: vibemake
gitea:
base-url: ${GITEA_BASE_URL}
owner: ${GITEA_ORGANIZATION:vibemake}
token: ${GITEA_TOKEN}
logging:
level:
com.linkerbot.community: INFO
@@ -0,0 +1,26 @@
spring:
datasource:
url: ${DB_URL:jdbc:mysql://localhost:3306/community?useUnicode=true&characterEncoding=utf8&serverTimezone=Asia/Shanghai}
username: ${DB_USERNAME:root}
password: ${DB_PASSWORD:}
driver-class-name: com.mysql.cj.jdbc.Driver
hikari:
pool-name: CommunityAdminHikariCP
minimum-idle: 1
maximum-pool-size: 10
connection-timeout: 5000
community:
cors:
allowed-origins:
- https://dev-vibemake.robotquan.com
oss:
root-directory: vibemake_dev
gitea:
base-url: ${GITEA_BASE_URL:http://localhost:3000}
owner: ${GITEA_ORGANIZATION:community}
token: ${GITEA_TOKEN:}
logging:
level:
com.linkerbot.community: DEBUG
@@ -0,0 +1,83 @@
server:
port: ${ADMIN_SERVER_PORT:8994}
spring:
application:
name: community-adminapi
profiles:
default: test
sql:
init:
mode: never
servlet:
multipart:
max-file-size: 10MB
max-request-size: 52MB
jackson:
time-zone: UTC
main:
banner-mode: off
mybatis-plus:
configuration:
map-underscore-to-camel-case: true
community:
admin:
allowed-origins: ${ADMIN_ALLOWED_ORIGINS:}
github-token: ${GITHUB_TOKEN:}
worker-enabled: ${ADMIN_IMPORT_WORKER_ENABLED:true}
migration-timeout: ${ADMIN_MIGRATION_TIMEOUT:30m}
import-concurrency: ${ADMIN_IMPORT_CONCURRENCY:2}
oss:
enabled: ${OSS_ENABLED:true}
region: ${OSS_REGION:cn-beijing}
bucket: ${OSS_BUCKET:linkerbot}
access-key-id: ${OSS_ACCESS_KEY_ID:}
access-key-secret: ${OSS_ACCESS_KEY_SECRET:}
public-base-url: ${OSS_PUBLIC_BASE_URL:https://linkerbot.oss-cn-beijing.aliyuncs.com}
legacy-public-base-urls: ${OSS_LEGACY_PUBLIC_BASE_URLS:}
urdf:
max-file-bytes: ${URDF_MAX_FILE_BYTES:52428800}
max-total-bytes: ${URDF_MAX_TOTAL_BYTES:209715200}
max-files: ${URDF_MAX_FILES:500}
ai:
enabled: false
api-key: ${ARK_API_KEY:}
model: ${ARK_MODEL:doubao-seed-2-1-lite-260915}
endpoint: ${ARK_ENDPOINT:https://ark.cn-beijing.volces.com/api/v3/responses}
request-timeout: ${AI_REQUEST_TIMEOUT:30s}
search-timeout: ${AI_SEARCH_TIMEOUT:65s}
heartbeat-interval: 10s
max-concurrent: ${AI_MAX_CONCURRENT:4}
requests-per-minute: ${AI_REQUESTS_PER_MINUTE:10}
types:
GENERAL: 通用
HUMANOID: 人形
QUADRUPED: 四足
WHEELED: 轮式
ROBOTIC_ARM: 机械臂
captcha-ttl: 2m
token-ttl: 8h
max-text-bytes: 1048576
max-file-bytes: 10485760
max-request-bytes: 52428800
gitea:
connect-timeout: 5s
request-timeout: 30s
fork:
git-executable: ${FORK_GIT_EXECUTABLE:git}
timeout: ${FORK_TIMEOUT:5m}
max-concurrent: ${FORK_MAX_CONCURRENT:2}
max-temporary-bytes: ${FORK_MAX_TEMPORARY_BYTES:5368709120}
temporary-directory: ${FORK_TEMPORARY_DIRECTORY:${java.io.tmpdir}/community-fork}
springdoc:
swagger-ui:
path: /swagger-ui.html
logging:
file:
path: ${LOG_PATH:logs}
level:
root: INFO
# 测试环境开启项目 DEBUG 时,仍不输出 SQL 参数或完整请求内容。
com.linkerbot.community.service.mapper: INFO
org.springframework.web: INFO
# 管理端有独立的跨域配置,不复用社区前端来源。
@@ -0,0 +1,589 @@
package com.linkerbot.community.adminapi;
import static org.assertj.core.api.Assertions.*;
import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.*;
import com.fasterxml.jackson.databind.*;
import com.linkerbot.community.adminapi.security.AdminAuthService;
import com.linkerbot.community.bean.entity.*;
import com.linkerbot.community.common.BusinessException;
import com.linkerbot.community.security.TokenService;
import com.linkerbot.community.service.*;
import com.linkerbot.community.service.admin.AdminImportService;
import com.linkerbot.community.service.mapper.*;
import com.linkerbot.community.service.migration.GitHubSourceChecks;
import com.linkerbot.community.service.support.*;
import org.junit.jupiter.api.*;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.autoconfigure.web.servlet.*;
import org.springframework.boot.test.context.*;
import org.springframework.context.annotation.*;
import org.springframework.http.MediaType;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.mock.web.MockMultipartFile;
import org.springframework.test.context.ActiveProfiles;
import org.springframework.test.web.servlet.*;
import org.springframework.test.web.servlet.request.MockHttpServletRequestBuilder;
import java.nio.file.*;
import java.util.*;
import java.util.regex.*;
@SpringBootTest(
properties = {
"spring.datasource.url=jdbc:h2:mem:admin;MODE=MySQL;DB_CLOSE_DELAY=-1",
"spring.datasource.username=sa",
"spring.datasource.password=",
"spring.datasource.driver-class-name=org.h2.Driver",
"community.admin.worker-enabled=false",
"community.oss.enabled=false",
"logging.level.root=WARN"
})
@ActiveProfiles("test")
@AutoConfigureMockMvc(print = MockMvcPrint.NONE)
@Import(AdminWorkflowTest.Fakes.class)
class AdminWorkflowTest {
private static final java.util.concurrent.atomic.AtomicLong SOURCES =
new java.util.concurrent.atomic.AtomicLong(12345);
@TestConfiguration
static class Fakes {
@Bean
@Primary
FakeGiteaClient fakeGitea() {
return new FakeGiteaClient();
}
@Bean
@Primary
GitHubSourceChecks fakeSources() {
return new GitHubSourceChecks("") {
@Override
public Repository repository(String url) {
return new Repository(
SOURCES.get(),
"https://github.com/example/robot",
"master",
"robot",
"robot project",
"MIT");
}
};
}
}
@Autowired MockMvc mvc;
@Autowired JdbcTemplate jdbc;
@Autowired UserMapper users;
@Autowired ProjectMapper projects;
@Autowired ProjectImportTaskMapper tasks;
@Autowired AdminAuthService auth;
@Autowired AdminImportService imports;
@Autowired TokenService tokens;
@Autowired ObjectMapper json;
@Autowired FakeGiteaClient gitea;
@Autowired ProjectService projectService;
@Autowired UserService userService;
@Autowired ProjectLocks locks;
private User admin;
private User ordinary;
private String token;
private static final String PASSWORD = "test-only-password-123";
@BeforeEach
void setup() throws Exception {
jdbc.execute("DROP ALL OBJECTS");
for (String name : List.of("database.md", "admin-database.md")) {
Matcher matcher =
Pattern.compile("CREATE TABLE [\\s\\S]*?;")
.matcher(Files.readString(Path.of("../docs/" + name)));
while (matcher.find())
jdbc.execute(
matcher.group()
.replace(
"JSON NOT NULL DEFAULT (JSON_ARRAY())",
"CLOB NOT NULL DEFAULT '[]'")
.replace(" CHARACTER SET utf8mb4 COLLATE utf8mb4_bin", "")
.replaceAll(" COMMENT '[^']*'", "")
.replaceAll("\\) ENGINE=InnoDB[^;]*;", ");"));
}
SOURCES.incrementAndGet();
gitea.reset();
admin = userService.login(UUID.randomUUID() + "@example.invalid");
ordinary = userService.login(UUID.randomUUID() + "@example.invalid");
auth.bootstrap(admin.getUserCode(), PASSWORD);
admin = users.selectById(admin.getId());
token = login(admin.getEmail(), PASSWORD, 200).path("data").path("token").asText();
}
private JsonNode request(MockHttpServletRequestBuilder request, int status) throws Exception {
MvcResult result = mvc.perform(request).andReturn();
assertThat(result.getResponse().getStatus())
.as(result.getResponse().getContentAsString())
.isEqualTo(status);
return json.readTree(result.getResponse().getContentAsString());
}
private MockHttpServletRequestBuilder body(MockHttpServletRequestBuilder request, Object value)
throws Exception {
return request.contentType(MediaType.APPLICATION_JSON)
.content(json.writeValueAsBytes(value));
}
private MockHttpServletRequestBuilder signed(MockHttpServletRequestBuilder request) {
return request.header("Authorization", "Bearer " + token);
}
private JsonNode login(String email, String password, int status) throws Exception {
return request(
body(post("/admin-api/auth/login"), Map.of("email", email, "password", password)),
status);
}
private Map<String, Object> metadata() {
return Map.of("name", "测试项目", "description", "内容", "projectType", "WHEELED");
}
private String createProject() throws Exception {
return request(
signed(body(post("/admin-api/projects"), Map.of("project", metadata()))),
200)
.path("data")
.path("projectCode")
.asText();
}
private JsonNode createImport() throws Exception {
return request(
signed(
body(
post("/admin-api/project-imports"),
Map.of(
"repositoryUrl",
"https://github.com/example/robot",
"project",
metadata()))),
202)
.path("data");
}
@Test
void authenticationRolesIsolationAndCodeOnlyResponses() throws Exception {
request(get("/admin-api/users"), 401);
request(get("/api/users/me"), 401);
login(ordinary.getEmail(), PASSWORD, 401);
String ordinaryToken = tokens.issue(ordinary.getId(), ordinary.getSessionVersion());
request(get("/admin-api/users").header("Authorization", "Bearer " + ordinaryToken), 403);
assertThatThrownBy(() -> userService.login(admin.getEmail()))
.isInstanceOf(BusinessException.class);
JsonNode me = request(signed(get("/admin-api/auth/me")), 200).path("data");
assertThat(me.path("role").asText()).isEqualTo("ADMIN");
assertThat(me.has("id")).isFalse();
assertThat(me.has("passwordHash")).isFalse();
TokenService anotherProcess =
new TokenService(new com.linkerbot.community.service.config.CommunityProperties());
assertThat(anotherProcess.resolve(token)).isNull();
request(signed(post("/admin-api/auth/logout")), 200);
request(signed(get("/admin-api/auth/me")), 401);
}
@Test
void disableAndEnableInvalidateExistingUserSession() throws Exception {
String old = tokens.issue(ordinary.getId(), 0);
request(
signed(
body(
put("/admin-api/users/" + ordinary.getUserCode() + "/status"),
Map.of("status", "DISABLED"))),
200);
assertThatThrownBy(() -> auth.authenticate(old)).isInstanceOf(BusinessException.class);
request(
signed(
body(
put("/admin-api/users/" + ordinary.getUserCode() + "/status"),
Map.of("status", "ACTIVE"))),
200);
assertThatThrownBy(() -> auth.authenticate(old)).isInstanceOf(BusinessException.class);
assertThat(users.selectById(ordinary.getId()).getSessionVersion()).isEqualTo(2);
request(
signed(
body(
put("/admin-api/users/" + admin.getUserCode() + "/status"),
Map.of("status", "DISABLED"))),
409);
}
@Test
void passwordChangeAndRoleChangeInvalidateSession() throws Exception {
request(
signed(
body(
put("/admin-api/auth/password"),
Map.of(
"oldPassword",
PASSWORD,
"newPassword",
"new-test-password-123"))),
200);
request(signed(get("/admin-api/auth/me")), 401);
login(admin.getEmail(), "new-test-password-123", 200);
jdbc.update(
"UPDATE sys_user SET role='USER', session_version=session_version+1 WHERE"
+ " user_code=?",
admin.getUserCode());
request(signed(get("/admin-api/auth/me")), 401);
}
@Test
void repeatedInvalidPasswordIsRateLimited() throws Exception {
for (int i = 0; i < 5; i++) login(admin.getEmail(), "wrong", 401);
login(admin.getEmail(), PASSWORD, 429);
}
@Test
void officialOwnershipFileUploadPublicationAndModeration() throws Exception {
String code = createProject();
JsonNode detail = request(signed(get("/admin-api/projects/" + code)), 200).path("data");
String ownerCode = detail.path("project").path("owner").path("userCode").asText();
User owner =
users.selectOne(
new com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper<
User>()
.eq(User::getUserCode, ownerCode));
assertThat(owner.getRole()).isEqualTo("OFFICIAL");
assertThatThrownBy(() -> userService.login(OfficialUsers.EMAIL))
.isInstanceOf(BusinessException.class);
request(
signed(
body(
put("/admin-api/users/" + ownerCode + "/status"),
Map.of("status", "DISABLED"))),
409);
String initial =
request(signed(get("/admin-api/projects/" + code + "/tree")), 200)
.path("data")
.path("commitSha")
.asText();
MockMultipartFile file =
new MockMultipartFile(
"files",
"机器人.txt",
"text/plain",
"hello".getBytes(java.nio.charset.StandardCharsets.UTF_8));
String sha =
request(
signed(
multipart("/admin-api/projects/" + code + "/uploads/commit")
.file(file)
.param("expectedCommitSha", initial)
.param("message", "上传文件")
.param("paths", "models/机器人.txt")),
200)
.path("data")
.path("commitSha")
.asText();
assertThat(gitea.lastAuthorCode).isEqualTo(admin.getUserCode());
request(
signed(
body(
put("/admin-api/projects/" + code + "/publication"),
Map.of("commitSha", initial))),
409);
request(
signed(
body(
put("/admin-api/projects/" + code + "/publication"),
Map.of("commitSha", sha))),
200);
assertThat(projectService.published(code).getPublishedCommitSha()).isEqualTo(sha);
request(
signed(
body(
post("/admin-api/projects/" + code + "/commits"),
Map.of(
"expectedCommitSha",
sha,
"message",
"添加文件",
"changes",
List.of(
Map.of(
"action",
"CREATE_FILE",
"path",
"next.txt",
"content",
"next"))))),
200);
assertThat(projectService.published(code).getPublishedCommitSha()).isEqualTo(sha);
request(
signed(
body(
put("/admin-api/projects/" + code + "/moderation"),
Map.of("reason", "测试下架"))),
200);
assertThatThrownBy(() -> projectService.published(code))
.isInstanceOf(BusinessException.class);
JsonNode blocked = request(signed(get("/admin-api/projects/" + code)), 200).path("data");
assertThat(blocked.path("project").path("publishStatus").asText()).isEqualTo("BLOCKED");
assertThat(blocked.path("moderationStatus").asText()).isEqualTo("BLOCKED");
assertThat(blocked.path("moderationReason").asText()).isEqualTo("测试下架");
projectService.withdraw(owner.getId(), code);
assertThat(projectService.owner(owner.getId(), code).getPublishStatus())
.isEqualTo("BLOCKED");
assertThatThrownBy(() -> projectService.publish(owner.getId(), code, sha))
.isInstanceOf(BusinessException.class)
.hasMessageContaining("下架");
request(signed(delete("/admin-api/projects/" + code + "/moderation")), 200);
assertThatThrownBy(() -> projectService.published(code))
.isInstanceOf(BusinessException.class);
request(signed(delete("/admin-api/projects/" + code)), 200);
}
@Test
void migrationPreservesSourceAndRequiresManualPublication() throws Exception {
JsonNode task = createImport();
String taskCode = task.path("taskCode").asText();
String code = task.path("projectCode").asText();
imports.run(taskCode);
JsonNode done =
request(signed(get("/admin-api/project-imports/" + taskCode)), 200).path("data");
assertThat(done.path("status").asText()).isEqualTo("SUCCEEDED");
assertThat(gitea.migrations).isEqualTo(1);
assertThatThrownBy(() -> projectService.published(code))
.isInstanceOf(BusinessException.class);
JsonNode preview =
request(
signed(
body(
post("/admin-api/project-imports/preview"),
Map.of(
"repositoryUrl",
"https://github.com/example/robot"))),
200)
.path("data");
assertThat(preview.path("existingProjectCode").asText()).isEqualTo(code);
request(
signed(
body(
post("/admin-api/project-imports"),
Map.of(
"repositoryUrl",
"https://github.com/example/robot",
"project",
metadata()))),
409);
String sha =
request(signed(get("/admin-api/projects/" + code + "/tree")), 200)
.path("data")
.path("commitSha")
.asText();
JsonNode published =
request(
signed(
body(
put("/admin-api/projects/" + code + "/publication"),
Map.of("commitSha", sha))),
200)
.path("data");
assertThat(published.path("sourceLicense").asText()).isEqualTo("MIT");
assertThat(published.path("sourceRepositoryUrl").asText())
.isEqualTo("https://github.com/example/robot");
ProjectImportTask source =
tasks.selectOne(
new com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper<
ProjectImportTask>()
.eq(ProjectImportTask::getTaskCode, taskCode));
assertThat(source.getSourceRepositoryId()).isEqualTo(SOURCES.get());
assertThat(source.getSourceDefaultBranch()).isEqualTo("master");
assertThat(source.getSourceLicense()).isEqualTo("MIT");
assertThat(published.has("externalRepositoryId")).isFalse();
}
@Test
void uncertainMigrationIsNotRepeatedAfterRestartRecovery() throws Exception {
JsonNode task = createImport();
String code = task.path("taskCode").asText();
gitea.migrationTimeout = true;
imports.run(code);
imports.recover();
assertThat(
request(signed(get("/admin-api/project-imports/" + code)), 200)
.path("data")
.path("status")
.asText())
.isEqualTo("REVIEW");
request(signed(post("/admin-api/project-imports/" + code + "/retry")), 409);
imports.run(code);
assertThat(gitea.migrations).isEqualTo(1);
}
@Test
void completedReceiptCanResumeAfterBranchConflictWithoutRemigrating() throws Exception {
JsonNode task = createImport();
String code = task.path("taskCode").asText();
gitea.mainConflict = true;
imports.run(code);
request(signed(post("/admin-api/project-imports/" + code + "/retry")), 409);
gitea.mainConflict = false;
Project project =
projects.selectOne(
new com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper<
Project>()
.eq(Project::getProjectCode, task.path("projectCode").asText()));
gitea.createBranch(
new com.linkerbot.community.service.gitea.GiteaClient.Repository(
1, "community", project.getGiteaRepoName()),
"main",
"");
com.linkerbot.community.service.gitea.GiteaClient.RemoteRepository remote =
gitea.inspect(project.getGiteaRepoName());
gitea.defaultBranch(remote.repository(), "main");
request(signed(post("/admin-api/project-imports/" + code + "/retry")), 202);
imports.run(code);
assertThat(
request(signed(get("/admin-api/project-imports/" + code)), 200)
.path("data")
.path("status")
.asText())
.isEqualTo("SUCCEEDED");
assertThat(gitea.migrations).isEqualTo(1);
String sha = gitea.head(remote.repository());
request(
signed(
body(
put(
"/admin-api/projects/"
+ project.getProjectCode()
+ "/publication"),
Map.of("commitSha", sha))),
200);
}
@Test
void invalidSourceAndUserCannotEscalateRole() throws Exception {
request(
signed(
body(
post("/admin-api/project-imports/preview"),
Map.of("repositoryUrl", "https://127.0.0.1/private"))),
400);
request(
signed(
body(
patch("/admin-api/users/" + ordinary.getUserCode()),
Map.of("nickname", "新昵称", "role", "ADMIN"))),
200);
assertThat(users.selectById(ordinary.getId()).getRole()).isEqualTo("USER");
}
@Test
void receiptPersistenceFailureCannotBeRetriedAsSuccessfulMigration() throws Exception {
JsonNode task = createImport();
String code = task.path("taskCode").asText();
jdbc.execute("ALTER TABLE project_import_task ALTER COLUMN receipt_json VARCHAR(1)");
imports.run(code);
ProjectImportTask failed =
tasks.selectOne(
new com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper<
ProjectImportTask>()
.eq(ProjectImportTask::getTaskCode, code));
assertThat(failed.getStatus()).isEqualTo("REVIEW");
assertThat(failed.getReceiptJson()).isNull();
assertThat(gitea.migrations).isEqualTo(1);
request(signed(post("/admin-api/project-imports/" + code + "/retry")), 409);
}
@Test
void receiptSurvivesAssociationFailureAndResumesWithoutSecondMigration() throws Exception {
JsonNode task = createImport();
String code = task.path("taskCode").asText();
jdbc.execute(
"ALTER TABLE project ADD CONSTRAINT simulate_link_failure CHECK (repo_status <>"
+ " 'READY')");
imports.run(code);
ProjectImportTask failed =
tasks.selectOne(
new com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper<
ProjectImportTask>()
.eq(ProjectImportTask::getTaskCode, code));
assertThat(failed.getStatus()).isEqualTo("REVIEW");
assertThat(failed.getReceiptJson()).isNotNull();
assertThat(failed.getPhase()).isEqualTo("BRANCH_READY");
assertThat(
jdbc.queryForObject(
"SELECT repo_status FROM project WHERE project_code=?",
String.class,
task.path("projectCode").asText()))
.isEqualTo("CREATING");
jdbc.execute("ALTER TABLE project DROP CONSTRAINT simulate_link_failure");
request(signed(post("/admin-api/project-imports/" + code + "/retry")), 202);
imports.run(code);
assertThat(
request(signed(get("/admin-api/project-imports/" + code)), 200)
.path("data")
.path("status")
.asText())
.isEqualTo("SUCCEEDED");
assertThat(gitea.migrations).isEqualTo(1);
}
@Test
void interruptedMigrationIsReviewedAndCompletedReceiptCanResume() throws Exception {
JsonNode task = createImport();
String code = task.path("taskCode").asText();
String projectCode = task.path("projectCode").asText();
com.linkerbot.community.service.gitea.GiteaClient.RemoteRepository receipt =
gitea.migrate(
"https://github.com/example/robot.git",
"p_" + projectCode,
java.time.Duration.ofMinutes(30));
jdbc.update(
"UPDATE project_import_task SET status='RUNNING',phase='MIGRATED',receipt_json=?"
+ " WHERE task_code=?",
json.writeValueAsString(receipt),
code);
jdbc.update(
"UPDATE project SET repo_write_status='WRITING' WHERE project_code=?", projectCode);
imports.recover();
assertThat(
request(signed(get("/admin-api/project-imports/" + code)), 200)
.path("data")
.path("status")
.asText())
.isEqualTo("REVIEW");
request(signed(post("/admin-api/project-imports/" + code + "/retry")), 202);
imports.run(code);
assertThat(
request(signed(get("/admin-api/project-imports/" + code)), 200)
.path("data")
.path("status")
.asText())
.isEqualTo("SUCCEEDED");
assertThat(gitea.migrations).isEqualTo(1);
}
@Test
void designatedOwnerRemainsSeparateAndAdminScopeIsAlwaysCleared() throws Exception {
String code =
request(
signed(
body(
post("/admin-api/projects"),
Map.of(
"ownerUserCode",
ordinary.getUserCode(),
"project",
metadata()))),
200)
.path("data")
.path("projectCode")
.asText();
assertThat(ProjectActors.isAdmin()).isFalse();
assertThat(projectService.owner(ordinary.getId(), code).getOwnerUserId())
.isEqualTo(ordinary.getId());
assertThatThrownBy(() -> projectService.owner(admin.getId(), code))
.isInstanceOf(BusinessException.class);
}
}
@@ -0,0 +1,222 @@
package com.linkerbot.community.adminapi;
import com.linkerbot.community.bean.Contracts.*;
import com.linkerbot.community.common.PageResponse;
import com.linkerbot.community.service.gitea.*;
import java.security.MessageDigest;
import java.util.*;
import java.util.concurrent.atomic.AtomicLong;
final class FakeGiteaClient implements GiteaClient {
private static final class RepoData {
Repository repository;
String head;
Map<String, Map<String, byte[]>> snapshots = new HashMap<>();
List<HistoryEntry> history = new ArrayList<>();
}
private final Map<Long, RepoData> repositories = new HashMap<>();
private final AtomicLong sequence = new AtomicLong();
String lastAuthorCode;
int migrations;
boolean migrationTimeout;
boolean mainConflict;
private final Map<String, RemoteRepository> migrated = new HashMap<>();
private final Set<String> mainReady = new HashSet<>();
boolean timeoutNextCommit;
boolean timeoutNextCreate;
boolean failNextCreate;
void reset() {
repositories.clear();
migrated.clear();
mainReady.clear();
migrations = 0;
migrationTimeout = false;
mainConflict = false;
timeoutNextCommit = false;
timeoutNextCreate = false;
failNextCreate = false;
}
private String sha() {
return String.format("%040x", sequence.incrementAndGet());
}
private String blob(byte[] data) {
try {
return HexFormat.of().formatHex(MessageDigest.getInstance("SHA-1").digest(data));
} catch (Exception error) {
throw new IllegalStateException(error);
}
}
private RepoData repo(Repository repository) {
return repositories.get(repository.id());
}
synchronized Map<String, byte[]> snapshot(Repository repository, String sha) {
RepoData data = repo(repository);
Map<String, byte[]> snapshot = data == null ? null : data.snapshots.get(sha);
if (snapshot == null) throw new GiteaFailure(404, "NOT_FOUND", "发布版本不存在", false);
Map<String, byte[]> copy = new TreeMap<>();
snapshot.forEach((name, bytes) -> copy.put(name, bytes.clone()));
return copy;
}
@Override
public synchronized Repository createRepository(String name, boolean initialize) {
if (failNextCreate) {
failNextCreate = false;
throw new GiteaFailure(502, "GITEA_ERROR", "服务拒绝请求", false);
}
RepoData data = new RepoData();
data.repository = new Repository(sequence.incrementAndGet(), "community", name);
repositories.put(data.repository.id(), data);
if (initialize)
commit(
data.repository,
List.of(new Change("create", "README.md", null, "initial".getBytes())),
"Initial commit",
"system",
"system");
if (timeoutNextCreate) {
timeoutNextCreate = false;
throw new GiteaFailure(504, "REMOTE_RESULT_UNKNOWN", "创建结果未知", true);
}
return data.repository;
}
@Override
public synchronized String head(Repository repository) {
return repo(repository).head;
}
@Override
public synchronized List<TreeEntry> tree(Repository repository, String sha, String path) {
Map<String, byte[]> snapshot = repo(repository).snapshots.get(sha);
if (snapshot == null) throw new GiteaFailure(404, "NOT_FOUND", "版本不存在", false);
String prefix = path.isEmpty() ? "" : path + "/";
Map<String, TreeEntry> entries = new TreeMap<>();
for (Map.Entry<String, byte[]> file : snapshot.entrySet()) {
if (!file.getKey().startsWith(prefix)) continue;
String remainder = file.getKey().substring(prefix.length());
int separator = remainder.indexOf('/');
String name = separator < 0 ? remainder : remainder.substring(0, separator);
String entryPath = prefix + name;
entries.put(
entryPath,
separator < 0
? new TreeEntry(
entryPath,
name,
"file",
file.getValue().length,
blob(file.getValue()))
: new TreeEntry(entryPath, name, "dir", 0, ""));
}
return new ArrayList<>(entries.values());
}
@Override
public synchronized StoredFile file(Repository repository, String sha, String path) {
Map<String, byte[]> snapshot = repo(repository).snapshots.get(sha);
byte[] bytes = snapshot == null ? null : snapshot.get(path);
if (bytes == null) throw new GiteaFailure(404, "NOT_FOUND", "文件不存在", false);
return new StoredFile(path, blob(bytes), bytes.clone());
}
@Override
public synchronized CommitView commit(
Repository repository,
List<Change> changes,
String message,
String authorName,
String authorCode) {
lastAuthorCode = authorCode;
RepoData repo = repo(repository);
Map<String, byte[]> snapshot =
repo.head == null ? new TreeMap<>() : new TreeMap<>(repo.snapshots.get(repo.head));
for (Change change : changes) {
byte[] old = snapshot.get(change.path());
if (change.operation().equals("create") && old != null)
throw new GiteaFailure(409, "GITEA_CONFLICT", "文件已存在", false);
if (!change.operation().equals("create")
&& (old == null || !blob(old).equals(change.blobSha())))
throw new GiteaFailure(409, "GITEA_CONFLICT", "文件版本冲突", false);
if (change.operation().equals("delete")) snapshot.remove(change.path());
else snapshot.put(change.path(), change.bytes().clone());
}
repo.head = sha();
repo.snapshots.put(repo.head, snapshot);
repo.history.addFirst(
new HistoryEntry(repo.head, message, authorName, "2026-10-13T00:00:00Z"));
if (timeoutNextCommit) {
timeoutNextCommit = false;
throw new GiteaFailure(504, "REMOTE_RESULT_UNKNOWN", "提交结果未知", true);
}
return new CommitView(repo.head, message);
}
@Override
public synchronized PageResponse<HistoryEntry> history(
Repository repository, long page, long size) {
List<HistoryEntry> history = repo(repository).history;
int from = (int) Math.min(history.size(), (page - 1) * size);
int to = (int) Math.min(history.size(), from + size);
return new PageResponse<>(history.subList(from, to), page, size, history.size());
}
@Override
public void checkMigrationSettings() {}
@Override
public synchronized RemoteRepository migrate(
String source, String name, java.time.Duration timeout) {
migrations++;
Repository repository = createRepository(name, true);
RemoteRepository remote =
new RemoteRepository(
repository.id(), "community", name, true, false, false, source, "master");
migrated.put(name, remote);
if (migrationTimeout) throw new GiteaFailure(504, "REMOTE_RESULT_UNKNOWN", "迁移结果不明", true);
return remote;
}
@Override
public synchronized RemoteRepository inspect(String name) {
RemoteRepository remote = migrated.get(name);
if (remote == null) throw new GiteaFailure(404, "NOT_FOUND", "仓库不存在", false);
return remote;
}
@Override
public synchronized String branch(Repository repository, String branch) {
if (branch.equals("main") && mainConflict) return "f".repeat(40);
if (branch.equals("main") && !mainReady.contains(repository.name()))
throw new GiteaFailure(404, "NOT_FOUND", "分支不存在", false);
return head(repository);
}
@Override
public synchronized void createBranch(Repository repository, String name, String sha) {
mainReady.add(repository.name());
}
@Override
public synchronized void defaultBranch(Repository repository, String branch) {
RemoteRepository remote = migrated.get(repository.name());
migrated.put(
repository.name(),
new RemoteRepository(
remote.id(),
remote.owner(),
remote.name(),
true,
false,
false,
remote.originalUrl(),
branch));
}
}
@@ -0,0 +1,202 @@
package com.linkerbot.community.adminapi;
import static org.assertj.core.api.Assertions.*;
import com.baomidou.mybatisplus.extension.spring.MybatisSqlSessionFactoryBean;
import com.linkerbot.community.common.BusinessException;
import com.linkerbot.community.service.mapper.ProjectMapper;
import com.linkerbot.community.service.support.ProjectLocks;
import com.zaxxer.hikari.*;
import org.apache.ibatis.session.SqlSessionFactory;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.condition.EnabledIfEnvironmentVariable;
import org.mybatis.spring.SqlSessionTemplate;
import org.springframework.jdbc.core.JdbcTemplate;
import java.nio.file.*;
import java.sql.*;
import java.util.*;
import java.util.concurrent.*;
import java.util.regex.*;
/** 只在新建的随机临时数据库验证,绝不对连接所指向的现有业务库执行DDL或写入。 */
@EnabledIfEnvironmentVariable(named = "ADMIN_MYSQL_TEST_URL", matches = "jdbc:mysql:.*")
class ProjectLocksMySqlTest {
@Test
void independentApplicationsShareLocksAndPersistUncertainWrites() throws Exception {
String url = System.getenv("ADMIN_MYSQL_TEST_URL");
String username = System.getenv("ADMIN_MYSQL_TEST_USERNAME");
String password = System.getenv("ADMIN_MYSQL_TEST_PASSWORD");
String database = "community_admin_verify_" + UUID.randomUUID().toString().replace("-", "");
try (Connection bootstrap =
DriverManager.getConnection(url, username, password == null ? "" : password)) {
assertThat(bootstrap.getMetaData().getDatabaseProductName()).isEqualTo("MySQL");
try (Statement statement = bootstrap.createStatement()) {
statement.execute(
"CREATE DATABASE "
+ database
+ " CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci");
}
try {
String plain = url.split("\\?", 2)[0];
String isolated =
plain.substring(0, plain.lastIndexOf('/') + 1)
+ database
+ (url.contains("?") ? "?" + url.split("\\?", 2)[1] : "");
HikariConfig config = new HikariConfig();
config.setJdbcUrl(isolated);
config.setUsername(username);
config.setPassword(password);
config.setMaximumPoolSize(4);
try (HikariDataSource source = new HikariDataSource(config)) {
JdbcTemplate jdbc = new JdbcTemplate(source);
for (String doc : List.of("database.md", "admin-database.md")) {
Matcher tables =
Pattern.compile("CREATE TABLE [\\s\\S]*?;")
.matcher(Files.readString(Path.of("../docs/" + doc)));
while (tables.find()) jdbc.execute(tables.group());
}
jdbc.update(
"INSERT INTO"
+ " sys_user(user_code,email,nickname,status,created_at,updated_at)"
+ " VALUES"
+ " ('111111111111','verify@example.invalid','verify','ACTIVE',UTC_TIMESTAMP(3),UTC_TIMESTAMP(3))");
jdbc.update(
"INSERT INTO"
+ " project(project_code,owner_user_id,name,description,project_type,gitea_owner,gitea_repo_name,created_at,updated_at)"
+ " SELECT"
+ " '222222222222',id,'verify','','GENERAL','community','p_222222222222',UTC_TIMESTAMP(3),UTC_TIMESTAMP(3)"
+ " FROM sys_user WHERE user_code='111111111111'");
MybatisSqlSessionFactoryBean factory = new MybatisSqlSessionFactoryBean();
factory.setDataSource(source);
SqlSessionFactory sessions = factory.getObject();
sessions.getConfiguration().addMapper(ProjectMapper.class);
ProjectMapper mapper =
new SqlSessionTemplate(sessions).getMapper(ProjectMapper.class);
ProjectLocks first = new ProjectLocks(source, mapper);
ProjectLocks second = new ProjectLocks(source, mapper);
ExecutorService executor = Executors.newFixedThreadPool(2);
try {
CountDownLatch entered = new CountDownLatch(1);
CountDownLatch release = new CountDownLatch(1);
Future<String> writer =
executor.submit(
() ->
first.write(
"222222222222",
() -> {
entered.countDown();
await(release);
return "done";
}));
assertThat(entered.await(5, TimeUnit.SECONDS)).isTrue();
assertThat(
jdbc.queryForObject(
"SELECT repo_write_status FROM project WHERE"
+ " project_code='222222222222'",
String.class))
.isEqualTo("WRITING");
Future<String> other =
executor.submit(() -> second.serial("222222222222", () -> "other"));
Thread.sleep(100);
assertThat(other.isDone()).isFalse();
release.countDown();
assertThat(writer.get(5, TimeUnit.SECONDS)).isEqualTo("done");
assertThat(other.get(5, TimeUnit.SECONDS)).isEqualTo("other");
assertThat(
first.serial(
"222222222222",
() ->
first.serial(
"222222222222", () -> "reentrant")))
.isEqualTo("reentrant");
assertThatThrownBy(
() ->
first.write(
"222222222222",
() -> {
throw new IllegalStateException(
"simulated remote failure");
}))
.isInstanceOf(IllegalStateException.class);
assertThat(
jdbc.queryForObject(
"SELECT repo_write_status FROM project WHERE"
+ " project_code='222222222222'",
String.class))
.isEqualTo("REVIEW");
first.close();
assertThatThrownBy(() -> second.write("222222222222", () -> "unsafe retry"))
.isInstanceOf(BusinessException.class);
// 模拟进程在写入中退出:只有持久化WRITING,新的进程仍必须阻断。
jdbc.update(
"UPDATE project SET repo_write_status='WRITING' WHERE"
+ " project_code='222222222222'");
ProjectLocks restarted = new ProjectLocks(source, mapper);
try {
assertThatThrownBy(
() ->
restarted.write(
"222222222222", () -> "unsafe restart"))
.isInstanceOf(BusinessException.class);
} finally {
restarted.close();
}
// 断开真正持锁连接,检查不会把未知状态清为空闲。
second.serial(
"222222222222",
() -> {
second.confirmKnownResult("222222222222");
return null;
});
String namespace = Integer.toHexString(database.hashCode());
assertThatThrownBy(
() ->
second.write(
"222222222222",
() -> {
Long connection =
jdbc.queryForObject(
"SELECT"
+ " IS_USED_LOCK(?)",
Long.class,
"community:"
+ namespace
+ ":222222222222");
jdbc.execute(
"KILL CONNECTION "
+ connection);
return "unconfirmed";
}))
.isInstanceOf(BusinessException.class);
assertThat(
jdbc.queryForObject(
"SELECT repo_write_status FROM project WHERE"
+ " project_code='222222222222'",
String.class))
.isEqualTo("REVIEW");
System.out.println("MySQL共享锁、重入、持久化阻断及连接断开验证完成,临时库=" + database);
} finally {
executor.shutdownNow();
first.close();
second.close();
}
}
} finally {
try (Statement statement = bootstrap.createStatement()) {
statement.execute("DROP DATABASE " + database);
}
}
}
}
private void await(CountDownLatch latch) {
try {
if (!latch.await(10, TimeUnit.SECONDS)) throw new IllegalStateException("锁测试超时");
} catch (InterruptedException error) {
Thread.currentThread().interrupt();
throw new IllegalStateException(error);
}
}
}
@@ -7,7 +7,8 @@ import org.springframework.boot.autoconfigure.security.servlet.UserDetailsServic
@MapperScan("com.linkerbot.community.service.mapper")
@SpringBootApplication(
scanBasePackages = "com.linkerbot.community",
scanBasePackages = {"com.linkerbot.community.api", "com.linkerbot.community.service",
"com.linkerbot.community.common", "com.linkerbot.community.security"},
exclude = UserDetailsServiceAutoConfiguration.class)
public class CommunityApplication {
public static void main(String[] args) {
@@ -41,7 +41,7 @@ public class AuthController {
User user = users.login(input.email());
return ApiResponse.success(
new LoginView(
tokens.issue(user.getId()),
tokens.issue(user.getId(), user.getSessionVersion()),
"Bearer",
config.getTokenTtl().toSeconds(),
users.view(user)));
@@ -11,7 +11,10 @@ final class SchemaSupport {
static void reset(JdbcTemplate jdbc) throws IOException {
Path path = Path.of("../docs/database.md");
if (!Files.exists(path)) path = Path.of("docs/database.md");
String content = Files.readString(path);
String content =
Files.readString(path)
+ "\n"
+ Files.readString(path.resolveSibling("admin-database.md"));
Matcher matcher = Pattern.compile("CREATE TABLE [\\s\\S]*?;").matcher(content);
jdbc.execute("DROP ALL OBJECTS");
int tables = 0;
@@ -19,13 +22,15 @@ final class SchemaSupport {
String ddl =
matcher.group()
// H2 的 JSON 字符串绑定与 MySQL 不同,测试以文本列承接服务层 JSON。
.replace("JSON NOT NULL DEFAULT (JSON_ARRAY())", "CLOB NOT NULL DEFAULT '[]'")
.replace(
"JSON NOT NULL DEFAULT (JSON_ARRAY())",
"CLOB NOT NULL DEFAULT '[]'")
.replace(" CHARACTER SET utf8mb4 COLLATE utf8mb4_bin", "")
.replaceAll(" COMMENT '[^']*'", "")
.replaceAll("\\) ENGINE=InnoDB[^;]*;", ");");
jdbc.execute(ddl);
tables++;
}
if (tables != 4) throw new IllegalStateException("必须只有四张业务表");
if (tables != 5) throw new IllegalStateException("必须包含四张基础表和迁移任务表");
}
}
@@ -683,7 +683,7 @@ class WorkflowTest {
}
@Test
void openApiMatchesRoutesAndPublicAuthenticationAndOnlyFourTablesExist() throws Exception {
void openApiMatchesRoutesAndPublicAuthenticationAndRequiredTablesExist() throws Exception {
JsonNode spec = perform(get("/v3/api-docs"), 200);
assertThat(spec.path("paths").has("/api/workspace/projects")).isTrue();
assertThat(spec.path("paths").has("/api/repository-operations/{operationCode}")).isFalse();
@@ -696,7 +696,7 @@ class WorkflowTest {
+ " table_schema='PUBLIC'",
String.class))
.containsExactlyInAnyOrder(
"SYS_USER", "PROJECT", "PROJECT_COMMENT", "PROJECT_LIKE");
"SYS_USER", "PROJECT", "PROJECT_COMMENT", "PROJECT_LIKE", "PROJECT_IMPORT_TASK");
}
@Test
@@ -1112,4 +1112,23 @@ class WorkflowTest {
perform(auth(delete("/api/workspace/projects/" + code + "/publication"), owner), 200);
perform(auth(delete("/api/workspace/projects/" + code), owner), 200);
}
@Test
void disabledReenabledAndPromotedAccountsCannotReuseCommunitySessions() throws Exception {
String email = "role-check@example.invalid";
JsonNode loggedIn = loginData(email);
String token = loggedIn.path("token").asText();
String code = loggedIn.path("user").path("userCode").asText();
perform(auth(get("/api/users/me"), token), 200);
jdbc.update("UPDATE sys_user SET status='DISABLED', session_version=session_version+1 WHERE user_code=?", code);
perform(auth(get("/api/users/me"), token), 401);
jdbc.update("UPDATE sys_user SET status='ACTIVE', session_version=session_version+1 WHERE user_code=?", code);
perform(auth(get("/api/users/me"), token), 401);
String fresh = login(email);
perform(auth(get("/api/users/me"), fresh), 200);
jdbc.update("UPDATE sys_user SET role='ADMIN', session_version=session_version+1 WHERE user_code=?", code);
perform(auth(get("/api/users/me"), fresh), 401);
String captcha = perform(get("/api/auth/captcha"), 200).path("data").path("captchaCode").asText();
perform(body(post("/api/auth/login"), Map.of("email",email,"captchaCode",captcha,"captchaAnswer","ABCDE")),403);
}
}
@@ -0,0 +1,75 @@
package com.linkerbot.community.bean;
import com.linkerbot.community.bean.Contracts.ProjectInput;
import com.linkerbot.community.bean.Contracts.ProjectView;
import jakarta.validation.Valid;
import jakarta.validation.constraints.*;
import java.time.LocalDateTime;
import java.util.List;
public final class AdminContracts {
private AdminContracts() {}
public record LoginInput(
@NotBlank @Email @Size(max = 254) String email,
@NotBlank @Size(max = 128) String password) {}
public record PasswordInput(
@NotBlank @Size(max = 128) String oldPassword,
@NotBlank @Size(min = 12, max = 128) String newPassword) {}
public record AdminView(String userCode, String email, String nickname, String role) {}
public record LoginView(String token, String tokenType, long expiresIn, AdminView admin) {}
public record NicknameInput(@NotBlank @Size(max = 64) String nickname) {}
public record StatusInput(@NotBlank @Pattern(regexp = "ACTIVE|DISABLED") String status) {}
public record ReasonInput(@NotBlank @Size(max = 500) String reason) {}
public record UserView(
String userCode,
String nickname,
String email,
String status,
String role,
long projectCount,
LocalDateTime createdAt) {}
public record CreateProjectInput(String ownerUserCode, @NotNull @Valid ProjectInput project) {}
public record ManagedProjectView(
ProjectView project,
String moderationStatus,
String moderationReason,
boolean deleted) {}
public record PreviewInput(@NotBlank @Size(max = 2048) String repositoryUrl) {}
public record ImportInput(
@NotBlank @Size(max = 2048) String repositoryUrl,
String ownerUserCode,
@NotNull @Valid ProjectInput project) {}
public record ImportPreview(
String repositoryUrl,
String defaultBranch,
String name,
String description,
String license,
String existingProjectCode) {}
public record TaskView(
String taskCode,
String projectCode,
String status,
String phase,
String errorCode,
List<String> warnings,
LocalDateTime createdAt,
LocalDateTime startedAt,
LocalDateTime finishedAt) {}
}
@@ -85,6 +85,8 @@ public final class Contracts {
long likeCount,
long commentCount,
boolean liked,
String sourceRepositoryUrl,
String sourceLicense,
LocalDateTime createdAt,
LocalDateTime updatedAt) {}
@@ -30,4 +30,9 @@ public class Project extends BaseEntity {
private Long sourceProjectId;
private String sourceCommitSha;
private Boolean deleted;
private String moderationReason;
@com.baomidou.mybatisplus.annotation.TableField(
updateStrategy = com.baomidou.mybatisplus.annotation.FieldStrategy.NEVER)
private String repoWriteStatus = "IDLE";
}
@@ -0,0 +1,27 @@
package com.linkerbot.community.bean.entity;
import com.baomidou.mybatisplus.annotation.TableName;
import lombok.Getter;
import lombok.Setter;
@Getter
@Setter
@TableName("project_import_task")
public class ProjectImportTask extends BaseEntity {
private String taskCode;
private String projectCode;
private String operatorUserCode;
private Long sourceRepositoryId;
private String sourceRepositoryUrl;
private String sourceDefaultBranch;
private String sourceLicense;
private String status;
private String phase;
private String receiptJson;
private String targetSha;
private String errorCode;
private String warningsJson;
private java.time.LocalDateTime startedAt;
private java.time.LocalDateTime finishedAt;
}
@@ -13,4 +13,7 @@ public class User extends BaseEntity {
private String email;
private String nickname;
private String status;
private String role = "USER";
private String passwordHash;
private Long sessionVersion = 0L;
}
@@ -234,7 +234,7 @@ public final class ImportApplication {
context.getBean(OssResourcePaths.class),
store,
owner,
options.migrationTimeout());
options.migrationTimeout()).locks(context.getBean(com.linkerbot.community.service.support.ProjectLocks.class));
try (ExecutorService executor =
Executors.newFixedThreadPool(options.concurrency())) {
try {
@@ -8,10 +8,10 @@ import com.linkerbot.community.bean.entity.User;
import com.linkerbot.community.common.BusinessException;
import com.linkerbot.community.service.ProjectService;
import com.linkerbot.community.service.config.CommunityProperties;
import com.linkerbot.community.service.oss.OssResourcePaths;
import com.linkerbot.community.service.gitea.*;
import com.linkerbot.community.service.gitea.GiteaClient.RemoteRepository;
import com.linkerbot.community.service.mapper.ProjectMapper;
import com.linkerbot.community.service.oss.OssResourcePaths;
import com.linkerbot.community.service.support.ProjectInputs;
import org.slf4j.Logger;
@@ -33,6 +33,13 @@ public final class ImportCoordinator {
private final User owner;
private final Duration timeout;
private final String batch;
private com.linkerbot.community.service.support.ProjectLocks sharedLocks =
new com.linkerbot.community.service.support.ProjectLocks();
public ImportCoordinator locks(com.linkerbot.community.service.support.ProjectLocks locks) {
this.sharedLocks = locks;
return this;
}
public static boolean canVerifyNormalization(Entry entry) {
return entry.phase == Phase.REVIEW
@@ -66,6 +73,24 @@ public final class ImportCoordinator {
}
public void run(Entry entry) throws IOException {
try {
com.linkerbot.community.service.support.ProjectLocks shared = sharedLocks;
shared.serial(
entry.projectCode,
() -> {
try {
runLocked(entry);
} catch (IOException error) {
throw new java.io.UncheckedIOException(error);
}
return null;
});
} catch (java.io.UncheckedIOException error) {
throw error.getCause();
}
}
private void runLocked(Entry entry) throws IOException {
long started = System.nanoTime();
Project project = null;
boolean owned = false;
@@ -169,56 +194,67 @@ public final class ImportCoordinator {
});
return;
}
sharedLocks.confirmKnownResult(entry.projectCode);
store.phase(entry, Phase.BRANCH_READY);
log.info("分支适配结果已核对 batch={} projectCode={}", batch, entry.projectCode);
}
if (entry.receipt == null) {
ensureAbsent(repoName);
store.phase(entry, Phase.MIGRATING);
log.info(
"原生迁移开始 batch={} projectCode={} source={}",
batch,
entry.projectCode,
entry.source.url());
RemoteRepository receipt =
gitea.migrate(entry.source.url() + ".git", repoName, timeout);
// 收据必须先持久化;之后数据库或主分支处理失败可安全续跑。
store.update(
entry,
e -> {
e.receipt = receipt;
e.phase = Phase.MIGRATED;
});
log.info("原生迁移完成 batch={} projectCode={}", batch, entry.projectCode);
}
RemoteRepository remote = gitea.inspect(repoName);
validateRemote(entry, remote);
if (remote.id() != entry.receipt.id())
throw new ImportFailure("REPOSITORY_REPLACED", "仓库标识与迁移收据不一致");
store.phase(entry, Phase.NORMALIZING);
String sourceSha = gitea.branch(remote.repository(), entry.source.defaultBranch());
if (entry.targetSha != null && !entry.targetSha.equals(sourceSha))
throw new ImportFailure("SOURCE_BRANCH_CHANGED", "目标仓库的源分支已变化,禁止自动改写冻结版本");
if (entry.targetSha == null) store.update(entry, e -> e.targetSha = sourceSha);
String mainSha;
com.linkerbot.community.service.migration.RepositoryMigration engine =
new com.linkerbot.community.service.migration.RepositoryMigration(
gitea, config.getGitea().getOwner());
com.linkerbot.community.service.migration.RepositoryMigration.Result migrated;
try {
mainSha = gitea.branch(remote.repository(), "main");
} catch (GiteaFailure missing) {
if (missing.status() != 404) throw missing;
gitea.createBranch(remote.repository(), "main", sourceSha);
mainSha = gitea.branch(remote.repository(), "main");
migrated =
sharedLocks.write(
entry.projectCode,
() -> {
try {
return engine.run(
entry.source.url(),
entry.candidate.repositoryUrl(),
entry.source.defaultBranch(),
repoName,
timeout,
new com.linkerbot.community.service.migration
.RepositoryMigration.Progress() {
public RemoteRepository receipt() {
return entry.receipt;
}
public String targetSha() {
return entry.targetSha;
}
public void receipt(RemoteRepository receipt)
throws IOException {
store.update(
entry,
e -> {
e.receipt = receipt;
e.phase = Phase.MIGRATED;
});
}
public void targetSha(String sha)
throws IOException {
store.update(entry, e -> e.targetSha = sha);
}
public void phase(String phase)
throws IOException {
store.phase(entry, Phase.valueOf(phase));
}
});
} catch (IOException error) {
throw new java.io.UncheckedIOException(error);
}
});
} catch (java.io.UncheckedIOException error) {
throw error.getCause();
}
if (!sourceSha.equals(mainSha))
throw new ImportFailure("MAIN_BRANCH_CONFLICT", "main 与源默认分支内容不同,禁止覆盖");
if (!"main".equals(remote.defaultBranch()))
gitea.defaultBranch(remote.repository(), "main");
store.update(
entry,
e -> {
e.targetSha = sourceSha;
e.phase = Phase.BRANCH_READY;
});
RemoteRepository remote = migrated.remote();
String sourceSha = migrated.sha();
project.setGiteaRepoId(remote.id());
project.setRepoStatus("READY");
@@ -1,206 +1,41 @@
package com.linkerbot.community.importer;
import static com.linkerbot.community.importer.ImportModels.*;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import java.io.InputStream;
import java.net.URI;
import java.net.http.*;
import java.time.Duration;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import com.linkerbot.community.service.migration.GitHubSourceChecks;
import com.linkerbot.community.service.migration.MigrationFailure;
/** 兼容运维导入的模型与异常类型,网络检查复用通用实现。 */
public class SourceChecks {
private final HttpClient http =
HttpClient.newBuilder()
.connectTimeout(Duration.ofSeconds(10))
.followRedirects(HttpClient.Redirect.NEVER)
.build();
private final ObjectMapper json = new ObjectMapper();
private final String token;
private boolean apiLimited;
private final GitHubSourceChecks delegate;
public SourceChecks(String token) {
this.token = token == null ? "" : token;
delegate = new GitHubSourceChecks(token);
}
public GitHubRepository repository(String url) throws Exception {
if (apiLimited) return publicPage(url);
URI uri = URI.create("https://api.github.com/repos/" + WorkbookReader.repositoryKey(url));
for (int redirects = 0; redirects <= 5; redirects++) {
if (!"https".equalsIgnoreCase(uri.getScheme())
|| !"api.github.com".equalsIgnoreCase(uri.getHost())
|| uri.getUserInfo() != null
|| (uri.getPort() != -1 && uri.getPort() != 443))
throw new ImportFailure("GITHUB_REDIRECT", "GitHub API 跳转地址不符合要求");
HttpRequest.Builder request =
HttpRequest.newBuilder(uri)
.timeout(Duration.ofSeconds(30))
.header("Accept", "application/vnd.github+json")
.header("User-Agent", "community-import/1");
if (!token.isBlank()) request.header("Authorization", "Bearer " + token);
HttpResponse<String> response =
http.send(request.GET().build(), HttpResponse.BodyHandlers.ofString());
int status = response.statusCode();
if (status >= 300 && status < 400) {
uri = redirect(uri, response);
continue;
}
if (status == 403 || status == 429) {
apiLimited = true;
return publicPage(url);
}
if (status == 404) throw new ImportFailure("GITHUB_NOT_FOUND", "GitHub 仓库不可访问");
if (status != 200) throw new ImportFailure("GITHUB_HTTP_" + status, "GitHub 返回异常状态");
JsonNode data = json.readTree(response.body());
if (data.path("id").asLong() <= 0
|| data.path("private").asBoolean()
|| data.path("default_branch").asText().isBlank()
|| data.path("size").asLong(-1) < 0)
throw new ImportFailure("GITHUB_INVALID_REPOSITORY", "仓库元数据无效或不是公开仓库");
String canonical =
"https://github.com/"
+ WorkbookReader.repositoryKey(data.path("html_url").asText());
return new GitHubRepository(
data.path("id").asLong(), canonical, data.path("default_branch").asText());
public ImportModels.GitHubRepository repository(String url) throws Exception {
try {
return convert(delegate.repository(url));
} catch (MigrationFailure error) {
throw new ImportFailure(error.code(), error.getMessage());
}
throw new ImportFailure("GITHUB_REDIRECT", "GitHub 跳转次数过多");
}
private GitHubRepository publicPage(String url) throws Exception {
URI uri = URI.create("https://github.com/" + WorkbookReader.repositoryKey(url));
for (int redirects = 0; redirects <= 5; redirects++) {
if (!"https".equalsIgnoreCase(uri.getScheme())
|| !"github.com".equalsIgnoreCase(uri.getHost())
|| uri.getUserInfo() != null
|| (uri.getPort() != -1 && uri.getPort() != 443))
throw new ImportFailure("GITHUB_REDIRECT", "GitHub 网页跳转地址不符合要求");
HttpRequest request =
HttpRequest.newBuilder(uri)
.timeout(Duration.ofSeconds(30))
.header("Accept", "text/html")
.header("User-Agent", "community-import/1")
.GET()
.build();
// 网页请求不携带任何 GitHub 或 Gitea 凭据。
HttpResponse<String> response =
http.send(request, HttpResponse.BodyHandlers.ofString());
int status = response.statusCode();
if (status >= 300 && status < 400) {
uri = redirect(uri, response);
continue;
}
if (status == 403 || status == 429)
throw new ImportFailure("GITHUB_ACCESS_OR_RATE_LIMIT", "GitHub API 及公开网页均不可访问");
if (status != 200)
throw new ImportFailure("GITHUB_PAGE_HTTP_" + status, "GitHub 公开仓库网页不可访问");
return parsePublicPage(response.body(), uri.toString());
}
throw new ImportFailure("GITHUB_REDIRECT", "GitHub 网页跳转次数过多");
}
static GitHubRepository parsePublicPage(String html, String url) throws Exception {
String key = WorkbookReader.repositoryKey(url);
String id = pageMeta(html, "repository_id");
if (!id.matches("[1-9][0-9]*")
|| !key.equalsIgnoreCase(pageMeta(html, "repository_nwo"))
|| !"true".equals(pageMeta(html, "repository_public")))
throw new ImportFailure("GITHUB_PAGE_METADATA", "GitHub 网页仓库身份或公开状态无法核验");
Matcher scripts =
Pattern.compile(
"<script[^>]*data-target=\"react-app.embeddedData\"[^>]*>(.*?)</script>",
Pattern.DOTALL)
.matcher(html);
while (scripts.find()) {
JsonNode repo =
new ObjectMapper()
.readTree(scripts.group(1))
.path("payload")
.path("codeViewLayoutRoute")
.path("repo");
if (repo.path("id").asLong() != Long.parseLong(id)) continue;
String name = repo.path("ownerLogin").asText() + "/" + repo.path("name").asText();
if (!name.equalsIgnoreCase(key)
|| !repo.path("public").asBoolean()
|| !repo.path("private").isBoolean()
|| repo.path("private").asBoolean()
|| !repo.path("isEmpty").isBoolean()
|| repo.path("isEmpty").asBoolean()
|| repo.path("defaultBranch").asText().isBlank()) break;
return new GitHubRepository(
Long.parseLong(id),
"https://github.com/" + name,
repo.path("defaultBranch").asText());
}
throw new ImportFailure("GITHUB_PAGE_METADATA", "GitHub 网页缺少一致的仓库 ID、公开状态或默认分支");
}
private static String pageMeta(String html, String field) {
Matcher meta =
Pattern.compile(
"<meta name=\"octolytics-dimension-"
+ field
+ "\" content=\"([^\"]*)\"[^>]*>")
.matcher(html);
return meta.find() ? meta.group(1) : "";
}
public void cover(String url) throws Exception {
URI uri = URI.create(url);
for (int redirects = 0; redirects <= 5; redirects++) {
if (!"https".equalsIgnoreCase(uri.getScheme())
|| uri.getHost() == null
|| uri.getUserInfo() != null)
throw new ImportFailure("COVER_REDIRECT", "封面跳转地址必须为 HTTPS");
HttpRequest request =
HttpRequest.newBuilder(uri)
.timeout(Duration.ofSeconds(30))
.header("User-Agent", "community-import/1")
.header("Range", "bytes=0-511")
.GET()
.build();
HttpResponse<InputStream> response =
http.send(request, HttpResponse.BodyHandlers.ofInputStream());
try (InputStream body = response.body()) {
int status = response.statusCode();
if (status >= 300 && status < 400) {
uri = redirect(uri, response);
continue;
}
if (status != 200 && status != 206)
throw new ImportFailure("COVER_HTTP_" + status, "封面链接无法读取");
byte[] signature = body.readNBytes(16);
boolean image =
signature.length >= 3
&& ((signature[0] & 255) == 255 && (signature[1] & 255) == 216
|| signature.length >= 8
&& (signature[0] & 255) == 137
&& signature[1] == 'P'
&& signature[2] == 'N'
&& signature[3] == 'G'
|| signature[0] == 'G'
&& signature[1] == 'I'
&& signature[2] == 'F'
|| signature.length >= 12
&& signature[0] == 'R'
&& signature[1] == 'I'
&& signature[8] == 'W'
&& signature[9] == 'E');
if (!image)
throw new ImportFailure("COVER_NOT_IMAGE", "封面未返回 JPG、PNG、GIF 或 WebP 图片内容");
return;
}
try {
delegate.cover(url);
} catch (MigrationFailure error) {
throw new ImportFailure(error.code(), error.getMessage());
}
throw new ImportFailure("COVER_REDIRECT", "封面跳转次数过多");
}
private URI redirect(URI uri, HttpResponse<?> response) {
return uri.resolve(
response.headers()
.firstValue("Location")
.orElseThrow(
() -> new ImportFailure("INVALID_REDIRECT", "跳转响应缺少 Location")));
static ImportModels.GitHubRepository parsePublicPage(String html, String url) throws Exception {
try {
return convert(GitHubSourceChecks.parsePublicPage(html, url));
} catch (MigrationFailure error) {
throw new ImportFailure(error.code(), error.getMessage());
}
}
private static ImportModels.GitHubRepository convert(GitHubSourceChecks.Repository source) {
return new ImportModels.GitHubRepository(source.id(), source.url(), source.defaultBranch());
}
}
@@ -238,24 +238,6 @@ public final class WorkbookReader {
}
public static String repositoryKey(String url) {
if (url == null) throw new IllegalArgumentException("仓库地址为空");
URI uri = URI.create(url.trim());
String path = uri.getPath() == null ? "" : uri.getPath().replaceAll("/$", "");
if (!"https".equalsIgnoreCase(uri.getScheme())
|| !"github.com".equalsIgnoreCase(uri.getHost())
|| uri.getUserInfo() != null
|| (uri.getPort() != -1 && uri.getPort() != 443)
|| uri.getQuery() != null
|| uri.getFragment() != null
|| !path.matches("/[A-Za-z0-9_.-]+/[A-Za-z0-9_.-]+"))
throw new IllegalArgumentException("必须是 GitHub 仓库根地址");
if (path.endsWith(".git")) path = path.substring(0, path.length() - 4);
String[] parts = path.substring(1).split("/");
if (parts.length != 2
|| parts[0].equals(".")
|| parts[0].equals("..")
|| parts[1].equals(".")
|| parts[1].equals("..")) throw new IllegalArgumentException("仓库地址不合法");
return path.substring(1).toLowerCase(Locale.ROOT);
return com.linkerbot.community.service.migration.GitHubSourceChecks.repositoryKey(url);
}
}
@@ -11,11 +11,11 @@ import com.linkerbot.community.bean.entity.User;
import com.linkerbot.community.common.*;
import com.linkerbot.community.service.ProjectService;
import com.linkerbot.community.service.config.CommunityProperties;
import com.linkerbot.community.service.oss.OssResourcePaths;
import com.linkerbot.community.service.oss.OssObjectStore;
import com.linkerbot.community.service.config.OssProperties;
import com.linkerbot.community.service.gitea.*;
import com.linkerbot.community.service.mapper.*;
import com.linkerbot.community.service.oss.OssObjectStore;
import com.linkerbot.community.service.oss.OssResourcePaths;
import org.junit.jupiter.api.*;
import org.junit.jupiter.api.io.TempDir;
@@ -73,12 +73,19 @@ class ImportCoordinatorTest {
void prepare() throws Exception {
Path schema = Path.of("../docs/database.md");
Matcher tables =
Pattern.compile("CREATE TABLE [\\s\\S]*?;").matcher(Files.readString(schema));
Pattern.compile("CREATE TABLE [\\s\\S]*?;")
.matcher(
Files.readString(schema)
+ "\n"
+ Files.readString(
schema.resolveSibling("admin-database.md")));
jdbc.execute("DROP ALL OBJECTS");
while (tables.find())
jdbc.execute(
tables.group()
.replace("JSON NOT NULL DEFAULT (JSON_ARRAY())", "CLOB NOT NULL DEFAULT '[]'")
.replace(
"JSON NOT NULL DEFAULT (JSON_ARRAY())",
"CLOB NOT NULL DEFAULT '[]'")
.replace(" CHARACTER SET utf8mb4 COLLATE utf8mb4_bin", "")
.replaceAll(" COMMENT '[^']*'", "")
.replaceAll("\\) ENGINE=InnoDB[^;]*;", ");"));
@@ -161,7 +168,14 @@ class ImportCoordinatorTest {
private ImportCoordinator worker(ProjectMapper mapper, ProjectService publisher) {
return new ImportCoordinator(
mapper, publisher, gitea, config, resourcePaths, store, owner, Duration.ofMinutes(30));
mapper,
publisher,
gitea,
config,
resourcePaths,
store,
owner,
Duration.ofMinutes(30));
}
@Test
@@ -188,8 +202,15 @@ class ImportCoordinatorTest {
void hostedImportCoversNormalizeWithoutOssCredentials() throws Exception {
String path = "/project-covers/111111111111/20261019/a.png";
Candidate original = entry.candidate;
entry.candidate = new Candidate(original.sheet(), original.row(), original.name(), original.repositoryUrl(),
original.description(), original.projectType(), oss.publicBaseUrl() + "/" + oss.getRootDirectory() + path);
entry.candidate =
new Candidate(
original.sheet(),
original.row(),
original.name(),
original.repositoryUrl(),
original.description(),
original.projectType(),
oss.publicBaseUrl() + "/" + oss.getRootDirectory() + path);
worker(projects, service).run(entry);
assertThat(entry.phase).isEqualTo(Phase.PUBLISHED);
Project project = service.published(entry.projectCode);
@@ -36,9 +36,12 @@ public class BearerTokenFilter extends OncePerRequestFilter {
String header = request.getHeader("Authorization");
if (header != null && header.startsWith("Bearer ")) {
try {
Long id = tokens.resolve(header.substring(7));
TokenService.Session session = tokens.session(header.substring(7));
Long id = session == null ? null : session.userId();
if (id == null) throw new BusinessException(401, "UNAUTHORIZED", "登录已失效,请重新登录");
users.require(id);
com.linkerbot.community.bean.entity.User user = users.require(id);
if (!"USER".equals(user.getRole()) || user.getSessionVersion() != session.version())
throw new BusinessException(401, "UNAUTHORIZED", "登录已失效,请重新登录");
UsernamePasswordAuthenticationToken auth =
new UsernamePasswordAuthenticationToken(
new CurrentUser.Principal(id),
@@ -11,7 +11,7 @@ import java.util.Base64;
@Service
public class TokenService {
private final Cache<String, Long> sessions;
private final Cache<String, Session> sessions;
private final SecureRandom random = new SecureRandom();
public TokenService(CommunityProperties config) {
@@ -22,18 +22,25 @@ public class TokenService {
.build();
}
public String issue(long userId) {
public record Session(long userId, long version) {}
public String issue(long userId) { return issue(userId, 0L); }
public String issue(long userId, long version) {
byte[] bytes = new byte[32];
random.nextBytes(bytes);
String token = Base64.getUrlEncoder().withoutPadding().encodeToString(bytes);
sessions.put(token, userId);
sessions.put(token, new Session(userId, version));
return token;
}
public Long resolve(String token) {
return sessions.getIfPresent(token);
Session session = session(token);
return session == null ? null : session.userId();
}
public Session session(String token) { return sessions.getIfPresent(token); }
public void revoke(String token) {
if (token != null) sessions.invalidate(token);
}
@@ -0,0 +1,527 @@
package com.linkerbot.community.service.admin;
import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper;
import com.baomidou.mybatisplus.core.conditions.update.LambdaUpdateWrapper;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.linkerbot.community.bean.AdminContracts.*;
import com.linkerbot.community.bean.entity.*;
import com.linkerbot.community.common.*;
import com.linkerbot.community.service.config.*;
import com.linkerbot.community.service.gitea.*;
import com.linkerbot.community.service.mapper.*;
import com.linkerbot.community.service.migration.*;
import com.linkerbot.community.service.oss.OssResourcePaths;
import com.linkerbot.community.service.support.*;
import lombok.RequiredArgsConstructor;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.dao.DuplicateKeyException;
import org.springframework.stereotype.Service;
import org.springframework.transaction.support.TransactionTemplate;
import java.io.IOException;
import java.util.*;
@Service
@RequiredArgsConstructor
public class AdminImportService {
private static final Logger log = LoggerFactory.getLogger(AdminImportService.class);
private final AdminManagementService management;
private final GitHubSourceChecks sources;
private final ProjectMapper projects;
private final ProjectImportTaskMapper tasks;
private final UserMapper users;
private final GiteaClient gitea;
private final CommunityProperties properties;
private final AdminProperties settings;
private final ProjectLocks locks;
private final OssResourcePaths paths;
private final ObjectMapper json;
private final TransactionTemplate transactions;
public ImportPreview preview(String url) {
management.requireAdmin();
GitHubSourceChecks.Repository source = check(url);
Project existing = existing(source.id());
return new ImportPreview(
source.url(),
source.defaultBranch(),
source.name(),
source.description(),
source.license(),
existing == null ? null : existing.getProjectCode());
}
private GitHubSourceChecks.Repository check(String url) {
try {
GitHubSourceChecks.repositoryKey(url);
return sources.repository(url);
} catch (BusinessException error) {
throw error;
} catch (IllegalArgumentException error) {
throw BusinessException.badRequest("必须填写HTTPS公开GitHub仓库根地址");
} catch (Exception error) {
if (error instanceof InterruptedException) Thread.currentThread().interrupt();
log.warn(
"GitHub预检失败 code=GITHUB_UNAVAILABLE errorType={}",
error.getClass().getSimpleName());
throw new BusinessException(502, "GITHUB_UNAVAILABLE", "GitHub预检未完成,请稍后重试");
}
}
private Project existing(long sourceId) {
ProjectImportTask task =
tasks.selectOne(
new LambdaQueryWrapper<ProjectImportTask>()
.eq(ProjectImportTask::getSourceRepositoryId, sourceId));
String code = task == null ? ProjectCodes.github(sourceId) : task.getProjectCode();
return projects.selectOne(
new LambdaQueryWrapper<Project>().eq(Project::getProjectCode, code));
}
public TaskView create(ImportInput input) {
management.requireAdmin();
ProjectInputs.validate(input.project(), properties);
GitHubSourceChecks.Repository source = check(input.repositoryUrl());
User owner = management.owner(input.ownerUserCode());
String code = ProjectCodes.github(source.id());
return locks.serial(
code,
() -> {
Project found = existing(source.id());
if (found != null)
throw new BusinessException(
409,
"DUPLICATE_SOURCE",
"该来源已存在项目,请检查现有项目",
Map.of("projectCode", found.getProjectCode()));
try {
return transactions.execute(
status -> {
Project project = new Project();
project.setProjectCode(code);
project.setOwnerUserId(owner.getId());
project.setName(input.project().name().trim());
project.setDescription(
input.project().description() == null
? ""
: input.project().description());
project.setCoverUrl(
paths.coverStorage(input.project().coverUrl()));
project.setProjectType(input.project().projectType());
project.setGiteaOwner(properties.getGitea().getOwner());
project.setGiteaRepoName("p_" + code);
project.setRepoStatus("CREATING");
project.setPublishStatus("UNPUBLISHED");
project.setDeleted(false);
project.setCreatedAt(ProjectSupport.now());
project.setUpdatedAt(project.getCreatedAt());
projects.insert(project);
ProjectImportTask task = new ProjectImportTask();
task.setTaskCode(Codes.next());
task.setProjectCode(code);
task.setSourceRepositoryId(source.id());
task.setSourceRepositoryUrl(source.url());
task.setSourceDefaultBranch(source.defaultBranch());
task.setSourceLicense(source.license());
task.setOperatorUserCode(ProjectActors.admin().userCode());
task.setStatus("QUEUED");
task.setPhase("QUEUED");
task.setWarningsJson("[]");
task.setCreatedAt(ProjectSupport.now());
task.setUpdatedAt(task.getCreatedAt());
tasks.insert(task);
log.info(
"创建迁移任务 userCode={} taskCode={} projectCode={}",
task.getOperatorUserCode(),
task.getTaskCode(),
code);
return view(task);
});
} catch (DuplicateKeyException duplicate) {
throw new BusinessException(
409,
"DUPLICATE_SOURCE",
"来源项目或编码已被占用,请刷新核对",
Map.of("projectCode", code));
}
});
}
public ProjectImportTask task(String code) {
management.requireAdmin();
ProjectImportTask task = find(code);
if (task == null) throw BusinessException.notFound();
return task;
}
private ProjectImportTask find(String code) {
return tasks.selectOne(
new LambdaQueryWrapper<ProjectImportTask>()
.eq(ProjectImportTask::getTaskCode, code));
}
public TaskView view(ProjectImportTask task) {
try {
List<String> warnings =
json.readValue(
task.getWarningsJson() == null ? "[]" : task.getWarningsJson(),
new TypeReference<List<String>>() {});
return new TaskView(
task.getTaskCode(),
task.getProjectCode(),
task.getStatus(),
task.getPhase(),
task.getErrorCode(),
warnings,
task.getCreatedAt(),
task.getStartedAt(),
task.getFinishedAt());
} catch (IOException error) {
throw new IllegalStateException("迁移任务告警数据无效");
}
}
public PageResponse<TaskView> list(long page, long size, String status, String projectCode) {
management.requireAdmin();
ProjectSupport.pageCheck(page, size);
Page<ProjectImportTask> result =
tasks.selectPage(
new Page<>(page, size),
new LambdaQueryWrapper<ProjectImportTask>()
.eq(status != null, ProjectImportTask::getStatus, status)
.eq(
projectCode != null,
ProjectImportTask::getProjectCode,
projectCode)
.orderByDesc(ProjectImportTask::getCreatedAt)
.orderByDesc(ProjectImportTask::getId));
return new PageResponse<>(
result.getRecords().stream().map(this::view).toList(),
page,
size,
result.getTotal());
}
public TaskView retry(String code) {
ProjectImportTask initial = task(code);
return locks.serial(
initial.getProjectCode(),
() -> {
ProjectImportTask task = task(code);
Project project =
projects.selectOne(
new LambdaQueryWrapper<Project>()
.eq(Project::getProjectCode, task.getProjectCode())
.eq(Project::getDeleted, false));
if (project == null || !"UNPUBLISHED".equals(project.getPublishStatus()))
throw BusinessException.conflict("项目已删除、发布或下架,不能重试迁移");
if ("REVIEW".equals(task.getStatus())) {
if (task.getReceiptJson() == null)
throw BusinessException.conflict("迁移结果不明,需要运维核对,禁止重发");
if ("MIGRATED".equals(task.getPhase())) {
engine().verifyReceipt(
task.getSourceRepositoryUrl(),
project.getGiteaRepoName(),
receipt(task));
} else if (Set.of("NORMALIZING", "BRANCH_READY").contains(task.getPhase())
&& task.getTargetSha() != null) {
engine().verifyNormalized(
task.getSourceRepositoryUrl(),
task.getSourceDefaultBranch(),
project.getGiteaRepoName(),
receipt(task),
task.getTargetSha());
task.setPhase("BRANCH_READY");
} else throw BusinessException.conflict("迁移结果需要运维核对,禁止重发");
locks.confirmKnownResult(project.getProjectCode());
} else if (!"FAILED".equals(task.getStatus()))
throw BusinessException.conflict("仅失败或可核验的待核对任务允许续跑");
if (!"IDLE".equals(project.getRepoWriteStatus())
&& !"REVIEW".equals(task.getStatus()))
throw BusinessException.conflict("仓库写入状态需要人工核对");
tasks.update(
null,
new LambdaUpdateWrapper<ProjectImportTask>()
.eq(ProjectImportTask::getId, task.getId())
.set(ProjectImportTask::getStatus, "QUEUED")
.set(ProjectImportTask::getPhase, task.getPhase())
.set(ProjectImportTask::getErrorCode, null)
.set(ProjectImportTask::getFinishedAt, null)
.set(ProjectImportTask::getUpdatedAt, ProjectSupport.now()));
return view(task(code));
});
}
public List<String> queued() {
return tasks
.selectList(
new LambdaQueryWrapper<ProjectImportTask>()
.eq(ProjectImportTask::getStatus, "QUEUED")
.orderByAsc(ProjectImportTask::getCreatedAt)
.last("LIMIT " + settings.getImportConcurrency()))
.stream()
.map(ProjectImportTask::getTaskCode)
.toList();
}
public void recover() {
// 单实例管理端重启后,远端阶段一律待核对;只读预检阶段可以重新排队。
tasks.update(
null,
new LambdaUpdateWrapper<ProjectImportTask>()
.eq(ProjectImportTask::getStatus, "RUNNING")
.in(ProjectImportTask::getPhase, List.of("QUEUED", "PRECHECK"))
.set(ProjectImportTask::getStatus, "QUEUED")
.set(ProjectImportTask::getUpdatedAt, ProjectSupport.now()));
tasks.update(
null,
new LambdaUpdateWrapper<ProjectImportTask>()
.eq(ProjectImportTask::getStatus, "RUNNING")
.set(ProjectImportTask::getStatus, "REVIEW")
.set(ProjectImportTask::getErrorCode, "PROCESS_INTERRUPTED")
.set(ProjectImportTask::getFinishedAt, ProjectSupport.now())
.set(ProjectImportTask::getUpdatedAt, ProjectSupport.now()));
}
public void run(String code) {
int claimed =
tasks.update(
null,
new LambdaUpdateWrapper<ProjectImportTask>()
.eq(ProjectImportTask::getTaskCode, code)
.eq(ProjectImportTask::getStatus, "QUEUED")
.set(ProjectImportTask::getStatus, "RUNNING")
.set(ProjectImportTask::getStartedAt, ProjectSupport.now())
.set(ProjectImportTask::getUpdatedAt, ProjectSupport.now()));
if (claimed != 1) return;
ProjectImportTask task = find(code);
try {
locks.serial(
task.getProjectCode(),
() -> {
try {
process(task);
} catch (RuntimeException error) {
fail(code, error);
}
return null;
});
} catch (RuntimeException error) {
fail(code, error);
}
}
private void process(ProjectImportTask task) {
Project project =
projects.selectOne(
new LambdaQueryWrapper<Project>()
.eq(Project::getProjectCode, task.getProjectCode())
.eq(Project::getDeleted, false));
if (project == null
|| !"CREATING".equals(project.getRepoStatus())
&& !"FAILED".equals(project.getRepoStatus()))
throw BusinessException.conflict("项目状态已变化,停止迁移");
User operator =
users.selectOne(
new LambdaQueryWrapper<User>()
.eq(User::getUserCode, task.getOperatorUserCode()));
if (operator == null
|| !"ADMIN".equals(operator.getRole())
|| !"ACTIVE".equals(operator.getStatus()))
throw new BusinessException(403, "OPERATOR_UNAVAILABLE", "任务创建人已无后台权限");
try {
RepositoryMigration.Result result;
if ("BRANCH_READY".equals(task.getPhase()) && task.getReceiptJson() != null) {
result =
engine().verifyNormalized(
task.getSourceRepositoryUrl(),
task.getSourceDefaultBranch(),
project.getGiteaRepoName(),
receipt(task),
task.getTargetSha());
} else {
if (task.getReceiptJson() == null) {
phase(task, "PRECHECK");
GitHubSourceChecks.Repository source = check(task.getSourceRepositoryUrl());
if (source.id() != task.getSourceRepositoryId())
throw new MigrationFailure("SOURCE_CHANGED", "来源仓库标识已变化");
gitea.checkMigrationSettings();
}
result =
engine().run(
task.getSourceRepositoryUrl(),
task.getSourceRepositoryUrl(),
task.getSourceDefaultBranch(),
project.getGiteaRepoName(),
settings.getMigrationTimeout(),
new RepositoryMigration.Progress() {
public GiteaClient.RemoteRepository receipt() {
return AdminImportService.this.receipt(task);
}
public String targetSha() {
return task.getTargetSha();
}
public void receipt(
GiteaClient.RemoteRepository receipt)
throws IOException {
String data = json.writeValueAsString(receipt);
tasks.update(
null,
new LambdaUpdateWrapper<ProjectImportTask>()
.eq(
ProjectImportTask::getId,
task.getId())
.set(
ProjectImportTask
::getReceiptJson,
data)
.set(
ProjectImportTask::getPhase,
"MIGRATED")
.set(
ProjectImportTask
::getUpdatedAt,
ProjectSupport.now()));
task.setReceiptJson(data);
task.setPhase("MIGRATED");
}
public void targetSha(String sha) {
task.setTargetSha(sha);
save(task);
}
public void phase(String next) {
AdminImportService.this.phase(task, next);
}
});
}
List<String> warnings;
try {
warnings =
gitea.migrationWarnings(
result.remote().repository(),
result.sha(),
properties.getMaxFileBytes());
} catch (RuntimeException error) {
warnings = List.of("兼容性扫描未完成,请检查大文件、LFS、子模块及符号链接");
}
String warningsJson = json.writeValueAsString(warnings);
transactions.executeWithoutResult(
status -> {
int updated =
projects.update(
null,
new LambdaUpdateWrapper<Project>()
.eq(Project::getProjectCode, task.getProjectCode())
.eq(Project::getDeleted, false)
.set(Project::getGiteaRepoId, result.remote().id())
.set(Project::getRepoStatus, "READY")
.set(Project::getRepoWriteStatus, "IDLE")
.set(Project::getUpdatedAt, ProjectSupport.now()));
if (updated != 1) throw BusinessException.conflict("项目已变化,不能关联迁移结果");
tasks.update(
null,
new LambdaUpdateWrapper<ProjectImportTask>()
.eq(ProjectImportTask::getId, task.getId())
.set(ProjectImportTask::getStatus, "SUCCEEDED")
.set(ProjectImportTask::getPhase, "LINKED")
.set(ProjectImportTask::getWarningsJson, warningsJson)
.set(ProjectImportTask::getErrorCode, null)
.set(ProjectImportTask::getFinishedAt, ProjectSupport.now())
.set(
ProjectImportTask::getUpdatedAt,
ProjectSupport.now()));
});
log.info(
"后台迁移完成 userCode={} taskCode={} projectCode={} commitSha={}",
task.getOperatorUserCode(),
task.getTaskCode(),
task.getProjectCode(),
result.sha());
} catch (IOException error) {
throw new IllegalStateException("迁移收据持久化未确认", error);
}
}
private void fail(String code, RuntimeException error) {
boolean interrupted = Thread.interrupted();
try {
ProjectImportTask task = find(code);
boolean review =
Set.of("MIGRATING", "MIGRATED", "NORMALIZING", "BRANCH_READY")
.contains(task.getPhase());
if (error instanceof GiteaFailure failure && failure.uncertain()) review = true;
if (error instanceof MigrationFailure failure
&& "REMOTE_ALREADY_EXISTS".equals(failure.code())) review = true;
String errorCode =
error instanceof BusinessException business
? business.code()
: "IMPORT_INTERNAL_ERROR";
if (review) locks.markUncertain(task.getProjectCode());
tasks.update(
null,
new LambdaUpdateWrapper<ProjectImportTask>()
.eq(ProjectImportTask::getId, task.getId())
.set(ProjectImportTask::getStatus, review ? "REVIEW" : "FAILED")
.set(ProjectImportTask::getErrorCode, errorCode)
.set(ProjectImportTask::getFinishedAt, ProjectSupport.now())
.set(ProjectImportTask::getUpdatedAt, ProjectSupport.now()));
log.error(
"后台迁移失败 userCode={} taskCode={} projectCode={} phase={} code={} review={}"
+ " errorType={}",
task.getOperatorUserCode(),
code,
task.getProjectCode(),
task.getPhase(),
errorCode,
review,
error.getClass().getSimpleName());
} finally {
if (interrupted) Thread.currentThread().interrupt();
}
}
private void phase(ProjectImportTask task, String phase) {
transactions.executeWithoutResult(
status -> {
task.setPhase(phase);
save(task);
if (Set.of("MIGRATING", "NORMALIZING").contains(phase)) {
int updated =
projects.update(
null,
new LambdaUpdateWrapper<Project>()
.eq(Project::getProjectCode, task.getProjectCode())
.eq(Project::getDeleted, false)
.set(Project::getRepoWriteStatus, "WRITING"));
if (updated != 1) throw BusinessException.conflict("项目状态已变化,停止迁移");
}
});
}
private void save(ProjectImportTask task) {
task.setUpdatedAt(ProjectSupport.now());
if (tasks.updateById(task) != 1) throw new IllegalStateException("迁移进度保存失败");
}
private GiteaClient.RemoteRepository receipt(ProjectImportTask task) {
if (task.getReceiptJson() == null) return null;
try {
return json.readValue(task.getReceiptJson(), GiteaClient.RemoteRepository.class);
} catch (IOException error) {
throw new IllegalStateException("迁移收据无效");
}
}
private RepositoryMigration engine() {
return new RepositoryMigration(gitea, properties.getGitea().getOwner());
}
}
@@ -0,0 +1,250 @@
package com.linkerbot.community.service.admin;
import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper;
import com.baomidou.mybatisplus.core.conditions.update.LambdaUpdateWrapper;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
import com.linkerbot.community.bean.AdminContracts;
import com.linkerbot.community.bean.Contracts;
import com.linkerbot.community.bean.entity.*;
import com.linkerbot.community.common.*;
import com.linkerbot.community.service.ProjectService;
import com.linkerbot.community.service.mapper.*;
import com.linkerbot.community.service.support.*;
import lombok.RequiredArgsConstructor;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.dao.DuplicateKeyException;
import org.springframework.stereotype.Service;
import java.util.Set;
@Service
@RequiredArgsConstructor
public class AdminManagementService {
private static final Logger log = LoggerFactory.getLogger(AdminManagementService.class);
private final UserMapper users;
private final ProjectMapper mapper;
private final ProjectService projects;
private final ProjectLocks locks;
public void requireAdmin() {
if (!ProjectActors.isAdmin()) throw new BusinessException(403, "FORBIDDEN", "此操作仅限后台管理员");
}
public User user(String code) {
requireAdmin();
User user = users.selectOne(new LambdaQueryWrapper<User>().eq(User::getUserCode, code));
if (user == null) throw BusinessException.notFound();
return user;
}
public AdminContracts.UserView userView(User user) {
requireAdmin();
long count =
mapper.selectCount(
new LambdaQueryWrapper<Project>()
.eq(Project::getOwnerUserId, user.getId())
.eq(Project::getDeleted, false));
return new AdminContracts.UserView(
user.getUserCode(),
user.getNickname(),
user.getEmail(),
user.getStatus(),
user.getRole(),
count,
user.getCreatedAt());
}
public PageResponse<AdminContracts.UserView> users(
long page,
long size,
String userCode,
String nickname,
String email,
String role,
String status) {
requireAdmin();
ProjectSupport.pageCheck(page, size);
checkKeyword(nickname);
checkKeyword(email);
if (role != null && !Set.of("USER", "ADMIN", "OFFICIAL").contains(role))
throw BusinessException.badRequest("角色不合法");
if (status != null && !Set.of("ACTIVE", "DISABLED").contains(status))
throw BusinessException.badRequest("状态不合法");
LambdaQueryWrapper<User> query =
new LambdaQueryWrapper<User>()
.eq(userCode != null, User::getUserCode, userCode)
.like(nickname != null, User::getNickname, nickname)
.like(email != null, User::getEmail, email)
.eq(role != null, User::getRole, role)
.eq(status != null, User::getStatus, status)
.orderByDesc(User::getCreatedAt)
.orderByDesc(User::getId);
Page<User> result = users.selectPage(new Page<>(page, size), query);
return new PageResponse<>(
result.getRecords().stream().map(this::userView).toList(),
page,
size,
result.getTotal());
}
public AdminContracts.UserView nickname(String code, String nickname) {
User user = user(code);
users.update(
null,
new LambdaUpdateWrapper<User>()
.eq(User::getId, user.getId())
.set(User::getNickname, nickname.trim())
.set(User::getUpdatedAt, ProjectSupport.now()));
return userView(user(code));
}
public void status(String code, String status) {
User user = user(code);
if ("DISABLED".equals(status)
&& (code.equals(ProjectActors.admin().userCode())
|| "OFFICIAL".equals(user.getRole())))
throw BusinessException.conflict("不能禁用当前管理员或官方托管账号");
if (status.equals(user.getStatus())) return;
int updated =
users.update(
null,
new LambdaUpdateWrapper<User>()
.eq(User::getId, user.getId())
.eq(User::getStatus, user.getStatus())
.eq(User::getSessionVersion, user.getSessionVersion())
.set(User::getStatus, status)
.setSql("session_version = session_version + 1")
.set(User::getUpdatedAt, ProjectSupport.now()));
if (updated != 1) throw BusinessException.conflict("用户状态已变化,请刷新重试");
log.info(
"用户状态修改 operatorUserCode={} userCode={} status={}",
ProjectActors.admin().userCode(),
code,
status);
}
public User owner(String code) {
requireAdmin();
if (code != null && !code.isBlank()) {
User user = user(code);
if (!"ACTIVE".equals(user.getStatus()) || !"USER".equals(user.getRole()))
throw BusinessException.badRequest("只能指定正常普通用户为所有者");
return user;
}
User official =
users.selectOne(
new LambdaQueryWrapper<User>().eq(User::getEmail, OfficialUsers.EMAIL));
if (official == null) {
official = new User();
official.setUserCode(UserCodes.next());
official.setEmail(OfficialUsers.EMAIL);
official.setNickname("社区官方");
official.setStatus("ACTIVE");
official.setRole("OFFICIAL");
official.setCreatedAt(ProjectSupport.now());
official.setUpdatedAt(official.getCreatedAt());
try {
users.insert(official);
} catch (DuplicateKeyException duplicate) {
official =
users.selectOne(
new LambdaQueryWrapper<User>()
.eq(User::getEmail, OfficialUsers.EMAIL));
}
}
if (official == null
|| !"OFFICIAL".equals(official.getRole())
|| !"ACTIVE".equals(official.getStatus()))
throw BusinessException.conflict("官方托管账号状态异常,请人工核对");
return official;
}
public PageResponse<AdminContracts.ManagedProjectView> projects(
long page,
long size,
String ownerCode,
String type,
String keyword,
String publishStatus,
String repoStatus,
String moderationStatus) {
requireAdmin();
ProjectSupport.pageCheck(page, size);
checkKeyword(keyword);
LambdaQueryWrapper<Project> query =
new LambdaQueryWrapper<Project>().eq(Project::getDeleted, false);
if (ownerCode != null) query.eq(Project::getOwnerUserId, user(ownerCode).getId());
query.eq(type != null, Project::getProjectType, type)
.eq(publishStatus != null, Project::getPublishStatus, publishStatus)
.eq(repoStatus != null, Project::getRepoStatus, repoStatus);
if (moderationStatus != null) {
if (!Set.of("NORMAL", "BLOCKED").contains(moderationStatus))
throw BusinessException.badRequest("治理状态不合法");
if ("BLOCKED".equals(moderationStatus)) query.eq(Project::getPublishStatus, "BLOCKED");
else query.ne(Project::getPublishStatus, "BLOCKED");
}
if (keyword != null && !keyword.isBlank())
query.and(
q ->
q.like(Project::getName, keyword)
.or()
.like(Project::getDescription, keyword));
query.orderByDesc(Project::getCreatedAt).orderByDesc(Project::getId);
Page<Project> result = mapper.selectPage(new Page<>(page, size), query);
return new PageResponse<>(
result.getRecords().stream().map(this::projectView).toList(),
page,
size,
result.getTotal());
}
public AdminContracts.ManagedProjectView projectView(Project project) {
requireAdmin();
return new AdminContracts.ManagedProjectView(
projects.view(project, null),
"BLOCKED".equals(project.getPublishStatus()) ? "BLOCKED" : "NORMAL",
project.getModerationReason(),
Boolean.TRUE.equals(project.getDeleted()));
}
public AdminContracts.ManagedProjectView detail(String code) {
requireAdmin();
return projectView(projects.owner(0, code));
}
public Contracts.ProjectView create(AdminContracts.CreateProjectInput input) {
User owner = owner(input.ownerUserCode());
return projects.create(owner.getId(), input.project());
}
public void moderate(String code, boolean blocked, String reason) {
requireAdmin();
locks.serial(
code,
() -> {
Project project = projects.owner(0, code);
LambdaUpdateWrapper<Project> update =
new LambdaUpdateWrapper<Project>()
.eq(Project::getId, project.getId())
.set(Project::getModerationReason, blocked ? reason : null)
.set(Project::getUpdatedAt, ProjectSupport.now());
if (blocked) update.set(Project::getPublishStatus, "BLOCKED");
else if ("BLOCKED".equals(project.getPublishStatus()))
update.set(Project::getPublishStatus, "UNPUBLISHED");
mapper.update(null, update);
log.info(
"项目治理 operatorUserCode={} projectCode={} blocked={}",
ProjectActors.admin().userCode(),
code,
blocked);
return null;
});
}
private void checkKeyword(String value) {
if (value != null && value.length() > 254) throw BusinessException.badRequest("查询条件过长");
}
}
@@ -0,0 +1,30 @@
package com.linkerbot.community.service.config;
import jakarta.validation.constraints.Max;
import jakarta.validation.constraints.Min;
import lombok.Getter;
import lombok.Setter;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.stereotype.Component;
import org.springframework.validation.annotation.Validated;
import java.time.Duration;
import java.util.List;
@Getter
@Setter
@Component
@Validated
@ConfigurationProperties(prefix = "community.admin")
public class AdminProperties {
private List<String> allowedOrigins = List.of();
private String githubToken = "";
private boolean workerEnabled = true;
private Duration migrationTimeout = Duration.ofMinutes(30);
@Min(1)
@Max(4)
private int importConcurrency = 2;
}
@@ -2,7 +2,7 @@ package com.linkerbot.community.service.impl;
import com.linkerbot.community.bean.Contracts.CoverUploadInput;
import com.linkerbot.community.bean.Contracts.CoverUploadSignature;
import com.linkerbot.community.bean.entity.User;
import com.linkerbot.community.service.support.ProjectActors;
import com.linkerbot.community.common.BusinessException;
import com.linkerbot.community.service.CoverUploadService;
import com.linkerbot.community.service.UserService;
@@ -32,10 +32,10 @@ public class CoverUploadServiceImpl implements CoverUploadService {
if (extension == null) throw BusinessException.badRequest("封面仅支持 JPG、PNG、WebP 图片");
if (input.fileSize() > OssPostSigner.MAX_FILE_BYTES)
throw new BusinessException(413, "UPLOAD_TOO_LARGE", "封面图片不能超过 5 MiB");
User user = users.require(userId);
CoverUploadSignature result = signer.sign(user.getUserCode(), input.contentType(), extension);
ProjectActors.Author author = ProjectActors.author(users, userId);
CoverUploadSignature result = signer.sign(author.code(), input.contentType(), extension);
log.info("生成封面上传签名 userCode={} objectKey={} expiresAt={}",
user.getUserCode(), result.objectKey(), result.expiresAt());
author.code(), result.objectKey(), result.expiresAt());
return result;
}
}
@@ -2,7 +2,7 @@ package com.linkerbot.community.service.impl;
import com.linkerbot.community.bean.Contracts.*;
import com.linkerbot.community.bean.entity.Project;
import com.linkerbot.community.bean.entity.User;
import com.linkerbot.community.service.support.ProjectActors;
import com.linkerbot.community.common.*;
import com.linkerbot.community.service.FileService;
import com.linkerbot.community.service.ProjectService;
@@ -184,14 +184,14 @@ public class FileServiceImpl implements FileService {
}
}
size(bytes, config.getMaxRequestBytes());
User user = users.require(userId);
ProjectActors.Author author = ProjectActors.author(users, userId);
CommitView result =
gitea.commit(
repo,
changes,
input.message(),
user.getNickname(),
user.getUserCode());
author.nickname(),
author.code());
log.info(
"文件提交 projectCode={} commitSha={} fileCount={}",
code,
@@ -256,10 +256,10 @@ public class FileServiceImpl implements FileService {
new GiteaClient.Change(
blob == null ? "create" : "update", path, blob, data));
}
User user = users.require(userId);
ProjectActors.Author author = ProjectActors.author(users, userId);
CommitView result =
gitea.commit(
repo, changes, message, user.getNickname(), user.getUserCode());
repo, changes, message, author.nickname(), author.code());
log.info(
"文件上传 projectCode={} commitSha={} fileCount={}",
code,
@@ -13,22 +13,23 @@ import com.linkerbot.community.common.*;
import com.linkerbot.community.service.ProjectService;
import com.linkerbot.community.service.UserService;
import com.linkerbot.community.service.config.CommunityProperties;
import com.linkerbot.community.service.gitea.*;
import com.linkerbot.community.service.fork.ForkDeadline;
import com.linkerbot.community.service.fork.ForkExecutor;
import com.linkerbot.community.service.fork.GitSnapshotCopier;
import com.linkerbot.community.service.gitea.*;
import com.linkerbot.community.service.mapper.*;
import com.linkerbot.community.service.oss.UrdfResources;
import com.linkerbot.community.service.oss.OssResourcePaths;
import com.linkerbot.community.service.support.ProjectLocks;
import com.linkerbot.community.service.oss.UrdfResources;
import com.linkerbot.community.service.support.ProjectActors;
import com.linkerbot.community.service.support.ProjectInputs;
import org.springframework.dao.DuplicateKeyException;
import org.springframework.beans.factory.ObjectProvider;
import com.linkerbot.community.service.support.ProjectLocks;
import lombok.RequiredArgsConstructor;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.ObjectProvider;
import org.springframework.dao.DuplicateKeyException;
import org.springframework.stereotype.Service;
import java.util.*;
@@ -39,6 +40,7 @@ import java.util.function.Function;
public class ProjectServiceImpl implements ProjectService {
private static final Logger log = LoggerFactory.getLogger(ProjectServiceImpl.class);
private final ProjectMapper projects;
private final ProjectImportTaskMapper importTasks;
private final LikeMapper likes;
private final CommentMapper comments;
private final UserService users;
@@ -72,7 +74,7 @@ public class ProjectServiceImpl implements ProjectService {
projects.selectOne(
new LambdaQueryWrapper<Project>()
.eq(Project::getProjectCode, code)
.eq(Project::getOwnerUserId, userId)
.eq(!ProjectActors.isAdmin(), Project::getOwnerUserId, userId)
.eq(Project::getDeleted, false));
if (project == null) throw BusinessException.notFound();
return project;
@@ -143,6 +145,10 @@ public class ProjectServiceImpl implements ProjectService {
log.warn("项目资源地址生成失败 projectCode={} code={}", project.getProjectCode(), error.code());
throw error;
}
ProjectImportTask imported =
importTasks.selectOne(
new LambdaQueryWrapper<ProjectImportTask>()
.eq(ProjectImportTask::getProjectCode, project.getProjectCode()));
return new ProjectView(
project.getProjectCode(),
project.getName(),
@@ -161,6 +167,8 @@ public class ProjectServiceImpl implements ProjectService {
likeCount,
commentCount,
liked,
imported == null ? null : imported.getSourceRepositoryUrl(),
imported == null ? null : imported.getSourceLicense(),
project.getCreatedAt(),
project.getUpdatedAt());
}
@@ -208,7 +216,8 @@ public class ProjectServiceImpl implements ProjectService {
return initialize(userId, input, source, null);
}
private Project initialize(long userId, ProjectInput input, Project source, ForkDeadline deadline) {
private Project initialize(
long userId, ProjectInput input, Project source, ForkDeadline deadline) {
if (source == null) validateInput(input);
else ProjectInputs.validateMetadata(input, config);
Project project = new Project();
@@ -216,8 +225,10 @@ public class ProjectServiceImpl implements ProjectService {
project.setOwnerUserId(userId);
project.setName(input.name().trim());
project.setDescription(input.description() == null ? "" : input.description());
project.setCoverUrl(source == null ? normalizeCoverUrl(input.coverUrl())
: resourcePaths.coverReference(source.getCoverUrl()));
project.setCoverUrl(
source == null
? normalizeCoverUrl(input.coverUrl())
: resourcePaths.coverReference(source.getCoverUrl()));
project.setProjectType(input.projectType());
project.setGiteaOwner(config.getGitea().getOwner());
project.setGiteaRepoName("p_" + project.getProjectCode());
@@ -242,8 +253,9 @@ public class ProjectServiceImpl implements ProjectService {
boolean codeConflict = false;
for (Throwable cause = duplicate; cause != null; cause = cause.getCause()) {
if (cause.getMessage() != null
&& cause.getMessage().toLowerCase(Locale.ROOT).contains("uk_project_code"))
codeConflict = true;
&& cause.getMessage()
.toLowerCase(Locale.ROOT)
.contains("uk_project_code")) codeConflict = true;
}
if (!codeConflict) throw duplicate;
log.warn("项目编码冲突 projectCode={} attempt={}", project.getProjectCode(), attempt);
@@ -272,31 +284,58 @@ public class ProjectServiceImpl implements ProjectService {
throw BusinessException.conflict("社区版本已更新,请刷新后 Fork");
ProjectInput data =
new ProjectInput(
input.name(), source.getDescription(), source.getProjectType(), source.getCoverUrl());
input.name(),
source.getDescription(),
source.getProjectType(),
source.getCoverUrl());
ProjectInputs.validateMetadata(data, config);
ForkExecutor executor = forks.getIfAvailable();
if (executor == null) throw new BusinessException(503, "FORK_UNAVAILABLE", "当前进程未开启 Fork");
return executor.execute(deadline -> {
deadline.check();
Project project;
try { project = initialize(userId, data, source, deadline); }
catch (BusinessException error) { throw error; }
catch (RuntimeException error) {
if (deadline.projectCode() != null) locks.markUncertain(deadline.projectCode());
log.error("Fork 项目记录写入异常 projectCode={} sourceProjectCode={} errorType={}",
deadline.projectCode(), sourceCode, error.getClass().getSimpleName());
throw new BusinessException(500, "PROJECT_INITIALIZATION_ERROR", "Fork 项目记录待核对,请按项目编码人工检查", deadline.details());
}
return locks.write(project.getProjectCode(), () -> forkRemote(project, source, user, deadline));
});
return executor.execute(
deadline -> {
deadline.check();
Project project;
try {
project = initialize(userId, data, source, deadline);
} catch (BusinessException error) {
throw error;
} catch (RuntimeException error) {
if (deadline.projectCode() != null)
locks.markUncertain(deadline.projectCode());
log.error(
"Fork 项目记录写入异常 projectCode={} sourceProjectCode={} errorType={}",
deadline.projectCode(),
sourceCode,
error.getClass().getSimpleName());
throw new BusinessException(
500,
"PROJECT_INITIALIZATION_ERROR",
"Fork 项目记录待核对,请按项目编码人工检查",
deadline.details());
}
return locks.write(
project.getProjectCode(),
() -> forkRemote(project, source, user, deadline));
});
}
private ProjectView forkRemote(Project project, Project source, User user, ForkDeadline deadline) {
private ProjectView forkRemote(
Project project, Project source, User user, ForkDeadline deadline) {
long started = System.nanoTime();
log.info("Fork 开始 projectCode={} sourceProjectCode={} sourceCommitSha={}",
project.getProjectCode(), source.getProjectCode(), project.getSourceCommitSha());
try (GitSnapshotCopier.PreparedSnapshot snapshot = snapshots.getObject().prepare(
repository(source), project.getSourceCommitSha(), user.getNickname(), user.getUserCode(), deadline)) {
log.info(
"Fork 开始 projectCode={} sourceProjectCode={} sourceCommitSha={}",
project.getProjectCode(),
source.getProjectCode(),
project.getSourceCommitSha());
try (GitSnapshotCopier.PreparedSnapshot snapshot =
snapshots
.getObject()
.prepare(
repository(source),
project.getSourceCommitSha(),
user.getNickname(),
user.getUserCode(),
deadline)) {
deadline.beginWrite();
GiteaClient.Repository repo;
try {
@@ -318,16 +357,30 @@ public class ProjectServiceImpl implements ProjectService {
project.setRepoStatus("READY");
project.setUpdatedAt(now());
if (projects.updateById(project) != 1) throw new IllegalStateException("Fork 完成状态回写失败");
log.info("Fork 完成 projectCode={} sourceProjectCode={} sourceCommitSha={} snapshotCommitSha={} durationMs={}",
project.getProjectCode(), source.getProjectCode(), project.getSourceCommitSha(), commit,
java.util.concurrent.TimeUnit.NANOSECONDS.toMillis(System.nanoTime() - started));
log.info(
"Fork 完成 projectCode={} sourceProjectCode={} sourceCommitSha={}"
+ " snapshotCommitSha={} durationMs={}",
project.getProjectCode(),
source.getProjectCode(),
project.getSourceCommitSha(),
commit,
java.util.concurrent.TimeUnit.NANOSECONDS.toMillis(
System.nanoTime() - started));
return view(project, project.getOwnerUserId());
} catch (RuntimeException error) {
boolean uncertain = deadline.writing() || !(error instanceof BusinessException)
|| error instanceof GiteaFailure remote && remote.uncertain();
String code = error instanceof BusinessException business ? business.code() : "PROJECT_INITIALIZATION_ERROR";
boolean uncertain =
deadline.writing()
|| !(error instanceof BusinessException)
|| error instanceof GiteaFailure remote && remote.uncertain();
String code =
error instanceof BusinessException business
? business.code()
: "PROJECT_INITIALIZATION_ERROR";
int status = error instanceof BusinessException business ? business.status() : 500;
String message = error instanceof BusinessException business ? business.getMessage() : "Fork 初始化未完成,请按项目编码核对";
String message =
error instanceof BusinessException business
? business.getMessage()
: "Fork 初始化未完成,请按项目编码核对";
if (deadline.writing() && error instanceof BusinessException) {
code = "REMOTE_RESULT_UNKNOWN";
status = 504;
@@ -338,27 +391,39 @@ public class ProjectServiceImpl implements ProjectService {
project.setRepoStatus(uncertain ? "CREATING" : "FAILED");
project.setUpdatedAt(now());
if (uncertain) locks.markUncertain(project.getProjectCode());
if (projects.updateById(project) != 1) throw new IllegalStateException("Fork 失败状态回写失败");
if (projects.updateById(project) != 1)
throw new IllegalStateException("Fork 失败状态回写失败");
} catch (RuntimeException persistence) {
locks.markUncertain(project.getProjectCode());
log.error("Fork 状态回写异常 projectCode={} errorType={}", project.getProjectCode(), persistence.getClass().getSimpleName());
log.error(
"Fork 状态回写异常 projectCode={} errorType={}",
project.getProjectCode(),
persistence.getClass().getSimpleName());
code = "PROJECT_INITIALIZATION_ERROR";
status = 500;
message = "Fork 状态未确认,请按项目编码人工核对";
} finally {
if (interrupted) Thread.currentThread().interrupt();
}
log.warn("Fork 失败 projectCode={} sourceProjectCode={} sourceCommitSha={} code={} uncertain={} errorType={} durationMs={}",
project.getProjectCode(), source.getProjectCode(), project.getSourceCommitSha(), code, uncertain,
error.getClass().getSimpleName(), java.util.concurrent.TimeUnit.NANOSECONDS.toMillis(System.nanoTime() - started));
throw new BusinessException(status, code, message, Map.of("projectCode", project.getProjectCode()));
log.warn(
"Fork 失败 projectCode={} sourceProjectCode={} sourceCommitSha={} code={}"
+ " uncertain={} errorType={} durationMs={}",
project.getProjectCode(),
source.getProjectCode(),
project.getSourceCommitSha(),
code,
uncertain,
error.getClass().getSimpleName(),
java.util.concurrent.TimeUnit.NANOSECONDS.toMillis(
System.nanoTime() - started));
throw new BusinessException(
status, code, message, Map.of("projectCode", project.getProjectCode()));
}
}
private ProjectView createRemote(Project project) {
try {
GiteaClient.Repository repo =
gitea.createRepository(project.getGiteaRepoName(), true);
GiteaClient.Repository repo = gitea.createRepository(project.getGiteaRepoName(), true);
project.setGiteaRepoId(repo.id());
project.setRepoStatus("READY");
project.setUpdatedAt(now());
@@ -478,6 +543,8 @@ public class ProjectServiceImpl implements ProjectService {
code,
project -> {
ready(project);
if ("BLOCKED".equals(project.getPublishStatus()))
throw new BusinessException(409, "PROJECT_BLOCKED", "项目已被管理员下架");
String head = gitea.head(repository(project));
if (!head.equals(FilePaths.sha(expectedSha)))
throw BusinessException.conflict("工作区版本已变化,请刷新后发布");
@@ -497,7 +564,9 @@ public class ProjectServiceImpl implements ProjectService {
userId,
code,
project -> {
project.setPublishStatus("UNPUBLISHED");
// 撤回不能解除管理员下架,避免所有者先撤回再重新发布。
if (!"BLOCKED".equals(project.getPublishStatus()))
project.setPublishStatus("UNPUBLISHED");
project.setUpdatedAt(now());
projects.updateById(project);
log.info("撤回社区 projectCode={}", code);
@@ -28,9 +28,11 @@ public class UserServiceImpl implements UserService {
@Override
public User login(String email) {
String normalized = email.trim().toLowerCase(Locale.ROOT);
if (com.linkerbot.community.service.support.OfficialUsers.EMAIL.equals(normalized))
throw new BusinessException(403, "ACCOUNT_DISABLED", "账号不可用");
User user = users.selectOne(new LambdaQueryWrapper<User>().eq(User::getEmail, normalized));
if (user == null) user = create(normalized);
if (!"ACTIVE".equals(user.getStatus()))
if (!"ACTIVE".equals(user.getStatus()) || !"USER".equals(user.getRole()))
throw new BusinessException(403, "ACCOUNT_DISABLED", "账号不可用");
log.info("用户登录 userCode={}", user.getUserCode());
return user;
@@ -0,0 +1,6 @@
package com.linkerbot.community.service.mapper;
import com.baomidou.mybatisplus.core.mapper.BaseMapper;
import com.linkerbot.community.bean.entity.ProjectImportTask;
public interface ProjectImportTaskMapper extends BaseMapper<ProjectImportTask> {}
@@ -0,0 +1,245 @@
package com.linkerbot.community.service.migration;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import java.io.InputStream;
import java.net.URI;
import java.net.http.*;
import java.time.Duration;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
public class GitHubSourceChecks {
public record Repository(
long id,
String url,
String defaultBranch,
String name,
String description,
String license) {
public Repository(long id, String url, String branch) {
this(id, url, branch, repositoryKey(url).split("/")[1], "", null);
}
}
private final HttpClient http =
HttpClient.newBuilder()
.connectTimeout(Duration.ofSeconds(10))
.followRedirects(HttpClient.Redirect.NEVER)
.build();
private final ObjectMapper json = new ObjectMapper();
private final String token;
private volatile long apiLimitedUntil;
public GitHubSourceChecks(String token) {
this.token = token == null ? "" : token;
}
public Repository repository(String url) throws Exception {
if (System.nanoTime() < apiLimitedUntil) return publicPage(url);
URI uri =
URI.create("https://api.github.com/repos/" + GitHubSourceChecks.repositoryKey(url));
for (int redirects = 0; redirects <= 5; redirects++) {
if (!"https".equalsIgnoreCase(uri.getScheme())
|| !"api.github.com".equalsIgnoreCase(uri.getHost())
|| uri.getUserInfo() != null
|| (uri.getPort() != -1 && uri.getPort() != 443))
throw new MigrationFailure("GITHUB_REDIRECT", "GitHub API 跳转地址不符合要求");
HttpRequest.Builder request =
HttpRequest.newBuilder(uri)
.timeout(Duration.ofSeconds(30))
.header("Accept", "application/vnd.github+json")
.header("User-Agent", "community-import/1");
if (!token.isBlank()) request.header("Authorization", "Bearer " + token);
HttpResponse<String> response =
http.send(request.GET().build(), HttpResponse.BodyHandlers.ofString());
int status = response.statusCode();
if (status >= 300 && status < 400) {
uri = redirect(uri, response);
continue;
}
if (status == 403 || status == 429) {
apiLimitedUntil = System.nanoTime() + Duration.ofMinutes(5).toNanos();
return publicPage(url);
}
if (status == 404) throw new MigrationFailure("GITHUB_NOT_FOUND", "GitHub 仓库不可访问");
if (status != 200) throw new MigrationFailure("GITHUB_HTTP_" + status, "GitHub 返回异常状态");
JsonNode data = json.readTree(response.body());
if (!data.path("private").isBoolean()
|| data.path("id").asLong() <= 0
|| data.path("private").asBoolean()
|| data.path("default_branch").asText().isBlank()
|| data.path("size").asLong(-1) < 0)
throw new MigrationFailure("GITHUB_INVALID_REPOSITORY", "仓库元数据无效或不是公开仓库");
String canonical =
"https://github.com/"
+ GitHubSourceChecks.repositoryKey(data.path("html_url").asText());
return new Repository(
data.path("id").asLong(),
canonical,
data.path("default_branch").asText(),
data.path("name").asText(),
data.path("description").asText(""),
data.path("license").path("spdx_id").asText(null));
}
throw new MigrationFailure("GITHUB_REDIRECT", "GitHub 跳转次数过多");
}
private Repository publicPage(String url) throws Exception {
URI uri = URI.create("https://github.com/" + GitHubSourceChecks.repositoryKey(url));
for (int redirects = 0; redirects <= 5; redirects++) {
if (!"https".equalsIgnoreCase(uri.getScheme())
|| !"github.com".equalsIgnoreCase(uri.getHost())
|| uri.getUserInfo() != null
|| (uri.getPort() != -1 && uri.getPort() != 443))
throw new MigrationFailure("GITHUB_REDIRECT", "GitHub 网页跳转地址不符合要求");
HttpRequest request =
HttpRequest.newBuilder(uri)
.timeout(Duration.ofSeconds(30))
.header("Accept", "text/html")
.header("User-Agent", "community-import/1")
.GET()
.build();
// 网页请求不携带任何 GitHub 或 Gitea 凭据。
HttpResponse<String> response =
http.send(request, HttpResponse.BodyHandlers.ofString());
int status = response.statusCode();
if (status >= 300 && status < 400) {
uri = redirect(uri, response);
continue;
}
if (status == 403 || status == 429)
throw new MigrationFailure("GITHUB_ACCESS_OR_RATE_LIMIT", "GitHub API 及公开网页均不可访问");
if (status != 200)
throw new MigrationFailure("GITHUB_PAGE_HTTP_" + status, "GitHub 公开仓库网页不可访问");
return parsePublicPage(response.body(), uri.toString());
}
throw new MigrationFailure("GITHUB_REDIRECT", "GitHub 网页跳转次数过多");
}
public static Repository parsePublicPage(String html, String url) throws Exception {
String key = GitHubSourceChecks.repositoryKey(url);
String id = pageMeta(html, "repository_id");
if (!id.matches("[1-9][0-9]*")
|| !key.equalsIgnoreCase(pageMeta(html, "repository_nwo"))
|| !"true".equals(pageMeta(html, "repository_public")))
throw new MigrationFailure("GITHUB_PAGE_METADATA", "GitHub 网页仓库身份或公开状态无法核验");
Matcher scripts =
Pattern.compile(
"<script[^>]*data-target=\"react-app.embeddedData\"[^>]*>(.*?)</script>",
Pattern.DOTALL)
.matcher(html);
while (scripts.find()) {
JsonNode repo =
new ObjectMapper()
.readTree(scripts.group(1))
.path("payload")
.path("codeViewLayoutRoute")
.path("repo");
if (repo.path("id").asLong() != Long.parseLong(id)) continue;
String name = repo.path("ownerLogin").asText() + "/" + repo.path("name").asText();
if (!name.equalsIgnoreCase(key)
|| !repo.path("public").asBoolean()
|| !repo.path("private").isBoolean()
|| repo.path("private").asBoolean()
|| !repo.path("isEmpty").isBoolean()
|| repo.path("isEmpty").asBoolean()
|| repo.path("defaultBranch").asText().isBlank()) break;
return new Repository(
Long.parseLong(id),
"https://github.com/" + name,
repo.path("defaultBranch").asText());
}
throw new MigrationFailure("GITHUB_PAGE_METADATA", "GitHub 网页缺少一致的仓库 ID、公开状态或默认分支");
}
private static String pageMeta(String html, String field) {
Matcher meta =
Pattern.compile(
"<meta name=\"octolytics-dimension-"
+ field
+ "\" content=\"([^\"]*)\"[^>]*>")
.matcher(html);
return meta.find() ? meta.group(1) : "";
}
public void cover(String url) throws Exception {
URI uri = URI.create(url);
for (int redirects = 0; redirects <= 5; redirects++) {
if (!"https".equalsIgnoreCase(uri.getScheme())
|| uri.getHost() == null
|| uri.getUserInfo() != null)
throw new MigrationFailure("COVER_REDIRECT", "封面跳转地址必须为 HTTPS");
HttpRequest request =
HttpRequest.newBuilder(uri)
.timeout(Duration.ofSeconds(30))
.header("User-Agent", "community-import/1")
.header("Range", "bytes=0-511")
.GET()
.build();
HttpResponse<InputStream> response =
http.send(request, HttpResponse.BodyHandlers.ofInputStream());
try (InputStream body = response.body()) {
int status = response.statusCode();
if (status >= 300 && status < 400) {
uri = redirect(uri, response);
continue;
}
if (status != 200 && status != 206)
throw new MigrationFailure("COVER_HTTP_" + status, "封面链接无法读取");
byte[] signature = body.readNBytes(16);
boolean image =
signature.length >= 3
&& ((signature[0] & 255) == 255 && (signature[1] & 255) == 216
|| signature.length >= 8
&& (signature[0] & 255) == 137
&& signature[1] == 'P'
&& signature[2] == 'N'
&& signature[3] == 'G'
|| signature[0] == 'G'
&& signature[1] == 'I'
&& signature[2] == 'F'
|| signature.length >= 12
&& signature[0] == 'R'
&& signature[1] == 'I'
&& signature[8] == 'W'
&& signature[9] == 'E');
if (!image)
throw new MigrationFailure("COVER_NOT_IMAGE", "封面未返回 JPG、PNG、GIF 或 WebP 图片内容");
return;
}
}
throw new MigrationFailure("COVER_REDIRECT", "封面跳转次数过多");
}
private URI redirect(URI uri, HttpResponse<?> response) {
return uri.resolve(
response.headers()
.firstValue("Location")
.orElseThrow(
() -> new MigrationFailure("INVALID_REDIRECT", "跳转响应缺少 Location")));
}
public static String repositoryKey(String url) {
if (url == null) throw new IllegalArgumentException("仓库地址为空");
URI uri = URI.create(url.trim());
String path = uri.getRawPath() == null ? "" : uri.getRawPath().replaceAll("/$", "");
if (!"https".equalsIgnoreCase(uri.getScheme())
|| !"github.com".equalsIgnoreCase(uri.getHost())
|| uri.getUserInfo() != null
|| (uri.getPort() != -1 && uri.getPort() != 443)
|| uri.getQuery() != null
|| uri.getFragment() != null
|| !path.matches("/[A-Za-z0-9_.-]+/[A-Za-z0-9_.-]+"))
throw new IllegalArgumentException("必须是 GitHub 仓库根地址");
if (path.endsWith(".git")) path = path.substring(0, path.length() - 4);
String[] parts = path.substring(1).split("/");
if (parts.length != 2
|| parts[0].equals(".")
|| parts[0].equals("..")
|| parts[1].equals(".")
|| parts[1].equals("..")) throw new IllegalArgumentException("仓库地址不合法");
return path.substring(1).toLowerCase(java.util.Locale.ROOT);
}
}
@@ -0,0 +1,13 @@
package com.linkerbot.community.service.migration;
import com.linkerbot.community.service.config.AdminProperties;
import org.springframework.context.annotation.*;
@Configuration
public class MigrationConfiguration {
@Bean
public GitHubSourceChecks gitHubSourceChecks(AdminProperties properties) {
return new GitHubSourceChecks(properties.getGithubToken());
}
}
@@ -0,0 +1,9 @@
package com.linkerbot.community.service.migration;
import com.linkerbot.community.common.BusinessException;
public class MigrationFailure extends BusinessException {
public MigrationFailure(String code, String message) {
super(409, code, message);
}
}
@@ -0,0 +1,126 @@
package com.linkerbot.community.service.migration;
import com.linkerbot.community.service.gitea.*;
import java.io.IOException;
import java.time.Duration;
/** 不依赖 Web、Excel 或状态存储;迁移收据在任何后续远端写入前持久化。 */
public final class RepositoryMigration {
public interface Progress {
GiteaClient.RemoteRepository receipt();
String targetSha();
void receipt(GiteaClient.RemoteRepository receipt) throws IOException;
void targetSha(String sha) throws IOException;
void phase(String phase) throws IOException;
}
public record Result(GiteaClient.RemoteRepository remote, String sha) {}
private final GiteaClient gitea;
private final String organization;
public RepositoryMigration(GiteaClient gitea, String organization) {
this.gitea = gitea;
this.organization = organization;
}
public Result run(
String sourceUrl,
String originalUrl,
String branch,
String name,
Duration timeout,
Progress progress)
throws IOException {
if (progress.receipt() == null) {
ensureAbsent(name);
progress.phase("MIGRATING");
GiteaClient.RemoteRepository receipt = gitea.migrate(sourceUrl + ".git", name, timeout);
progress.receipt(receipt);
}
GiteaClient.RemoteRepository remote = gitea.inspect(name);
validate(remote, sourceUrl, originalUrl, name, progress.receipt());
progress.phase("NORMALIZING");
String sha = gitea.branch(remote.repository(), branch);
if (progress.targetSha() != null && !progress.targetSha().equals(sha))
throw new MigrationFailure("SOURCE_BRANCH_CHANGED", "目标仓库的源分支已变化,禁止改写冻结版本");
if (progress.targetSha() == null) progress.targetSha(sha);
String main;
try {
main = gitea.branch(remote.repository(), "main");
} catch (GiteaFailure missing) {
if (missing.status() != 404) throw missing;
gitea.createBranch(remote.repository(), "main", sha);
main = gitea.branch(remote.repository(), "main");
}
if (!sha.equals(main))
throw new MigrationFailure("MAIN_BRANCH_CONFLICT", "main与原默认分支内容不同,禁止覆盖");
if (!"main".equals(remote.defaultBranch()))
gitea.defaultBranch(remote.repository(), "main");
progress.phase("BRANCH_READY");
return new Result(remote, sha);
}
/** 结果不明时只读核对,绝不重发分支写入。 */
public Result verifyNormalized(
String sourceUrl,
String branch,
String name,
GiteaClient.RemoteRepository receipt,
String sha) {
GiteaClient.RemoteRepository remote = gitea.inspect(name);
validate(remote, sourceUrl, sourceUrl, name, receipt);
if (sha == null
|| !"main".equals(remote.defaultBranch())
|| !sha.equals(gitea.branch(remote.repository(), branch))
|| !sha.equals(gitea.branch(remote.repository(), "main")))
throw new MigrationFailure("NORMALIZATION_NOT_CONFIRMED", "分支适配结果尚未确认,禁止重新写入");
return new Result(remote, sha);
}
public GiteaClient.RemoteRepository verifyReceipt(
String source, String name, GiteaClient.RemoteRepository receipt) {
GiteaClient.RemoteRepository remote = gitea.inspect(name);
validate(remote, source, source, name, receipt);
return remote;
}
public void ensureAbsent(String name) {
try {
gitea.inspect(name);
} catch (GiteaFailure missing) {
if (missing.status() == 404) return;
throw missing;
}
throw new MigrationFailure("REMOTE_ALREADY_EXISTS", "目标仓库已存在但没有成功收据,需人工核对");
}
private void validate(
GiteaClient.RemoteRepository remote,
String source,
String original,
String name,
GiteaClient.RemoteRepository receipt) {
String key;
try {
key = GitHubSourceChecks.repositoryKey(remote.originalUrl());
} catch (RuntimeException error) {
throw new MigrationFailure("SOURCE_CONFLICT", "目标仓库缺少合法来源");
}
if (!organization.equals(remote.owner())
|| !name.equals(remote.name())
|| !remote.privateRepo()
|| remote.mirror()
|| remote.empty()
|| !(key.equals(GitHubSourceChecks.repositoryKey(source))
|| key.equals(GitHubSourceChecks.repositoryKey(original))))
throw new MigrationFailure("REPOSITORY_CONFLICT", "目标仓库归属、来源或可见性不符合要求");
if (receipt == null || remote.id() != receipt.id())
throw new MigrationFailure("REPOSITORY_REPLACED", "目标仓库与迁移收据不一致");
}
}
@@ -0,0 +1,7 @@
package com.linkerbot.community.service.support;
public final class OfficialUsers {
public static final String EMAIL = "community-official@internal.invalid";
private OfficialUsers() {}
}
@@ -0,0 +1,43 @@
package com.linkerbot.community.service.support;
import com.linkerbot.community.bean.entity.User;
import com.linkerbot.community.service.UserService;
/** 仅由已鉴权的管理端入口设置;不从请求参数推断管理员身份。 */
public final class ProjectActors {
private static final ThreadLocal<AdminActor> ADMIN = new ThreadLocal<>();
private ProjectActors() {}
public record AdminActor(String userCode, String nickname) {}
public record Author(String code, String nickname) {}
public static AdminActor admin() {
return ADMIN.get();
}
public static boolean isAdmin() {
return admin() != null;
}
public static Scope enter(AdminActor actor) {
AdminActor previous = ADMIN.get();
ADMIN.set(actor);
return () -> {
if (previous == null) ADMIN.remove();
else ADMIN.set(previous);
};
}
public interface Scope extends AutoCloseable {
@Override
void close();
}
public static Author author(UserService users, long userId) {
if (isAdmin()) return new Author(admin().userCode(), admin().nickname());
User user = users.require(userId);
return new Author(user.getUserCode(), user.getNickname());
}
}
@@ -1,20 +1,79 @@
package com.linkerbot.community.service.support;
import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper;
import com.baomidou.mybatisplus.core.conditions.update.LambdaUpdateWrapper;
import com.linkerbot.community.bean.entity.Project;
import com.linkerbot.community.common.BusinessException;
import com.linkerbot.community.service.gitea.GiteaFailure;
import com.linkerbot.community.service.mapper.ProjectMapper;
import com.zaxxer.hikari.HikariConfig;
import com.zaxxer.hikari.HikariDataSource;
import jakarta.annotation.PreDestroy;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;
import java.util.Set;
import java.sql.Connection;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.util.*;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.locks.ReentrantLock;
import java.util.function.Supplier;
import javax.sql.DataSource;
/** 数据库命名锁不持有业务事务;专用连接在整个项目操作期间保持占用。 */
@Component
public class ProjectLocks {
private static final org.slf4j.Logger log =
org.slf4j.LoggerFactory.getLogger(ProjectLocks.class);
private final ReentrantLock[] locks = new ReentrantLock[128];
private final Set<String> uncertain = ConcurrentHashMap.newKeySet();
private final ThreadLocal<Set<String>> held = ThreadLocal.withInitial(HashSet::new);
private final ThreadLocal<Map<String, Connection>> connections =
ThreadLocal.withInitial(HashMap::new);
private final ThreadLocal<Set<String>> writing = ThreadLocal.withInitial(HashSet::new);
private final ProjectMapper projects;
private final HikariDataSource lockPool;
private final String namespace;
/** 供不启动 Spring 的单元测试使用。生产构造器不会降级为本地锁。 */
public ProjectLocks() {
this.projects = null;
this.lockPool = null;
this.namespace = "unit";
initialize();
}
@Autowired
public ProjectLocks(DataSource source, ProjectMapper projects) {
this.projects = projects;
if (!(source instanceof HikariDataSource hikari))
throw new IllegalStateException("项目锁需要 Hikari 数据源");
String database = hikari.getJdbcUrl().split("\\?", 2)[0];
this.namespace =
Integer.toHexString(database.substring(database.lastIndexOf('/') + 1).hashCode());
if (hikari.getJdbcUrl().startsWith("jdbc:h2:")) {
this.lockPool = null; // H2 工作流测试;MySQL 互斥由真实数据库集成测试覆盖。
} else {
HikariConfig config = new HikariConfig();
config.setJdbcUrl(hikari.getJdbcUrl());
config.setUsername(hikari.getUsername());
config.setPassword(hikari.getPassword());
config.setMaximumPoolSize(8);
config.setMinimumIdle(0);
config.setConnectionTimeout(5000);
config.setPoolName("ProjectLockPool");
this.lockPool = new HikariDataSource(config);
}
initialize();
}
private void initialize() {
for (int i = 0; i < locks.length; i++) locks[i] = new ReentrantLock();
}
@@ -22,24 +81,135 @@ public class ProjectLocks {
return serial(
code,
() -> {
if (uncertain.contains(code))
throw new BusinessException(
409, "REMOTE_RESULT_UNKNOWN", "上次仓库写入结果待人工核对,暂不能继续写入");
return action.get();
if (writing.get().contains(code)) return action.get();
Project guard = guard(code);
if (uncertain.contains(code)
|| guard != null && !"IDLE".equals(guard.getRepoWriteStatus())) {
markUncertain(code);
throw blocked();
}
state(code, "WRITING");
writing.get().add(code);
try {
T result = action.get();
Connection connection = connections.get().get(code);
if (connection != null) {
try {
if (!connection.isValid(1)) throw new SQLException("持锁连接已断开");
} catch (SQLException error) {
throw new BusinessException(
503, "PROJECT_LOCK_UNAVAILABLE", "持锁连接已断开,请核对操作结果");
}
}
if (!uncertain.contains(code)) state(code, "IDLE");
return result;
} catch (RuntimeException error) {
boolean unknown =
!(error instanceof BusinessException)
|| error instanceof GiteaFailure remote
&& remote.uncertain()
|| error instanceof BusinessException business
&& business.status() >= 500
&& !(error instanceof GiteaFailure remote
&& !remote.uncertain());
if (unknown || uncertain.contains(code)) markUncertain(code);
else state(code, "IDLE");
throw error;
} finally {
writing.get().remove(code);
if (writing.get().isEmpty()) writing.remove();
}
});
}
public <T> T serial(String code, Supplier<T> action) {
if (held.get().contains(code)) return action.get();
ReentrantLock lock = locks[Math.floorMod(code.hashCode(), locks.length)];
lock.lock();
try {
return action.get();
if (!lock.tryLock(3, TimeUnit.SECONDS))
throw new BusinessException(409, "PROJECT_BUSY", "项目正在处理,请稍后重试");
} catch (InterruptedException error) {
Thread.currentThread().interrupt();
throw new BusinessException(503, "PROJECT_LOCK_INTERRUPTED", "等待项目锁被中断");
}
boolean executed = false;
String name = "community:" + namespace + ":" + code;
try (Connection connection = lockPool == null ? null : lockPool.getConnection()) {
if (connection != null && !lockFunction(connection, "GET_LOCK", name, true))
throw new BusinessException(409, "PROJECT_BUSY", "项目正在处理,请稍后重试");
held.get().add(code);
if (connection != null) connections.get().put(code, connection);
executed = true;
try {
return action.get();
} finally {
held.get().remove(code);
connections.get().remove(code);
if (connections.get().isEmpty()) connections.remove();
if (held.get().isEmpty()) held.remove();
if (connection != null && !lockFunction(connection, "RELEASE_LOCK", name, false))
throw new SQLException("项目锁释放失败");
}
} catch (SQLException error) {
log.warn(
"项目锁连接异常 projectCode={} code=PROJECT_LOCK_UNAVAILABLE sqlState={} errorType={}",
code,
error.getSQLState(),
error.getClass().getSimpleName());
if (executed) markUncertain(code);
throw new BusinessException(503, "PROJECT_LOCK_UNAVAILABLE", "项目锁连接异常,请核对操作结果");
} finally {
lock.unlock();
}
}
private boolean lockFunction(
Connection connection, String function, String name, boolean acquire)
throws SQLException {
try (PreparedStatement statement =
connection.prepareStatement("SELECT " + function + (acquire ? "(?, 3)" : "(?)"))) {
statement.setString(1, name);
try (ResultSet result = statement.executeQuery()) {
return result.next() && result.getInt(1) == 1;
}
}
}
private Project guard(String code) {
return projects == null
? null
: projects.selectOne(
new LambdaQueryWrapper<Project>().eq(Project::getProjectCode, code));
}
private void state(String code, String status) {
if (projects == null) return;
projects.update(
null,
new LambdaUpdateWrapper<Project>()
.eq(Project::getProjectCode, code)
.set(Project::getRepoWriteStatus, status));
}
public void markUncertain(String code) {
uncertain.add(code);
state(code, "REVIEW");
log.warn("仓库写入待核对 projectCode={}", code);
}
public void confirmKnownResult(String code) {
if (!held.get().contains(code)) throw new IllegalStateException("恢复必须持有项目锁");
state(code, "IDLE");
uncertain.remove(code);
log.info("仓库写入结果已核对 projectCode={}", code);
}
private BusinessException blocked() {
return new BusinessException(409, "REMOTE_RESULT_UNKNOWN", "上次仓库写入结果待人工核对,暂不能继续写入");
}
@PreDestroy
public void close() {
if (lockPool != null) lockPool.close();
}
}
@@ -85,6 +85,7 @@ class ProjectCodeAllocationTest {
when(users.require(1)).thenReturn(new User());
return new ProjectServiceImpl(
projects,
mock(ProjectImportTaskMapper.class),
mock(LikeMapper.class),
mock(CommentMapper.class),
users,
@@ -74,7 +74,7 @@ class ProjectForkStateTest {
factory.addBean("forks", executor);
factory.addBean("snapshots", copier);
ProjectLocks locks = new ProjectLocks();
ProjectServiceImpl service = new ProjectServiceImpl(projects, mock(LikeMapper.class), mock(CommentMapper.class), users,
ProjectServiceImpl service = new ProjectServiceImpl(projects, mock(ProjectImportTaskMapper.class), mock(LikeMapper.class), mock(CommentMapper.class), users,
gitea, new CommunityProperties(), locks, new OssResourcePaths(new OssProperties()), factory.getBeanProvider(UrdfResources.class),
factory.getBeanProvider(ForkExecutor.class), factory.getBeanProvider(GitSnapshotCopier.class));
try {
+1026
View File
File diff suppressed because it is too large Load Diff
+87
View File
@@ -0,0 +1,87 @@
# 管理端数据库变更 SQL
以下 SQL 由维护人员手工执行,应用不会自动迁库。已有数据库执行下面的增量 SQL;新建数据库先执行 database.md 中的四表 SQL,然后只创建本文件的任务表,不重复执行 ALTER。
执行前暂停社区端、后台和导入工具的写入并备份数据库。不要对已有列重复执行。所有时间字段由应用写入 UTC。项目来源改为从任务表读取,因此社区端、后台和导入工具升级前都应创建任务表,即使暂不启动后台也需要执行。
## 项目字段收敛
project 只新增 moderation_reason(管理员下架原因)和 repo_write_status(跨进程写入状态)两个字段。下架直接使用现有 publish_status=BLOCKED,不再增加独立治理状态。GitHub来源标识、地址、默认分支和许可证属于迁移任务,统一放在 project_import_task;第一版只支持GitHub,不存平台类型,也不额外区分用户创建与后台创建。
## 已有数据库增加字段
```sql
ALTER TABLE sys_user ADD COLUMN role VARCHAR(16) NOT NULL DEFAULT 'USER' COMMENT '用户角色 USER普通用户 ADMIN管理员 OFFICIAL官方托管账号';
ALTER TABLE sys_user ADD COLUMN password_hash VARCHAR(100) NULL COMMENT '管理员BCrypt密码哈希,普通用户为空';
ALTER TABLE sys_user ADD COLUMN session_version BIGINT NOT NULL DEFAULT 0 COMMENT '登录会话版本,禁用或修改密码时递增';
ALTER TABLE project MODIFY COLUMN publish_status VARCHAR(16) NOT NULL DEFAULT 'UNPUBLISHED' COMMENT '发布状态 UNPUBLISHED未发布 PUBLISHED已发布 BLOCKED管理员下架';
ALTER TABLE project ADD COLUMN moderation_reason VARCHAR(500) NULL COMMENT '管理员下架原因';
ALTER TABLE project ADD COLUMN repo_write_status VARCHAR(16) NOT NULL DEFAULT 'IDLE' COMMENT '仓库写入状态 IDLE空闲 WRITING写入中 REVIEW待核对';
```
## 新建迁移任务表
```sql
CREATE TABLE project_import_task (
id BIGINT NOT NULL AUTO_INCREMENT COMMENT '内部任务主键',
task_code CHAR(32) NOT NULL COMMENT '迁移任务业务编码',
project_code CHAR(32) NOT NULL COMMENT '目标项目业务编码',
operator_user_code CHAR(32) NOT NULL COMMENT '任务创建管理员用户编码',
source_repository_id BIGINT NOT NULL COMMENT 'GitHub稳定仓库标识,用于来源去重,仅内部使用',
source_repository_url VARCHAR(2048) NOT NULL COMMENT 'GitHub仓库规范公开地址',
source_default_branch VARCHAR(255) NOT NULL COMMENT 'GitHub仓库原默认分支,用于分支适配和续跑校验',
source_license VARCHAR(128) NULL COMMENT 'GitHub许可证标识,无法识别时为空',
status VARCHAR(16) NOT NULL DEFAULT 'QUEUED' COMMENT '任务状态 QUEUED排队 RUNNING执行 SUCCEEDED成功 FAILED失败 REVIEW待核对',
phase VARCHAR(32) NOT NULL DEFAULT 'QUEUED' COMMENT '迁移阶段 QUEUED PRECHECK MIGRATING MIGRATED NORMALIZING BRANCH_READY LINKED',
receipt_json TEXT NULL COMMENT '已确认迁移完成收据JSON,仅内部使用,不包含认证凭据',
target_sha VARCHAR(64) NULL COMMENT '迁移后冻结的目标提交SHA',
error_code VARCHAR(64) NULL COMMENT '最近失败错误码',
warnings_json TEXT NOT NULL COMMENT '兼容性提醒JSON数组',
started_at DATETIME(3) NULL COMMENT '最近开始时间UTC',
finished_at DATETIME(3) NULL COMMENT '最近结束时间UTC',
created_at DATETIME(3) NOT NULL COMMENT '创建时间UTC',
updated_at DATETIME(3) NOT NULL COMMENT '更新时间UTC',
PRIMARY KEY (id),
UNIQUE KEY uk_import_task_code (task_code),
UNIQUE KEY uk_import_project (project_code),
UNIQUE KEY uk_import_source (source_repository_id),
KEY idx_import_queue (status, created_at),
CONSTRAINT fk_import_project FOREIGN KEY (project_code) REFERENCES project(project_code),
CONSTRAINT fk_import_operator FOREIGN KEY (operator_user_code) REFERENCES sys_user(user_code)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci COMMENT='后台公开GitHub项目迁移任务';
```
没有管理员账号表、管理员会话表、审计表或仓库状态表。第一位管理员使用初始化命令设置,禁止手写明文密码入库。后续手工修改角色时必须同时递增 session_version,且 ADMIN 必须具有通过 BCrypt 编码的密码。官方托管账号由后台首次创建或迁移项目时自动建立,保留邮箱为 community-official@internal.invalid。
REVIEW 或遗留 WRITING 不应直接改为 IDLE:先核对 Gitea 写入、迁移收据、LFS 完成情况、仓库身份及提交。后台仅为已持久化成功收据的明确阶段提供只读核对后的续跑,不提供强制成功操作。
## 已经执行过上一版管理端 SQL 的数据库
本节只供已经存在旧字段/旧任务表的数据库手工升级;尚未执行旧版 SQL 的数据库不要执行本节。先暂停写入并备份,不自动删除旧字段。
```sql
ALTER TABLE project_import_task ADD COLUMN source_repository_id BIGINT NULL COMMENT 'GitHub稳定仓库标识,用于来源去重,仅内部使用';
ALTER TABLE project_import_task ADD COLUMN source_repository_url VARCHAR(2048) NULL COMMENT 'GitHub仓库规范公开地址';
ALTER TABLE project_import_task ADD COLUMN source_default_branch VARCHAR(255) NULL COMMENT 'GitHub仓库原默认分支,用于分支适配和续跑校验';
ALTER TABLE project_import_task ADD COLUMN source_license VARCHAR(128) NULL COMMENT 'GitHub许可证标识,无法识别时为空';
UPDATE project_import_task t JOIN project p ON p.project_code = t.project_code
SET t.source_repository_id = p.external_repository_id,
t.source_repository_url = p.external_repository_url,
t.source_default_branch = p.external_default_branch,
t.source_license = p.external_license;
-- 检查结果必须为空,才能把三个必需来源字段设为 NOT NULL。
SELECT task_code, project_code FROM project_import_task
WHERE source_repository_id IS NULL OR source_repository_url IS NULL OR source_default_branch IS NULL;
ALTER TABLE project_import_task MODIFY COLUMN source_repository_id BIGINT NOT NULL COMMENT 'GitHub稳定仓库标识,用于来源去重,仅内部使用';
ALTER TABLE project_import_task MODIFY COLUMN source_repository_url VARCHAR(2048) NOT NULL COMMENT 'GitHub仓库规范公开地址';
ALTER TABLE project_import_task MODIFY COLUMN source_default_branch VARCHAR(255) NOT NULL COMMENT 'GitHub仓库原默认分支,用于分支适配和续跑校验';
ALTER TABLE project_import_task ADD UNIQUE KEY uk_import_source (source_repository_id);
UPDATE project SET publish_status = 'BLOCKED' WHERE moderation_status = 'BLOCKED' AND deleted = 0;
ALTER TABLE project MODIFY COLUMN publish_status VARCHAR(16) NOT NULL DEFAULT 'UNPUBLISHED' COMMENT '发布状态 UNPUBLISHED未发布 PUBLISHED已发布 BLOCKED管理员下架';
```
旧版七个字段及其索引可以暂留,代码已不再读写;确认备份及来源回填完成后再清理。特别是只有旧 Excel 批次、没有后台任务的历史项目,其来源需保留原批次清单或另行归档,不能直接删除唯一来源信息。本次不对实际数据库执行增删字段操作。
+75
View File
@@ -0,0 +1,75 @@
# Community 管理端部署与运维
管理端为独立 Spring Boot 应用 `community-adminapi`,默认端口 8994,接口前缀 `/admin-api`。项目文件复用现有 Gitea 私有组织仓库,封面与 URDF 复用 OSS;不需要新建 Gitea 账号或组织。前端对接见 [管理后台接口文档](admin-api.md)。
## 部署与初始化
1. 备份数据库、暂停写入、排空旧进程。已有数据库手工执行 [增量 SQL](admin-database.md),新数据库先执行 [基础四表 SQL](database.md),再创建迁移任务表。应用不自动执行 SQL。社区端也会读取迁移任务,因此暂不启用管理端时仍需创建该表。
2. 更新社区端、管理端和 Excel 导入工具,确保全部写入端使用数据库共享锁。
3. 在 `.env` 或部署环境中提供现有 `DB_*`、`GITEA_*`、`OSS_*`;后台默认关闭 AI。默认环境为 `test`,正式部署显式设置 `SPRING_PROFILES_ACTIVE=prod`。真实 `.env` 不提交,不用示例覆盖已有凭据。
4. 选择一个已有、正常的普通用户,执行初始化命令,交互输入两次管理员密码。密码至少 12 字符、UTF-8 最多 72 字节,不会写入命令参数或日志。该命令只初始化 USER,不用于重置已有管理员密码;成功后进程退出。
5. 正常启动管理端,配置前端代理或后台跨域来源。第一版两个应用分别单实例运行,不能通过负载均衡随机分发同一应用的登录会话。
```bash
bash scripts/run-admin-local.sh --bootstrap-admin-user-code='<已有用户编码>'
bash scripts/run-admin-local.sh
```
如仅初始化管理员、当前没有 OSS 凭据,可以传 `--community.oss.enabled=false`。直接运行 jar 时不会自动读取 `.env`,必须先在部署环境提供连接变量。本地脚本以 ADMIN_SERVER_PORT 设置后台端口,避免共用 `.env` 中的社区 SERVER_PORT 覆盖;直接部署时也应避免传入社区端的 SERVER_PORT。
```bash
SPRING_PROFILES_ACTIVE=prod java -jar community-adminapi/target/community-adminapi-0.0.1-SNAPSHOT.jar
```
| 配置 | 默认值/说明 |
| --- | --- |
| `ADMIN_SERVER_PORT` | 8994;独立于社区端 `SERVER_PORT` |
| `ADMIN_ALLOWED_ORIGINS` | 空;完整前端来源用英文逗号分隔,不使用通配符 |
| `ADMIN_IMPORT_WORKER_ENABLED` | true;维护期间可关闭消费 |
| `ADMIN_IMPORT_CONCURRENCY` | 2,范围 1~4 |
| `ADMIN_MIGRATION_TIMEOUT` | 30m,仅迁移请求使用,普通 Gitea 超时不变 |
| `GITHUB_TOKEN` | 可选,只访问 GitHub 元数据 API,不能填写 Gitea Token |
| `community.token-ttl` | 后台默认 8h,可通过 Spring 配置覆盖 |
日志为 `community-adminapi.log` 和 `community-adminapi-error.log`,沿用 requestId、归档和脱敏规范。写操作记录实际操作用户编码、目标编码、接口动作、HTTP 结果和耗时;关键项目与迁移步骤另有业务日志。
## 账号与权限
复用 sys_user:USER 普通用户、ADMIN 管理员、OFFICIAL 官方托管账号。后台必须同时验证邮箱、密码、正常状态和 ADMIN 角色。管理员及官方账号不允许通过社区无密码入口登录,两个应用的本机 Token 不互通。后台重启后需要重新登录。
每次请求检查角色、状态和会话版本。禁用、修改密码或手工变更角色并递增 session_version 后,旧 Token 失效;恢复状态不会恢复旧会话。第一版无角色修改接口。
连续登录失败按账号 5 次、来源地址 30 次限速,15 分钟后重试;只使用连接来源地址,不信任任意客户端提交的转发 IP 头。
项目不指定归属用户时,自动创建或复用官方账号,保留邮箱 `community-official@internal.invalid`。该地址在账号建立前也禁止社区登录。若已有账号的角色或状态异常,停止创建并人工核对,不接管普通用户。
## 仓库写入与迁移
project 表只新增 moderation_reason 和 repo_write_status,来源标识、地址、默认分支和许可证保存在 project_import_task。成功迁移任务用于项目来源展示与去重,不能按普通日志过期删除。Excel 运维导入仍通过原批次清单保留来源,不伪造后台任务。
全部写入端按项目编码取得同一 MySQL 命名锁,使用固定连接取得、持有和释放,支持内部重入。锁连接池每应用最多 8 个连接,另有普通业务连接池,需要预留数据库容量。远端请求期间不持有数据库长事务;写入前持久化状态,结果不明时重启后仍阻断写入。
上线前确认目标 Gitea 已开启迁移、HTTP Git、LFS,技术账号具有组织仓库写权限。GitHub 预检优先读取 API;受限时严格核对公开网页身份、可见性和默认分支,无法确认时失败。网页回退拿不到的简介、许可证可能为空,需要管理员检查原仓库文件。
迁移成功收据先持久化,再处理分支、关联和就绪状态。迁移保留 Git 历史、分支、标签并请求迁移 LFS;不导入 Issue、PR、Wiki、Release 附件,不递归复制子模块或外链。原默认分支不是 main 时补建 main;已有 main 与源默认分支内容不同则停止,不覆盖。
## 异常核对
- 超时、进程在 MIGRATING 阶段中断或远端结果不明确时进入 REVIEW。禁止自动重发,也不能仅凭仓库存在判定成功。
- REMOTE_RESULT_UNKNOWN 需要按 requestId、项目编码、任务编码核对日志和 Gitea,不应删除目标仓库后直接重跑。
- 已持久化成功收据且停在 MIGRATED 的任务,retry 会只读验证仓库身份、来源等后续跑。
- 停在 NORMALIZING/BRANCH_READY 且有冻结目标 SHA 的任务,retry 会核对已适配分支和收据后续跑;验证不通过继续阻断。
- 无强制成功、直接清除未知写入状态、任务取消接口。不能直接把 REVIEW 改成成功。
- 迁移成功不自动发布,先在 test 验证单个仓库及人工发布,再开放生产入口。
- 停用后台先关闭任务消费、排空正在执行的任务,再停止应用。不要在后台或 Excel 导入仍写入时回退社区端到旧写入逻辑。
逻辑删除、撤回和下架不会物理删除 Gitea 仓库或 OSS 对象;已公开 OSS 链接沿用现有生命周期。
## 验证
```bash
AI_ENABLED=false OSS_ENABLED=false mvn clean verify
bash scripts/test-admin-mysql.sh
```
普通测试使用 H2 和模拟 GitHub/Gitea/OSS,不访问现有业务库或真实仓库。真实 MySQL 测试使用 `.env` 的 DB_* 或 ADMIN_MYSQL_TEST_* 连接配置,只创建/删除随机 `community_admin_verify_*` 临时库;需要创建临时库权限。验证跨实例互斥、重入、持久化阻断、断开持锁连接及新进程恢复,不对已有业务库执行 DDL。
+4 -1
View File
@@ -203,11 +203,13 @@ export interface Project {
projectType: string;
owner: PublicUser;
repoStatus: 'CREATING' | 'READY' | 'FAILED';
publishStatus: 'UNPUBLISHED' | 'PUBLISHED';
publishStatus: 'UNPUBLISHED' | 'PUBLISHED' | 'BLOCKED';
publishedCommitSha: string | null;
publishedAt: UtcDateTime | null;
sourceProjectCode: string | null;
sourceCommitSha: string | null;
sourceRepositoryUrl: string | null; // 在线迁移任务记录的原仓库地址
sourceLicense: string | null; // 原仓库许可证标识,未识别时为 null
likeCount: number;
commentCount: number;
liked: boolean;
@@ -322,6 +324,7 @@ export interface Comment {
| repoStatus = FAILED | 初始化失败,显示失败状态;仍可修改项目信息或删除项目 |
| publishStatus = UNPUBLISHED | 当前不在社区展示;这是私有项目状态,不是草稿 |
| publishStatus = PUBLISHED | 当前在社区展示 |
| publishStatus = BLOCKED | 管理员下架,不在社区展示;普通撤回不会解除此状态,只有管理员可解除 |
| publishedCommitSha / publishedAt | 撤回后可能仍保留上一次发布值,是否公开必须判断 publishStatus |
| sourceProjectCode / sourceCommitSha | Fork 来源及复制的源版本;普通新项目为 null |
| liked | 当前请求用户是否点赞,匿名访问为 false |
+6 -1
View File
@@ -12,6 +12,9 @@ CREATE TABLE sys_user (
email VARCHAR(254) NOT NULL COMMENT '规范化后的登录邮箱',
nickname VARCHAR(100) NOT NULL COMMENT '用户昵称',
status VARCHAR(16) NOT NULL DEFAULT 'ACTIVE' COMMENT '账号状态',
role VARCHAR(16) NOT NULL DEFAULT 'USER' COMMENT '用户角色 USER普通用户 ADMIN管理员 OFFICIAL官方托管账号',
password_hash VARCHAR(100) NULL COMMENT '管理员BCrypt密码哈希,普通用户为空',
session_version BIGINT NOT NULL DEFAULT 0 COMMENT '登录会话版本,禁用或修改密码时递增',
created_at DATETIME(3) NOT NULL COMMENT '创建时间 UTC',
updated_at DATETIME(3) NOT NULL COMMENT '更新时间 UTC',
PRIMARY KEY (id),
@@ -34,9 +37,11 @@ CREATE TABLE project (
gitea_owner VARCHAR(100) NOT NULL COMMENT 'Gitea 仓库所属组织',
gitea_repo_name VARCHAR(100) NOT NULL COMMENT 'Gitea 仓库名称',
repo_status VARCHAR(16) NOT NULL DEFAULT 'CREATING' COMMENT '仓库状态 CREATING READY FAILED',
publish_status VARCHAR(16) NOT NULL DEFAULT 'UNPUBLISHED' COMMENT '社区状态 UNPUBLISHED PUBLISHED',
publish_status VARCHAR(16) NOT NULL DEFAULT 'UNPUBLISHED' COMMENT '发布状态 UNPUBLISHED未发布 PUBLISHED已发布 BLOCKED管理员下架',
published_commit_sha VARCHAR(64) NULL COMMENT '社区已发布 Commit SHA',
published_at DATETIME(3) NULL COMMENT '最近发布时间 UTC',
moderation_reason VARCHAR(500) NULL COMMENT '管理员下架原因',
repo_write_status VARCHAR(16) NOT NULL DEFAULT 'IDLE' COMMENT '仓库写入状态 IDLE空闲 WRITING写入中 REVIEW待核对',
source_project_id BIGINT NULL COMMENT 'Fork 来源项目',
source_commit_sha VARCHAR(64) NULL COMMENT 'Fork 来源已发布 Commit SHA',
deleted TINYINT(1) NOT NULL DEFAULT 0 COMMENT '逻辑删除标识',
+1
View File
@@ -18,6 +18,7 @@
<module>community-security</module>
<module>community-api</module>
<module>community-import</module>
<module>community-adminapi</module>
</modules>
<properties>
<java.version>21</java.version>
+13
View File
@@ -0,0 +1,13 @@
#!/usr/bin/env bash
set -euo pipefail
COMMUNITY_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
if [[ -f "$COMMUNITY_ROOT/.env" ]]; then
set -a
source "$COMMUNITY_ROOT/.env"
set +a
fi
export SERVER_PORT="${ADMIN_SERVER_PORT:-8994}"
export SPRING_PROFILES_ACTIVE="${SPRING_PROFILES_ACTIVE:-test}"
cd "$COMMUNITY_ROOT"
mvn -q -pl community-adminapi -am -DskipTests package
exec java -Djava.awt.headless=true -jar community-adminapi/target/community-adminapi-0.0.1-SNAPSHOT.jar "$@"
+19
View File
@@ -0,0 +1,19 @@
#!/usr/bin/env bash
set -euo pipefail
COMMUNITY_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
if [[ -f "$COMMUNITY_ROOT/.env" ]]; then
set -a
source "$COMMUNITY_ROOT/.env"
set +a
fi
export ADMIN_MYSQL_TEST_URL="${ADMIN_MYSQL_TEST_URL:-${DB_URL:-}}"
export ADMIN_MYSQL_TEST_USERNAME="${ADMIN_MYSQL_TEST_USERNAME:-${DB_USERNAME:-}}"
export ADMIN_MYSQL_TEST_PASSWORD="${ADMIN_MYSQL_TEST_PASSWORD:-${DB_PASSWORD:-}}"
if [[ "$ADMIN_MYSQL_TEST_URL" != jdbc:mysql:* || -z "$ADMIN_MYSQL_TEST_USERNAME" ]]; then
echo '请配置 ADMIN_MYSQL_TEST_URL、ADMIN_MYSQL_TEST_USERNAME、ADMIN_MYSQL_TEST_PASSWORD,或 .env 中的 DB_*。' >&2
exit 1
fi
export AI_ENABLED=false OSS_ENABLED=false
cd "$COMMUNITY_ROOT"
# 只创建和删除随机 community_admin_verify_* 临时库,不修改已有数据库;账号需有创建临时库的权限。
mvn -q -pl community-adminapi -am -Dtest=ProjectLocksMySqlTest -Dsurefire.failIfNoSpecifiedTests=false test