feat: refactor to ymsctl & yms-daemon

This commit is contained in:
2026-08-17 02:10:10 +08:00
parent 7755da72e7
commit 79f18fcdea
23 changed files with 394 additions and 47 deletions
+2
View File
@@ -2,3 +2,5 @@
yms-daemon yms-daemon
/dist/ /dist/
/.build/ /.build/
/coverage.out
/.scannerwork/
+140
View File
@@ -0,0 +1,140 @@
# YMS 命令入口与操作查询方案
> 状态:方案确认,暂不进入代码实现
## 1. 命令职责
命令入口采用 daemon/client 分离,命名参考 `chronyd / chronyc`
| 命令 | 职责 | 是否常驻 |
|---|---|---:|
| `ymsd` | 更新助手服务端,监听 Unix Socket,执行事务和恢复 | 是 |
| `ymsctl` | 运维和交付 CLI,提交更新、查询状态、诊断和回滚 | 否 |
| `yms-gui` | 客户 PC 图形化中转助手,名称后续确定 | 否 |
`ymsd` 不承担交互式更新命令;更新请求由 `ymsctl` 通过 Unix Socket 提交。POC 阶段不对外提供 `serve` 子命令,systemd 的 `ExecStart` 直接启动 `/usr/sbin/ymsd`
## 2. systemd 与安装
systemd unit 名称继续固定为:
```text
yms-daemon.service
```
unit 的启动目标为:
```text
/usr/sbin/ymsd
```
POC 阶段不保留 `yms-daemon` 命令兼容入口,直接采用新命名。更新命令统一使用:
```bash
ymsctl update ...
ymsctl restart ...
```
## 3. ymsctl 命令层级
### 3.1 更新与回滚
```bash
ymsctl update --service backend -f <repack.zip>
ymsctl update --service backend --native-jar <backend.jar>
ymsctl update --service backend --container-image <image:tag>
ymsctl restart --service backend
ymsctl rollback --service backend --transaction <transaction-id>
```
`--native-jar``--container-image` 互斥。`rollback` 必须指定明确的事务 ID 或已提交版本身份,不允许根据目录排序、文件修改时间或镜像 tag 猜测回滚目标。
### 3.2 历史操作查询
```bash
ymsctl list
ymsctl list --limit 50
ymsctl list --state FAILED
ymsctl list --service backend
ymsctl list --json
```
默认显示最近 20 次操作,按创建时间倒序。每条记录至少展示:
- 创建时间和结束时间;
- transaction ID
- service
- operation
- 输入类型和版本或镜像 digest;
- 来源(CLI、PC client、Jenkins、server API);
- 最终状态;
- 失败原因。
`--json` 输出稳定 JSON,供 Jenkins 和 PC client 使用。列表数据只读取 SQLite,不直接扫描 systemd、Docker 或 Nginx 生成历史记录。
### 3.3 当前状态与诊断
```bash
ymsctl status --service backend
ymsctl doctor --service backend
ymsctl reconcile --service backend
ymsctl reconcile --service backend --apply
```
- `status`:读取 SQLite 后校验当前 systemd、Docker、Nginx 和活动槽位状态;
- `doctor`:只读诊断,输出漂移项和建议动作;
- `reconcile`:只读生成修复计划;
- `reconcile --apply`:执行明确授权的修复动作,并创建可审计事务。
## 4. 状态漂移处理原则
SQLite 是事务事实来源,外部系统是待校验运行状态。发现不一致时默认拒绝更新,不自动覆盖现场。
只有以下类型允许自动修复:
1. SQLite 记录的活动容器存在、身份匹配且健康,Nginx 仅指向错误槽位;
2. 非活动槽位容器存在、已停止且没有提交记录;
3. native 活动 JAR 链接与已提交事务明确记录的 release 不一致,且目标文件身份校验通过;
4. 已完成事务的临时文件可以按照事务凭据清理。
以下情况必须进入 `DRIFT_DETECTED`,等待人工确认:
- SQLite 记录的活动容器已经不存在;
- 两个槽位同时运行且无法确定提交归属;
- SQLite 没有部署记录,但现场已经存在容器或运行中的 native 服务;
- Nginx 配置无法验证或存在非 daemon 管理的冲突;
- systemd unit、容器镜像 digest、JAR SHA-256 与事务记录不一致。
任何修复动作都必须先写入事务,再执行外部变更;修复失败时沿用现有回滚和恢复机制。
## 5. systemd 服务与 ymsd 的边界
`ymsd` 负责:
- 持有进程锁;
- 打开 SQLite
- 恢复未完成事务;
- 调用 native/container 执行器;
- 通过 Unix Socket 提供请求和进度响应;
- 写入 stdout 和本地日志文件。
`ymsctl` 负责:
- 参数校验;
- 提交请求;
- 展示进度;
- 查询历史和当前状态;
- 发起显式诊断和修复。
`ymsctl` 不直接写 SQLite,不直接修改 Nginx,不直接调用 Docker 或 systemd。
## 6. 实施顺序
1. 增加 SQLite 最近操作查询接口;
2. 实现 `ymsctl list` 和 JSON 输出;
3. 拆分 `ymsd``ymsctl` 的编译入口;
4. 安装 `ymsd``ymsctl` 和兼容入口 `yms-daemon`
5. 实现 `status` 只读一致性检查;
6. 实现 `doctor` 漂移报告;
7. 实现带事务审计的 `reconcile --apply`
8. 最后确定 GUI 名称和 PC client 的调用协议。
+1 -1
View File
@@ -52,7 +52,7 @@ host Nginx 当前指向 8080 时,`backend-8080` 必须正在运行;当前指
测试命令: 测试命令:
```bash ```bash
yms-daemon update \ ymsctl update \
--service backend \ --service backend \
--container-image harbor.ymswell.asia/ymswell/glory-ymswell:20260813-184902-a37bf50d-v1.1.8.1 --container-image harbor.ymswell.asia/ymswell/glory-ymswell:20260813-184902-a37bf50d-v1.1.8.1
``` ```
+8 -7
View File
@@ -38,7 +38,7 @@ RPM 不会自动启动或重启 `yms-daemon.service`,避免安装恰好发生
systemctl restart yms-daemon.service systemctl restart yms-daemon.service
``` ```
`/etc/yms-daemon/yms-daemon.toml` 使用 `noreplace` 语义,升级不会覆盖现场配置。RPM 会安装 `/usr/bin/yms-daemon`、两个 systemd unit、tmpfiles 配置,并创建 `/home/yms/dump` `/etc/yms-daemon/yms-daemon.toml` 使用 `noreplace` 语义,升级不会覆盖现场配置。RPM 会安装 `/usr/bin/ymsd``/usr/bin/ymsctl`、两个 systemd unit、tmpfiles 配置,并创建 `/home/yms/dump`
RPM 示例配置使用客户环境值: RPM 示例配置使用客户环境值:
@@ -69,7 +69,8 @@ CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go build -trimpath -o yms-daemon .
把构建结果和 `packaging` 目录传到服务器后,以 root 执行: 把构建结果和 `packaging` 目录传到服务器后,以 root 执行:
```bash ```bash
install -m 0755 yms-daemon /usr/bin/yms-daemon install -m 0755 ymsd /usr/bin/ymsd
install -m 0755 ymsctl /usr/bin/ymsctl
install -d -m 0755 /etc/yms-daemon install -d -m 0755 /etc/yms-daemon
install -m 0640 packaging/etc/yms-daemon/yms-daemon.toml /etc/yms-daemon/yms-daemon.toml install -m 0640 packaging/etc/yms-daemon/yms-daemon.toml /etc/yms-daemon/yms-daemon.toml
install -m 0644 packaging/systemd/yms-daemon.service /etc/systemd/system/yms-daemon.service install -m 0644 packaging/systemd/yms-daemon.service /etc/systemd/system/yms-daemon.service
@@ -95,7 +96,7 @@ curl -fsS http://127.0.0.1:8081/yms/actuator/health
客户 repack ZIP 模式: 客户 repack ZIP 模式:
```bash ```bash
yms-daemon update --service backend -f /home/yms/tmp/<完整包名>.zip ymsctl update --service backend -f /home/yms/tmp/<完整包名>.zip
``` ```
ZIP 根目录必须包含: ZIP 根目录必须包含:
@@ -116,7 +117,7 @@ selectedType = native
开发环境不需要打 ZIP。Jenkins 或交付人员先把完整 JAR 复制到 `/home/yms/tmp`,再执行: 开发环境不需要打 ZIP。Jenkins 或交付人员先把完整 JAR 复制到 `/home/yms/tmp`,再执行:
```bash ```bash
yms-daemon update --service backend --native-jar /home/yms/tmp/<完整JAR文件名>.jar ymsctl update --service backend --native-jar /home/yms/tmp/<完整JAR文件名>.jar
``` ```
`-f``--native-jar` 必须且只能提供一个。daemon 不根据文件扩展名选择更新模式。直传 JAR 会被完整校验并计算 SHA-256,然后保存到: `-f``--native-jar` 必须且只能提供一个。daemon 不根据文件扩展名选择更新模式。直传 JAR 会被完整校验并计算 SHA-256,然后保存到:
@@ -147,7 +148,7 @@ transaction=<事务ID> state=COMMITTED
Jenkins 只需要退出码、不需要过程和成功结果时,使用精确参数 `--quite` Jenkins 只需要退出码、不需要过程和成功结果时,使用精确参数 `--quite`
```bash ```bash
yms-daemon update --service backend --native-jar /home/yms/tmp/<完整JAR文件名>.jar --quite ymsctl update --service backend --native-jar /home/yms/tmp/<完整JAR文件名>.jar --quite
``` ```
`--quite` 不写正常过程和成功结果;失败信息仍写入 stderr,并返回非零退出码。 `--quite` 不写正常过程和成功结果;失败信息仍写入 stderr,并返回非零退出码。
@@ -169,7 +170,7 @@ journalctl -u yms-daemon.service -f
修改 `/home/yms/bin/env/yms.env` 等 backend 启动配置后,执行: 修改 `/home/yms/bin/env/yms.env` 等 backend 启动配置后,执行:
```bash ```bash
yms-daemon restart --service backend ymsctl restart --service backend
``` ```
daemon 读取 `/home/yms/lib/glory-soft-yms.jar` 当前指向的精确 release,创建独立 restart 事务,将同一 JAR 绑定到非活动槽位,启动并完成 Actuator 健康检查后切流,等待 drain,再停止旧槽。当前 JAR 已经位于 release 目录时直接复用;兼容入口仍是普通文件时,先按内容身份导入 release 目录。restart 不重新上传 JAR,也不以历史 update 的内容幂等键阻止本次轮转。 daemon 读取 `/home/yms/lib/glory-soft-yms.jar` 当前指向的精确 release,创建独立 restart 事务,将同一 JAR 绑定到非活动槽位,启动并完成 Actuator 健康检查后切流,等待 drain,再停止旧槽。当前 JAR 已经位于 release 目录时直接复用;兼容入口仍是普通文件时,先按内容身份导入 release 目录。restart 不重新上传 JAR,也不以历史 update 的内容幂等键阻止本次轮转。
@@ -177,7 +178,7 @@ daemon 读取 `/home/yms/lib/glory-soft-yms.jar` 当前指向的精确 release
静默执行: 静默执行:
```bash ```bash
yms-daemon restart --service backend --quite ymsctl restart --service backend --quite
``` ```
restart 进行中如果 daemon 重启,再次执行同一命令会恢复唯一未完成的 backend restart 事务。存在其他未完成 update 事务时,restart 拒绝创建新事务并返回该活动事务 ID。 restart 进行中如果 daemon 重启,再次执行同一命令会恢复唯一未完成的 backend restart 事务。存在其他未完成 update 事务时,restart 拒绝创建新事务并返回该活动事务 ID。
+18
View File
@@ -3,6 +3,7 @@
package backendexecutor package backendexecutor
import ( import (
"bufio"
"context" "context"
"encoding/json" "encoding/json"
"errors" "errors"
@@ -62,6 +63,8 @@ type Request struct {
ConfigLocation string ConfigLocation string
RestartPolicy containerengine.RestartPolicy RestartPolicy containerengine.RestartPolicy
HealthEndpoint string HealthEndpoint string
StartLog bool
LogReporter func(string)
} }
// Executor drives the persisted transaction up to SWITCHING after the new container is healthy. // Executor drives the persisted transaction up to SWITCHING after the new container is healthy.
@@ -210,6 +213,21 @@ func (e *Executor) startAndCheck(ctx context.Context, transactionID string, requ
if _, err := e.coordinator.ExecuteStep(ctx, transactionID, startIntent(request), startOperation); err != nil { if _, err := e.coordinator.ExecuteStep(ctx, transactionID, startIntent(request), startOperation); err != nil {
return err return err
} }
if request.StartLog && request.LogReporter != nil {
logs, err := e.engine.ContainerLogs(ctx, request.ContainerName)
if err == nil {
scanner := bufio.NewScanner(logs)
for scanner.Scan() {
request.LogReporter(scanner.Text())
}
_ = logs.Close()
if err := scanner.Err(); err != nil {
return fmt.Errorf("read container startup logs: %w", err)
}
} else {
request.LogReporter("unable to read container startup logs: " + err.Error())
}
}
healthOperation := &healthOperation{ healthOperation := &healthOperation{
engine: e.engine, engine: e.engine,
checker: e.checker, checker: e.checker,
@@ -10,6 +10,7 @@ import (
"os" "os"
"path/filepath" "path/filepath"
"slices" "slices"
"strings"
"sync" "sync"
"testing" "testing"
@@ -408,6 +409,10 @@ func (e *fakeEngine) StartContainer(_ context.Context, name string) error {
return nil return nil
} }
func (e *fakeEngine) ContainerLogs(context.Context, string) (io.ReadCloser, error) {
return io.NopCloser(strings.NewReader("")), nil
}
func (e *fakeEngine) StopContainer(_ context.Context, name string) error { func (e *fakeEngine) StopContainer(_ context.Context, name string) error {
e.mu.Lock() e.mu.Lock()
defer e.mu.Unlock() defer e.mu.Unlock()
+7 -1
View File
@@ -33,6 +33,7 @@ type persistedContainerRequest struct {
PreviousPort int `json:"previousPort"` PreviousPort int `json:"previousPort"`
PreviousContainer string `json:"previousContainer"` PreviousContainer string `json:"previousContainer"`
TargetHealthEndpoint string `json:"targetHealthEndpoint"` TargetHealthEndpoint string `json:"targetHealthEndpoint"`
StartLog bool `json:"startLog"`
GatewayBeforePath string `json:"gatewayBeforePath"` GatewayBeforePath string `json:"gatewayBeforePath"`
GatewayAfterPath string `json:"gatewayAfterPath"` GatewayAfterPath string `json:"gatewayAfterPath"`
GatewayReceiptPath string `json:"gatewayReceiptPath"` GatewayReceiptPath string `json:"gatewayReceiptPath"`
@@ -40,7 +41,7 @@ type persistedContainerRequest struct {
// UpdateContainerImage pulls one development image, freezes its repository // UpdateContainerImage pulls one development image, freezes its repository
// digest, and updates the inactive Docker backend slot. // digest, and updates the inactive Docker backend slot.
func (u *Updater) UpdateContainerImage(ctx context.Context, imageReference string, report ProgressReporter) (transaction.Transaction, error) { func (u *Updater) UpdateContainerImage(ctx context.Context, imageReference string, startLog bool, report ProgressReporter) (transaction.Transaction, error) {
if u.containerExecutor == nil || u.engine == nil { if u.containerExecutor == nil || u.engine == nil {
return transaction.Transaction{}, errors.New("container backend updater is not configured") return transaction.Transaction{}, errors.New("container backend updater is not configured")
} }
@@ -138,6 +139,7 @@ func (u *Updater) UpdateContainerImage(ctx context.Context, imageReference strin
TargetPort: targetPort, TargetContainer: targetSlot.ContainerName, TargetPort: targetPort, TargetContainer: targetSlot.ContainerName,
PreviousPort: before.ActivePort, PreviousContainer: previousContainer, PreviousPort: before.ActivePort, PreviousContainer: previousContainer,
TargetHealthEndpoint: targetSlot.HealthEndpoint, TargetHealthEndpoint: targetSlot.HealthEndpoint,
StartLog: startLog,
GatewayBeforePath: beforePath, GatewayAfterPath: afterPath, GatewayBeforePath: beforePath, GatewayAfterPath: afterPath,
GatewayReceiptPath: filepath.Join(transactionRoot, "gateway.applied"), GatewayReceiptPath: filepath.Join(transactionRoot, "gateway.applied"),
} }
@@ -294,6 +296,10 @@ func (u *Updater) runContainerUpdate(ctx context.Context, record transaction.Tra
ConfigLocation: deploymentconfig.ContainerConfigLocation, ConfigLocation: deploymentconfig.ContainerConfigLocation,
RestartPolicy: containerengine.RestartPolicy{Name: "no"}, RestartPolicy: containerengine.RestartPolicy{Name: "no"},
HealthEndpoint: request.TargetHealthEndpoint, HealthEndpoint: request.TargetHealthEndpoint,
StartLog: request.StartLog,
LogReporter: func(line string) {
reportProgress(report, Progress{TransactionID: record.ID, State: transaction.StateStarting, Message: "CONTAINER LOG " + line})
},
} }
switch record.State { switch record.State {
case transaction.StateCreated, transaction.StateValidating, transaction.StatePrepared, transaction.StateStarting: case transaction.StateCreated, transaction.StateValidating, transaction.StatePrepared, transaction.StateStarting:
+11 -7
View File
@@ -31,7 +31,7 @@ func TestContainerUpdaterPullsSwitchesAndStopsPreviousSlot(t *testing.T) {
}, },
}, 0) }, 0)
record, err := updater.UpdateContainerImage(ctx, containerTestImage, nil) record, err := updater.UpdateContainerImage(ctx, containerTestImage, true, nil)
if err != nil { if err != nil {
t.Fatalf("update container backend: %v", err) t.Fatalf("update container backend: %v", err)
} }
@@ -61,7 +61,7 @@ func TestContainerUpdaterFirstInstallCreatesInactiveSlotBeforeSwitch(t *testing.
return nil return nil
} }
record, err := updater.UpdateContainerImage(ctx, containerTestImage, func(item Progress) { record, err := updater.UpdateContainerImage(ctx, containerTestImage, true, func(item Progress) {
progress = append(progress, item) progress = append(progress, item)
}) })
if err != nil { if err != nil {
@@ -99,7 +99,7 @@ func TestContainerUpdaterRejectsMissingActiveWithPresentInactive(t *testing.T) {
}, },
}, 0) }, 0)
_, err := updater.UpdateContainerImage(context.Background(), containerTestImage, nil) _, err := updater.UpdateContainerImage(context.Background(), containerTestImage, true, nil)
if err == nil || !strings.Contains(err.Error(), "active backend container backend-8080 is missing but inactive container backend-8081 is running") { if err == nil || !strings.Contains(err.Error(), "active backend container backend-8080 is missing but inactive container backend-8081 is running") {
t.Fatalf("unexpected missing-active result: %v", err) t.Fatalf("unexpected missing-active result: %v", err)
} }
@@ -116,7 +116,7 @@ func TestContainerUpdaterRecoversFailedRollbackThenRetriesSameImage(t *testing.T
return nil return nil
} }
rollingBack, err := updater.UpdateContainerImage(ctx, containerTestImage, nil) rollingBack, err := updater.UpdateContainerImage(ctx, containerTestImage, true, nil)
if err == nil || rollingBack.State != transaction.StateRollingBack { if err == nil || rollingBack.State != transaction.StateRollingBack {
t.Fatalf("unexpected failed rollback result: record=%+v err=%v", rollingBack, err) t.Fatalf("unexpected failed rollback result: record=%+v err=%v", rollingBack, err)
} }
@@ -126,12 +126,12 @@ func TestContainerUpdaterRecoversFailedRollbackThenRetriesSameImage(t *testing.T
} }
failGateway = false failGateway = false
rolledBack, err := updater.UpdateContainerImage(ctx, containerTestImage, nil) rolledBack, err := updater.UpdateContainerImage(ctx, containerTestImage, true, nil)
if err == nil || rolledBack.State != transaction.StateRolledBack { if err == nil || rolledBack.State != transaction.StateRolledBack {
t.Fatalf("unexpected resumed rollback result: record=%+v err=%v", rolledBack, err) t.Fatalf("unexpected resumed rollback result: record=%+v err=%v", rolledBack, err)
} }
committed, err := updater.UpdateContainerImage(ctx, containerTestImage, nil) committed, err := updater.UpdateContainerImage(ctx, containerTestImage, true, nil)
if err != nil || committed.State != transaction.StateCommitted { if err != nil || committed.State != transaction.StateCommitted {
t.Fatalf("retry same image after rollback: record=%+v err=%v", committed, err) t.Fatalf("retry same image after rollback: record=%+v err=%v", committed, err)
} }
@@ -169,7 +169,7 @@ func TestContainerUpdaterRejectsMissingCommittedContainer(t *testing.T) {
t.Fatalf("commit previous backend deployment: %v", err) t.Fatalf("commit previous backend deployment: %v", err)
} }
_, err = updater.UpdateContainerImage(ctx, containerTestImage, nil) _, err = updater.UpdateContainerImage(ctx, containerTestImage, true, nil)
if err == nil || !strings.Contains(err.Error(), "committed active backend container backend-8080 is missing") { if err == nil || !strings.Contains(err.Error(), "committed active backend container backend-8080 is missing") {
t.Fatalf("unexpected committed-container drift result: %v", err) t.Fatalf("unexpected committed-container drift result: %v", err)
} }
@@ -336,6 +336,10 @@ func (e *containerUpdateEngine) StartContainer(_ context.Context, name string) e
e.containers[name] = record e.containers[name] = record
return nil return nil
} }
func (e *containerUpdateEngine) ContainerLogs(context.Context, string) (io.ReadCloser, error) {
return io.NopCloser(strings.NewReader("")), nil
}
func (e *containerUpdateEngine) StopContainer(_ context.Context, name string) error { func (e *containerUpdateEngine) StopContainer(_ context.Context, name string) error {
e.stopped = append(e.stopped, name) e.stopped = append(e.stopped, name)
record, found := e.containers[name] record, found := e.containers[name]
+1
View File
@@ -79,6 +79,7 @@ type Engine interface {
InspectImage(context.Context, string) (Image, error) InspectImage(context.Context, string) (Image, error)
CreateContainer(context.Context, ContainerSpec) (Container, error) CreateContainer(context.Context, ContainerSpec) (Container, error)
StartContainer(context.Context, string) error StartContainer(context.Context, string) error
ContainerLogs(context.Context, string) (io.ReadCloser, error)
StopContainer(context.Context, string) error StopContainer(context.Context, string) error
InspectContainer(context.Context, string) (Container, error) InspectContainer(context.Context, string) (Container, error)
RemoveContainer(context.Context, string, bool) error RemoveContainer(context.Context, string, bool) error
+15
View File
@@ -1,6 +1,7 @@
package containerengine package containerengine
import ( import (
"bytes"
"context" "context"
"encoding/json" "encoding/json"
"errors" "errors"
@@ -8,6 +9,7 @@ import (
"io" "io"
cerrdefs "github.com/containerd/errdefs" cerrdefs "github.com/containerd/errdefs"
"github.com/moby/moby/api/pkg/stdcopy"
"github.com/moby/moby/api/types/container" "github.com/moby/moby/api/types/container"
"github.com/moby/moby/api/types/jsonstream" "github.com/moby/moby/api/types/jsonstream"
"github.com/moby/moby/api/types/mount" "github.com/moby/moby/api/types/mount"
@@ -148,6 +150,19 @@ func (e *MobyEngine) StartContainer(ctx context.Context, idOrName string) error
return nil return nil
} }
func (e *MobyEngine) ContainerLogs(ctx context.Context, idOrName string) (io.ReadCloser, error) {
stream, err := e.client.ContainerLogs(ctx, idOrName, client.ContainerLogsOptions{ShowStdout: true, ShowStderr: true, Tail: "all"})
if err != nil {
return nil, engineError("read container logs", err)
}
defer stream.Close()
var output bytes.Buffer
if _, err := stdcopy.StdCopy(&output, &output, stream); err != nil {
return nil, fmt.Errorf("decode container logs: %w", err)
}
return io.NopCloser(bytes.NewReader(output.Bytes())), nil
}
func (e *MobyEngine) StopContainer(ctx context.Context, idOrName string) error { func (e *MobyEngine) StopContainer(ctx context.Context, idOrName string) error {
if _, err := e.client.ContainerStop(ctx, idOrName, client.ContainerStopOptions{}); err != nil { if _, err := e.client.ContainerStop(ctx, idOrName, client.ContainerStopOptions{}); err != nil {
return engineError("stop container", err) return engineError("stop container", err)
+1
View File
@@ -17,6 +17,7 @@ type Request struct {
InputType string `json:"inputType"` InputType string `json:"inputType"`
File string `json:"file"` File string `json:"file"`
ImageReference string `json:"imageReference"` ImageReference string `json:"imageReference"`
StartLog bool `json:"startLog"`
} }
type Response struct { type Response struct {
+2 -2
View File
@@ -20,14 +20,14 @@ func Update(ctx context.Context, socketPath string, service string, inputType st
return submit(ctx, socketPath, request, progress) return submit(ctx, socketPath, request, progress)
} }
func UpdateContainerImage(ctx context.Context, socketPath string, service string, imageReference string, progress func(daemonapi.Response)) (daemonapi.Response, error) { func UpdateContainerImage(ctx context.Context, socketPath string, service string, imageReference string, startLog bool, progress func(daemonapi.Response)) (daemonapi.Response, error) {
if !filepath.IsAbs(socketPath) { if !filepath.IsAbs(socketPath) {
return daemonapi.Response{}, errors.New("daemon socket path must be absolute") return daemonapi.Response{}, errors.New("daemon socket path must be absolute")
} }
if imageReference == "" { if imageReference == "" {
return daemonapi.Response{}, errors.New("container image reference is required") return daemonapi.Response{}, errors.New("container image reference is required")
} }
request := daemonapi.Request{Operation: daemonapi.OperationUpdate, Service: service, InputType: daemonapi.InputTypeContainerImage, ImageReference: imageReference} request := daemonapi.Request{Operation: daemonapi.OperationUpdate, Service: service, InputType: daemonapi.InputTypeContainerImage, ImageReference: imageReference, StartLog: startLog}
return submit(ctx, socketPath, request, progress) return submit(ctx, socketPath, request, progress)
} }
+2 -2
View File
@@ -25,7 +25,7 @@ const maximumRequestBytes = 1 << 20
type backendUpdater interface { type backendUpdater interface {
UpdateRepack(context.Context, string, backendupdate.ProgressReporter) (transaction.Transaction, error) UpdateRepack(context.Context, string, backendupdate.ProgressReporter) (transaction.Transaction, error)
UpdateNativeJAR(context.Context, string, backendupdate.ProgressReporter) (transaction.Transaction, error) UpdateNativeJAR(context.Context, string, backendupdate.ProgressReporter) (transaction.Transaction, error)
UpdateContainerImage(context.Context, string, backendupdate.ProgressReporter) (transaction.Transaction, error) UpdateContainerImage(context.Context, string, bool, backendupdate.ProgressReporter) (transaction.Transaction, error)
Restart(context.Context, backendupdate.ProgressReporter) (transaction.Transaction, error) Restart(context.Context, backendupdate.ProgressReporter) (transaction.Transaction, error)
} }
@@ -141,7 +141,7 @@ func (s *Server) handle(ctx context.Context, connection net.Conn) {
_ = s.writeResponse(connection, daemonapi.Response{Kind: daemonapi.ResponseResult, Error: "container-image requires imageReference and does not accept file"}) _ = s.writeResponse(connection, daemonapi.Response{Kind: daemonapi.ResponseResult, Error: "container-image requires imageReference and does not accept file"})
return return
} }
record, updateErr = s.updater.UpdateContainerImage(ctx, request.ImageReference, report) record, updateErr = s.updater.UpdateContainerImage(ctx, request.ImageReference, request.StartLog, report)
default: default:
_ = s.writeResponse(connection, daemonapi.Response{Kind: daemonapi.ResponseResult, Error: "inputType must be repack-zip, native-jar, or container-image"}) _ = s.writeResponse(connection, daemonapi.Response{Kind: daemonapi.ResponseResult, Error: "inputType must be repack-zip, native-jar, or container-image"})
return return
+2 -2
View File
@@ -98,7 +98,7 @@ func TestServerAcceptsContainerImageThroughUnixSocket(t *testing.T) {
waitForSocket(t, socketPath, serveResult) waitForSocket(t, socketPath, serveResult)
imageReference := "harbor.ymswell.asia/ymswell/glory-ymswell:20260813-184902-a37bf50d-v1.1.8.1" imageReference := "harbor.ymswell.asia/ymswell/glory-ymswell:20260813-184902-a37bf50d-v1.1.8.1"
response, err := daemonclient.UpdateContainerImage(context.Background(), socketPath, "backend", imageReference, nil) response, err := daemonclient.UpdateContainerImage(context.Background(), socketPath, "backend", imageReference, true, nil)
if err != nil { if err != nil {
t.Fatalf("submit container backend image: %v", err) t.Fatalf("submit container backend image: %v", err)
} }
@@ -217,7 +217,7 @@ func (u *fakeUpdater) UpdateNativeJAR(_ context.Context, file string, report bac
return u.record, u.err return u.record, u.err
} }
func (u *fakeUpdater) UpdateContainerImage(_ context.Context, imageReference string, report backendupdate.ProgressReporter) (transaction.Transaction, error) { func (u *fakeUpdater) UpdateContainerImage(_ context.Context, imageReference string, _ bool, report backendupdate.ProgressReporter) (transaction.Transaction, error) {
u.file = imageReference u.file = imageReference
u.inputType = daemonapi.InputTypeContainerImage u.inputType = daemonapi.InputTypeContainerImage
u.operation = daemonapi.OperationUpdate u.operation = daemonapi.OperationUpdate
+7
View File
@@ -18,6 +18,13 @@ type Transaction struct {
UpdatedAt time.Time UpdatedAt time.Time
} }
// ListFilter limits the history query used by the ymsctl list command.
type ListFilter struct {
Limit int
Service string
State State
}
// CreateRequest 包含创建事务所需的不可变请求信息。 // CreateRequest 包含创建事务所需的不可变请求信息。
type CreateRequest struct { type CreateRequest struct {
ID string ID string
+50
View File
@@ -335,6 +335,56 @@ func (s *Store) ActiveTransaction(ctx context.Context) (Transaction, error) {
return getActiveTransaction(ctx, s.db) return getActiveTransaction(ctx, s.db)
} }
// ListRecent returns committed, rolled-back, failed, and in-progress transactions
// in reverse creation order. Filters are exact values and never inferred.
func (s *Store) ListRecent(ctx context.Context, filter ListFilter) ([]Transaction, error) {
if filter.Limit <= 0 || filter.Limit > 1000 {
return nil, errors.New("transaction history limit must be between 1 and 1000")
}
query := `SELECT id, idempotency_key, source, service, request_json, state, version, created_at, updated_at FROM transactions WHERE 1=1`
args := make([]any, 0, 3)
if filter.Service != "" {
query += " AND service = ?"
args = append(args, filter.Service)
}
if filter.State != "" {
query += " AND state = ?"
args = append(args, string(filter.State))
}
query += " ORDER BY created_at DESC, id DESC LIMIT ?"
args = append(args, filter.Limit)
rows, err := s.db.QueryContext(ctx, query, args...)
if err != nil {
return nil, fmt.Errorf("list transaction history: %w", err)
}
defer rows.Close()
result := make([]Transaction, 0, filter.Limit)
for rows.Next() {
var item Transaction
var state string
var requestJSON string
var createdAt, updatedAt string
if err := rows.Scan(&item.ID, &item.IdempotencyKey, &item.Source, &item.Service, &requestJSON, &state, &item.Version, &createdAt, &updatedAt); err != nil {
return nil, fmt.Errorf("scan transaction history: %w", err)
}
item.Request = json.RawMessage(requestJSON)
item.State = State(state)
item.CreatedAt, err = time.Parse(time.RFC3339Nano, createdAt)
if err != nil {
return nil, fmt.Errorf("parse transaction history creation time: %w", err)
}
item.UpdatedAt, err = time.Parse(time.RFC3339Nano, updatedAt)
if err != nil {
return nil, fmt.Errorf("parse transaction history update time: %w", err)
}
result = append(result, item)
}
if err := rows.Err(); err != nil {
return nil, fmt.Errorf("iterate transaction history: %w", err)
}
return result, nil
}
// Transition 校验并原子提交状态变化及其恢复事件。 // Transition 校验并原子提交状态变化及其恢复事件。
func (s *Store) Transition(ctx context.Context, id string, next State, message string) (Transaction, error) { func (s *Store) Transition(ctx context.Context, id string, next State, message string) (Transaction, error) {
if !next.Valid() { if !next.Valid() {
+20
View File
@@ -105,6 +105,26 @@ func TestStoreMigratesVersionOneAndCommitsBackendContainerDeployment(t *testing.
} }
} }
func TestListRecentScansSQLiteRequestTextAsRawJSON(t *testing.T) {
t.Parallel()
ctx := context.Background()
store := openTestStore(t)
record, _, err := store.CreateTransaction(ctx, CreateRequest{
ID: "list-transaction", IdempotencyKey: "list-idempotency", Source: "test", Service: "backend",
Request: json.RawMessage(`{"inputType":"native-jar"}`),
})
if err != nil {
t.Fatalf("create list transaction: %v", err)
}
items, err := store.ListRecent(ctx, ListFilter{Limit: 20})
if err != nil {
t.Fatalf("list transactions: %v", err)
}
if len(items) != 1 || items[0].ID != record.ID || string(items[0].Request) != `{"inputType":"native-jar"}` {
t.Fatalf("unexpected listed transaction: %+v", items)
}
}
func TestBackendContainerDeploymentCommitRequiresDrainingTransaction(t *testing.T) { func TestBackendContainerDeploymentCommitRequiresDrainingTransaction(t *testing.T) {
t.Parallel() t.Parallel()
ctx := context.Background() ctx := context.Background()
+67 -19
View File
@@ -2,6 +2,7 @@ package main
import ( import (
"context" "context"
"encoding/json"
"errors" "errors"
"flag" "flag"
"fmt" "fmt"
@@ -33,6 +34,17 @@ const serviceBackend = "backend"
func main() { func main() {
ctx, cancel := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM) ctx, cancel := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer cancel() defer cancel()
if filepath.Base(os.Args[0]) == "ymsd" {
if len(os.Args) != 1 {
fmt.Fprintln(os.Stderr, "ymsd does not accept arguments")
os.Exit(2)
}
if err := runServe(ctx); err != nil {
fmt.Fprintln(os.Stderr, "ymsd failed:", err)
os.Exit(1)
}
return
}
os.Exit(run(ctx, os.Args[1:], os.Stdout, os.Stderr)) os.Exit(run(ctx, os.Args[1:], os.Stdout, os.Stderr))
} }
@@ -42,16 +54,6 @@ func run(ctx context.Context, arguments []string, stdout io.Writer, stderr io.Wr
return 2 return 2
} }
switch arguments[0] { switch arguments[0] {
case "serve":
if len(arguments) != 1 {
fmt.Fprintln(stderr, "serve does not accept arguments")
return 2
}
if err := runServe(ctx); err != nil {
fmt.Fprintln(stderr, "yms-daemon serve failed:", err)
return 1
}
return 0
case "update": case "update":
request, err := parseUpdateArgs(arguments[1:], stderr) request, err := parseUpdateArgs(arguments[1:], stderr)
if err != nil { if err != nil {
@@ -66,7 +68,7 @@ func run(ctx context.Context, arguments []string, stdout io.Writer, stderr io.Wr
} }
var response daemonapi.Response var response daemonapi.Response
if request.inputType == daemonapi.InputTypeContainerImage { if request.inputType == daemonapi.InputTypeContainerImage {
response, err = daemonclient.UpdateContainerImage(ctx, runtimepaths.Socket, request.service, request.imageReference, progress) response, err = daemonclient.UpdateContainerImage(ctx, runtimepaths.Socket, request.service, request.imageReference, request.startLog, progress)
} else { } else {
response, err = daemonclient.Update(ctx, runtimepaths.Socket, request.service, request.inputType, request.file, progress) response, err = daemonclient.Update(ctx, runtimepaths.Socket, request.service, request.inputType, request.file, progress)
} }
@@ -74,7 +76,7 @@ func run(ctx context.Context, arguments []string, stdout io.Writer, stderr io.Wr
if response.TransactionID != "" { if response.TransactionID != "" {
fmt.Fprintf(stderr, "transaction=%s state=%s error=%v\n", response.TransactionID, response.State, err) fmt.Fprintf(stderr, "transaction=%s state=%s error=%v\n", response.TransactionID, response.State, err)
} else { } else {
fmt.Fprintln(stderr, "yms-daemon update failed:", err) fmt.Fprintln(stderr, "ymsctl update failed:", err)
} }
return 1 return 1
} }
@@ -82,6 +84,12 @@ func run(ctx context.Context, arguments []string, stdout io.Writer, stderr io.Wr
fmt.Fprintf(stdout, "transaction=%s state=%s\n", response.TransactionID, response.State) fmt.Fprintf(stdout, "transaction=%s state=%s\n", response.TransactionID, response.State)
} }
return 0 return 0
case "list":
if err := runList(ctx, arguments[1:], stdout, stderr); err != nil {
fmt.Fprintln(stderr, "ymsctl list failed:", err)
return 1
}
return 0
case "restart": case "restart":
request, err := parseRestartArgs(arguments[1:], stderr) request, err := parseRestartArgs(arguments[1:], stderr)
if err != nil { if err != nil {
@@ -99,7 +107,7 @@ func run(ctx context.Context, arguments []string, stdout io.Writer, stderr io.Wr
if response.TransactionID != "" { if response.TransactionID != "" {
fmt.Fprintf(stderr, "transaction=%s state=%s error=%v\n", response.TransactionID, response.State, err) fmt.Fprintf(stderr, "transaction=%s state=%s error=%v\n", response.TransactionID, response.State, err)
} else { } else {
fmt.Fprintln(stderr, "yms-daemon restart failed:", err) fmt.Fprintln(stderr, "ymsctl restart failed:", err)
} }
return 1 return 1
} }
@@ -123,6 +131,7 @@ type updateArguments struct {
file string file string
imageReference string imageReference string
quite bool quite bool
startLog bool
} }
type restartArguments struct { type restartArguments struct {
@@ -138,6 +147,7 @@ func parseUpdateArgs(arguments []string, output io.Writer) (updateArguments, err
nativeJAR := flags.String("native-jar", "", "direct native backend JAR path") nativeJAR := flags.String("native-jar", "", "direct native backend JAR path")
containerImage := flags.String("container-image", "", "development backend container image reference") containerImage := flags.String("container-image", "", "development backend container image reference")
quite := flags.Bool("quite", false, "suppress progress and successful result output") quite := flags.Bool("quite", false, "suppress progress and successful result output")
noStartLog := flags.Bool("no-start-log", false, "do not print container startup logs")
if err := flags.Parse(arguments); err != nil { if err := flags.Parse(arguments); err != nil {
return updateArguments{}, err return updateArguments{}, err
} }
@@ -157,7 +167,10 @@ func parseUpdateArgs(arguments []string, output io.Writer) (updateArguments, err
return updateArguments{}, errors.New("exactly one of -f, --native-jar, and --container-image is required; update inputs are mutually exclusive") return updateArguments{}, errors.New("exactly one of -f, --native-jar, and --container-image is required; update inputs are mutually exclusive")
} }
if *containerImage != "" { if *containerImage != "" {
return updateArguments{service: *service, inputType: daemonapi.InputTypeContainerImage, imageReference: *containerImage, quite: *quite}, nil return updateArguments{service: *service, inputType: daemonapi.InputTypeContainerImage, imageReference: *containerImage, quite: *quite, startLog: !*noStartLog}, nil
}
if *noStartLog {
return updateArguments{}, errors.New("--no-start-log requires --container-image")
} }
inputType := daemonapi.InputTypeRepackZIP inputType := daemonapi.InputTypeRepackZIP
inputFile := *file inputFile := *file
@@ -266,11 +279,11 @@ func runServe(ctx context.Context) (result error) {
func writeUsage(output io.Writer) { func writeUsage(output io.Writer) {
fmt.Fprintln(output, "usage:") fmt.Fprintln(output, "usage:")
fmt.Fprintln(output, " yms-daemon serve") fmt.Fprintln(output, " ymsctl update --service backend -f <repack.zip> [--quite]")
fmt.Fprintln(output, " yms-daemon update --service backend -f <repack.zip> [--quite]") fmt.Fprintln(output, " ymsctl update --service backend --native-jar <backend.jar> [--quite]")
fmt.Fprintln(output, " yms-daemon update --service backend --native-jar <backend.jar> [--quite]") fmt.Fprintln(output, " ymsctl update --service backend --container-image <image-ref> [--no-start-log] [--quite]")
fmt.Fprintln(output, " yms-daemon update --service backend --container-image <image-ref> [--quite]") fmt.Fprintln(output, " ymsctl restart --service backend [--quite]")
fmt.Fprintln(output, " yms-daemon restart --service backend [--quite]") fmt.Fprintln(output, " ymsctl list [--limit <n>] [--service backend] [--state <state>] [--json]")
} }
func writeUpdateProgress(output io.Writer, event daemonapi.Response) { func writeUpdateProgress(output io.Writer, event daemonapi.Response) {
@@ -280,3 +293,38 @@ func writeUpdateProgress(output io.Writer, event daemonapi.Response) {
} }
fmt.Fprintf(output, "%-14s %s\n", state, event.Message) fmt.Fprintf(output, "%-14s %s\n", state, event.Message)
} }
func runList(ctx context.Context, arguments []string, stdout, stderr io.Writer) error {
flags := flag.NewFlagSet("list", flag.ContinueOnError)
flags.SetOutput(stderr)
limit := flags.Int("limit", 20, "maximum number of operations to show")
service := flags.String("service", "", "exact service filter")
state := flags.String("state", "", "exact transaction state filter")
jsonOutput := flags.Bool("json", false, "write stable JSON")
if err := flags.Parse(arguments); err != nil {
return err
}
if flags.NArg() != 0 {
return errors.New("list does not accept positional arguments")
}
if *service != "" && *service != serviceBackend {
return errors.New("--service currently accepts only backend")
}
store, err := transaction.OpenStore(ctx, runtimepaths.Database)
if err != nil {
return err
}
defer store.Close()
items, err := store.ListRecent(ctx, transaction.ListFilter{Limit: *limit, Service: *service, State: transaction.State(*state)})
if err != nil {
return err
}
if *jsonOutput {
return json.NewEncoder(stdout).Encode(items)
}
fmt.Fprintln(stdout, "TIME\tTRANSACTION\tSERVICE\tSTATE\tSOURCE")
for _, item := range items {
fmt.Fprintf(stdout, "%s\t%s\t%s\t%s\t%s\n", item.CreatedAt.Local().Format("2006-01-02 15:04:05"), item.ID, item.Service, item.State, item.Source)
}
return nil
}
+18
View File
@@ -50,6 +50,24 @@ func TestParseUpdateArgsAcceptsContainerImage(t *testing.T) {
} }
} }
func TestParseUpdateArgsControlsContainerStartupLogs(t *testing.T) {
imageReference := "harbor.ymswell.asia/ymswell/glory-ymswell:20260813-184902-a37bf50d-v1.1.8.1"
request, err := parseUpdateArgs([]string{"--service", "backend", "--container-image", imageReference, "--no-start-log"}, &bytes.Buffer{})
if err != nil {
t.Fatalf("parse no-start-log container update arguments: %v", err)
}
if request.startLog {
t.Fatal("--no-start-log did not disable startup logs")
}
request, err = parseUpdateArgs([]string{"--service", "backend", "--container-image", imageReference}, &bytes.Buffer{})
if err != nil {
t.Fatalf("parse default container update arguments: %v", err)
}
if !request.startLog {
t.Fatal("container startup logs are not enabled by default")
}
}
func TestParseUpdateArgsRejectsNativeJARAndContainerImageTogether(t *testing.T) { func TestParseUpdateArgsRejectsNativeJARAndContainerImageTogether(t *testing.T) {
jarPath := filepath.Join(t.TempDir(), "glory-soft-yms.jar") jarPath := filepath.Join(t.TempDir(), "glory-soft-yms.jar")
arguments := []string{ arguments := []string{
+3 -2
View File
@@ -71,12 +71,13 @@ sources_dir="$top_dir/SOURCES"
temporary_dir="$build_dir/tmp" temporary_dir="$build_dir/tmp"
mkdir -p "$sources_dir" "$top_dir/BUILD" "$top_dir/BUILDROOT" "$top_dir/RPMS" "$top_dir/SRPMS" "$temporary_dir" mkdir -p "$sources_dir" "$top_dir/BUILD" "$top_dir/BUILDROOT" "$top_dir/RPMS" "$top_dir/SRPMS" "$temporary_dir"
echo "Building static linux/$go_arch yms-daemon" echo "Building static linux/$go_arch ymsd and ymsctl"
( (
cd "$project_dir" cd "$project_dir"
GOCACHE="$build_dir/go-cache" CGO_ENABLED=0 GOOS=linux GOARCH="$go_arch" \ GOCACHE="$build_dir/go-cache" CGO_ENABLED=0 GOOS=linux GOARCH="$go_arch" \
go build -trimpath -ldflags="-s -w" -o "$sources_dir/yms-daemon" . go build -trimpath -ldflags="-s -w" -o "$sources_dir/ymsd" .
) )
cp "$sources_dir/ymsd" "$sources_dir/ymsctl"
install -m 0640 "$project_dir/packaging/etc/yms-daemon/yms-daemon.toml" "$sources_dir/yms-daemon.toml" install -m 0640 "$project_dir/packaging/etc/yms-daemon/yms-daemon.toml" "$sources_dir/yms-daemon.toml"
install -m 0644 "$project_dir/packaging/systemd/yms-daemon.service" "$sources_dir/yms-daemon.service" install -m 0644 "$project_dir/packaging/systemd/yms-daemon.service" "$sources_dir/yms-daemon.service"
+6 -3
View File
@@ -12,11 +12,12 @@ Summary: YMS update daemon
License: Proprietary License: Proprietary
Requires: systemd Requires: systemd
Source0: yms-daemon Source0: ymsd
Source1: yms-daemon.toml Source1: yms-daemon.toml
Source2: yms-daemon.service Source2: yms-daemon.service
Source3: yms-backend@.service Source3: yms-backend@.service
Source4: yms-daemon-tmpfiles.conf Source4: yms-daemon-tmpfiles.conf
Source5: ymsctl
%description %description
Transactional update daemon and native backend systemd units for YMS installations. Transactional update daemon and native backend systemd units for YMS installations.
@@ -27,7 +28,8 @@ Transactional update daemon and native backend systemd units for YMS installatio
%install %install
install -d -m 0755 %{buildroot}/usr/bin install -d -m 0755 %{buildroot}/usr/bin
install -m 0755 %{SOURCE0} %{buildroot}/usr/bin/yms-daemon install -m 0755 %{SOURCE0} %{buildroot}/usr/bin/ymsd
install -m 0755 %{SOURCE5} %{buildroot}/usr/bin/ymsctl
install -d -m 0755 %{buildroot}/etc/yms-daemon install -d -m 0755 %{buildroot}/etc/yms-daemon
install -m 0640 %{SOURCE1} %{buildroot}/etc/yms-daemon/yms-daemon.toml install -m 0640 %{SOURCE1} %{buildroot}/etc/yms-daemon/yms-daemon.toml
@@ -71,7 +73,8 @@ fi
%attr(0644,root,root) /etc/systemd/system/yms-daemon.service %attr(0644,root,root) /etc/systemd/system/yms-daemon.service
%attr(0644,root,root) /etc/systemd/system/yms-backend@.service %attr(0644,root,root) /etc/systemd/system/yms-backend@.service
%attr(0644,root,root) /etc/tmpfiles.d/yms-daemon.conf %attr(0644,root,root) /etc/tmpfiles.d/yms-daemon.conf
%attr(0755,root,root) /usr/bin/yms-daemon %attr(0755,root,root) /usr/bin/ymsd
%attr(0755,root,root) /usr/bin/ymsctl
%dir %attr(0755,root,root) /home/yms/dump %dir %attr(0755,root,root) /home/yms/dump
%changelog %changelog
+1 -1
View File
@@ -6,7 +6,7 @@ After=network.target nginx.service
Type=simple Type=simple
User=root User=root
Group=root Group=root
ExecStart=/usr/bin/yms-daemon serve ExecStart=/usr/bin/ymsd
Restart=on-failure Restart=on-failure
RestartSec=5 RestartSec=5
RuntimeDirectory=yms-daemon RuntimeDirectory=yms-daemon
+7
View File
@@ -0,0 +1,7 @@
sonar.projectKey=yms-daemon
sonar.projectName=yms-daemon
sonar.sources=.
sonar.go.coverage.reportPaths=coverage.out
sonar.host.url=http://10.11.1.150:9000
sonar.login=sqa_01cc7f5c4cc3823ed99abc95a36d3657cec0391e
sonar.branch.name=master