Files
larksuite__cli/shortcuts/apps/db_common_test.go
jinjiuzhe 158d15b3fd feat: add apps database sync shortcuts for Base-to-database import (#2251)
* feat: add apps database sync shortcuts

Add Base-to-database sync shortcuts for preview, create, list, get, enable, disable, update, and delete flows.

Cover OpenAPI request contracts, typed sync error classification, dry-run E2E coverage, and lark-apps skill guidance.

Co-authored-by: TRAE CLI <noreply@bytedance.com>

* fix(apps): send db-sync task_id and config in request body

The enable/disable/delete/update sync commands placed task_id (and
update's config) in query params, but the OpenAPI contract binds these
fields via api.json (request body). BOE testing returned
"field validation failed" (99992402) because the body was empty.

Move task_id to the request body for enable/disable/delete, and move
both task_id and config to the body for update. Dry-run previews now
render these under body, and unit tests pin the body binding so a
regression to query params fails.

* fix(apps): use POST for db-sync-delete action endpoint

The delete command issued an HTTP DELETE to db/sync_del, but the
action-style endpoint is registered as POST (like sync_create and
sync_disable). The method mismatch made the gateway return a plaintext
404, surfacing as "API returned a non-object JSON response".

Switch the request and dry-run preview to POST, and pin the method in
the delete unit tests so a regression to DELETE fails.

* test: pin db-sync update base_url as optional contract

* test: pin db-sync update omits base_url without silent default

* docs(skills): clarify db-sync source.base_url create-required update-optional contract

* fix(apps): send db-sync env in request body not query params

The +db-sync-create and +db-sync-update endpoints read env from the
request body (peer of config/preview/task_id), not the query string.
Placing env in query params left the body env empty, so the server
treated every request as online and rejected DDL operations
(code 500002776: forbid ddl/dcl operation in online env), making it
impossible to create/update sync tasks against a dev environment.

Move env into the request body via a new dbEnvBody helper that mirrors
dbEnvParams' omit-empty contract, so unset env still lets the server
auto-select the branch. Pin the contract in unit and e2e dry-run tests
by asserting body.env and that env is absent from query params.

* test: align db-sync operate/delete e2e with request-body contract

The enable/disable/delete dry-run e2e still asserted the pre-migration
wire shape: delete on DELETE and task_id in query params. The shortcuts
now POST these actions with task_id in the request body (commits moving
task_id and the delete verb), so the stale assertions failed against a
current binary.

Assert POST + body.task_id and that task_id is absent from query params,
pinning the same body-over-query contract the env fix established.

* fix(apps): improve db-sync create ergonomics and error guidance

Refine +db-sync-create/update validation, error hints, and docs so AI
agents recover from common Base-to-database sync failures without guessing:

- source.table.name: document that a user-named table must be set, name
  takes precedence over the base_url ?table= token; fix test fixtures that
  used a fictional source.table.url instead of source.base_url.
- Preflight source table locate: reject create locally when base_url has no
  ?table= and source.table.name is empty, pointing at base +table-list.
- Online DDL ban: attach a precise hint for code 500002776 + subcode
  k_dl_4000001 telling multi-env apps to create tables on --environment dev.
- Missing record-id column: extend the 500002783 hint to add a unique text
  column via +db-execute before retrying.
- Optional field_maps on create: allow omitted or empty field_maps so the
  server auto-matches and creates the task; keep update requiring an enabled
  mapping and still reject an all-disabled array.
- Environment default: db-sync commands use online when --environment is
  omitted; align help text, comments, and skill docs.

* fix(apps): migrate db-sync error codes to the 4xx client-error range

The backend moved the seven db-sync error codes from the 5000027xx
server-error range to the 4000024xx client-input range to reflect that
they are client-input errors. Mirror the new codes in the CLI so error
classification and recovery hints keep matching:

- 500002783 -> 400002477 (mapping invalid)
- 500002784 -> 400002478 (target schema mismatch)
- 500002785 -> 400002479 (operation not allowed)
- 500002786 -> 400002480 (task not found)
- 500002787 -> 400002481 (invalid task id)
- 500002788 -> 400002482 (source table not found)
- 500002789 -> 400002483 (target table not found)

Category, subtype, hint text, and behavior are unchanged; 500002776
(online DDL ban) is untouched.

* fix(apps): tighten db-sync preview validation and pretty output

Address review follow-ups on the db-sync shortcuts:

- +db-sync-get pretty output no longer prints <nil> for a missing
  schema_only nor Go map syntax for statistics; render a bare bool and
  deterministic key=value pairs instead.
- Reject a non-array field_maps in +db-sync-create --preview as well as
  commit, so the malformed shape is caught locally rather than forwarded
  to the backend.
- Clarify in lark-apps-db.md that +db-sync-create --preview needs no
  confirmation and only a real create requires --yes.
- Harden the db-sync dry-run validation tests to assert exit code 2 and
  the structured stderr envelope (type/subtype/param), and add coverage
  for the preview non-array field_maps rejection and batch pretty output.

* fix(apps): guard db-sync preview output and neutralize update hint

Address the next db-sync review round:

- +db-sync-create --preview --output no longer writes a "null" file and
  exits success when the response omits data.config; project config into
  a typed object and return internal/invalid_response without writing.
- Make the 400002482 code hint command-neutral so +db-sync-update is not
  steered into a create-only recovery path that risks duplicate tasks.
- lark-apps-db.md: carry --environment on the update lifecycle examples
  and split failure recovery by streaming (can update) vs batch (cannot
  update; recreate instead), removing the batch/update contradiction.

---------

Co-authored-by: TRAE CLI <noreply@bytedance.com>
2026-08-11 18:21:32 +08:00

323 lines
14 KiB
Go
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
// Copyright (c) 2026 Lark Technologies Pte. Ltd.
// SPDX-License-Identifier: MIT
package apps
import (
"errors"
"strings"
"testing"
"github.com/larksuite/cli/errs"
"github.com/larksuite/cli/shortcuts/common"
)
func TestAppTablesPath_ReusesExistingURL(t *testing.T) {
if got := appTablesPath("app_x"); got != "/open-apis/spark/v1/apps/app_x/tables" {
t.Fatalf("appTablesPath = %q (want existing /apps/{id}/tables, not /db/tables)", got)
}
}
func TestAppTablePath_EncodesSegments(t *testing.T) {
if got := appTablePath("app_x", "my table"); got != "/open-apis/spark/v1/apps/app_x/tables/my%20table" {
t.Fatalf("appTablePath = %q", got)
}
}
func TestAppSQLPath_ReusesExistingURL(t *testing.T) {
if got := appSQLPath("app_x"); got != "/open-apis/spark/v1/apps/app_x/sql_commands" {
t.Fatalf("appSQLPath = %q (want /apps/{id}/sql_commands)", got)
}
}
func TestAppDbEnvCreatePath_NewURL(t *testing.T) {
// db-env-create 是本期新增接口,URL 走 /db_dev_init(与上面三条复用 URL 不同)。
if got := appDbEnvCreatePath("app_x"); got != "/open-apis/spark/v1/apps/app_x/db_dev_init" {
t.Fatalf("appDbEnvCreatePath = %q", got)
}
}
func TestRequireAppID_BlankRejected(t *testing.T) {
if _, err := requireAppID(" "); err == nil {
t.Fatal("expected error for blank app-id")
}
got, err := requireAppID(" app_x ")
if err != nil || got != "app_x" {
t.Fatalf("requireAppID trimmed = %q err=%v", got, err)
}
}
func TestAppDbSyncPaths(t *testing.T) {
if got := appDbSyncCreatePath("app x"); got != "/open-apis/spark/v1/apps/app%20x/db/sync_create" {
t.Fatalf("appDbSyncCreatePath = %q", got)
}
if got := appDbSyncTaskPath("app_x"); got != "/open-apis/spark/v1/apps/app_x/db/sync_task" {
t.Fatalf("appDbSyncTaskPath = %q", got)
}
if got := appDbSyncActionPath("app_x", "enable"); got != "/open-apis/spark/v1/apps/app_x/db/sync_enable" {
t.Fatalf("appDbSyncActionPath = %q", got)
}
if got := appDbSyncDeletePath("app_x"); got != "/open-apis/spark/v1/apps/app_x/db/sync_del" {
t.Fatalf("appDbSyncDeletePath = %q", got)
}
}
func TestDBSyncConfig(t *testing.T) {
validWithoutFieldMaps := `{
"mode": "batch",
"source": {"type": "base"},
"target": {"type": "postgresql", "table": {"name": "orders", "action": "create"}}
}`
validWithFieldMaps := `{
"mode": "streaming",
"source": {"type": "base"},
"target": {"type": "postgresql", "table": {"name": "orders", "action": "use_existing"}},
"field_maps": [{"source": "record_id", "target": "record_id", "enabled": true}]
}`
t.Run("preview allows omitted field maps", func(t *testing.T) {
cfg, err := parseDBSyncConfigFlag(validWithoutFieldMaps, false, false)
if err != nil {
t.Fatalf("parseDBSyncConfigFlag preview = %v", err)
}
if cfg["mode"] != "batch" {
t.Fatalf("mode = %v", cfg["mode"])
}
})
t.Run("create-commit auto-matches when field_maps omitted or empty", func(t *testing.T) {
// create passes allowAutoMatch=true: absent field_maps or an empty array is
// allowed — the server auto-matches and creates the task.
if _, err := parseDBSyncConfigFlag(validWithoutFieldMaps, true, true); err != nil {
t.Fatalf("create without field_maps = %v, want nil (server auto-match)", err)
}
emptyArray := strings.Replace(validWithoutFieldMaps, `"target"`, `"field_maps": [], "target"`, 1)
if _, err := parseDBSyncConfigFlag(emptyArray, true, true); err != nil {
t.Fatalf("create with empty field_maps = %v, want nil (server auto-match)", err)
}
nullValue := strings.Replace(validWithoutFieldMaps, `"target"`, `"field_maps": null, "target"`, 1)
assertDBSyncConfigValidation(t, nullValue, true, true, "array")
objectValue := strings.Replace(validWithoutFieldMaps, `"target"`, `"field_maps": {}, "target"`, 1)
assertDBSyncConfigValidation(t, objectValue, true, true, "array")
// A present-but-all-disabled array is still a suspected mistake → rejected.
disabledOnly := `{
"mode": "batch",
"source": {"type": "base"},
"target": {"type": "postgresql", "table": {"name": "orders", "action": "create"}},
"field_maps": [{"source": "a", "target": "b", "enabled": false}]
}`
assertDBSyncConfigValidation(t, disabledOnly, true, true, "disabled")
if _, err := parseDBSyncConfigFlag(validWithFieldMaps, true, true); err != nil {
t.Fatalf("parseDBSyncConfigFlag valid field_maps = %v", err)
}
})
t.Run("update still requires field maps", func(t *testing.T) {
// update passes allowAutoMatch=false: absent field_maps is rejected (update
// means changing the mapping, so an explicit mapping is required).
assertDBSyncConfigValidation(t, validWithoutFieldMaps, true, false, "field_maps")
emptyArray := strings.Replace(validWithoutFieldMaps, `"target"`, `"field_maps": [], "target"`, 1)
assertDBSyncConfigValidation(t, emptyArray, true, false, "field_maps")
disabledOnly := strings.Replace(validWithFieldMaps, `"enabled": true`, `"enabled": false`, 1)
assertDBSyncConfigValidation(t, disabledOnly, true, false, "disabled")
})
t.Run("must be JSON object", func(t *testing.T) {
assertDBSyncConfigValidation(t, `[1,2,3]`, false, false, "JSON object")
assertDBSyncConfigValidation(t, `{`, false, false, "JSON object")
assertDBSyncConfigValidation(t, validWithoutFieldMaps+` {}`, false, false, "one JSON object")
})
t.Run("singular map keys are rejected", func(t *testing.T) {
assertDBSyncConfigValidation(t, strings.Replace(validWithoutFieldMaps, `"source"`, `"field_map": [], "source"`, 1), false, false, "field_maps")
assertDBSyncConfigValidation(t, strings.Replace(validWithoutFieldMaps, `"source"`, `"option_mapping": {}, "source"`, 1), false, false, "option_mappings")
nestedSingular := `{
"mode": "streaming",
"source": {"type": "base"},
"target": {"type": "postgresql", "table": {"name": "orders", "action": "use_existing"}},
"field_maps": [{"source": "record_id", "target": "record_id", "option_mapping": []}]
}`
assertDBSyncConfigValidation(t, nestedSingular, true, true, "option_mappings")
})
t.Run("mode and endpoints are constrained", func(t *testing.T) {
assertDBSyncConfigValidation(t, strings.Replace(validWithoutFieldMaps, `"batch"`, `"full"`, 1), false, false, "mode")
assertDBSyncConfigValidation(t, strings.Replace(validWithoutFieldMaps, `"base"`, `"sheet"`, 1), false, false, "source.type")
assertDBSyncConfigValidation(t, strings.Replace(validWithoutFieldMaps, `"postgresql"`, `"mysql"`, 1), false, false, "target.type")
assertDBSyncConfigValidation(t, strings.Replace(validWithoutFieldMaps, `"orders"`, `""`, 1), false, false, "target.table.name")
assertDBSyncConfigValidation(t, strings.Replace(validWithoutFieldMaps, `"create"`, `"drop"`, 1), false, false, "target.table.action")
})
t.Run("schema only is batch create only", func(t *testing.T) {
validSchemaOnly := strings.Replace(validWithoutFieldMaps, `"mode": "batch"`, `"schema_only": true, "mode": "batch"`, 1)
if _, err := parseDBSyncConfigFlag(validSchemaOnly, false, false); err != nil {
t.Fatalf("parseDBSyncConfigFlag valid schema_only = %v", err)
}
streamingSchemaOnly := strings.Replace(validWithFieldMaps, `"mode": "streaming"`, `"schema_only": true, "mode": "streaming"`, 1)
assertDBSyncConfigValidation(t, streamingSchemaOnly, false, false, "schema_only")
useExistingSchemaOnly := strings.Replace(validSchemaOnly, `"create"`, `"use_existing"`, 1)
assertDBSyncConfigValidation(t, useExistingSchemaOnly, false, false, "schema_only")
})
t.Run("update allows omitted source base_url", func(t *testing.T) {
// 契约:+db-sync-update 的 base_url 可选——省略时后端从原 syncTask 复用源 URL。
// parseDBSyncConfigFlag 不得对 base_url 做前置校验(即便 requireFieldMaps=true)。
noBaseURL := `{
"mode": "streaming",
"source": {"type": "base", "table": {"name": "数据表"}},
"target": {"type": "postgresql", "table": {"name": "orders", "action": "use_existing"}},
"field_maps": [{"source_field": "record_id", "target_field": "record_id", "enabled": true}]
}`
cfg, err := parseDBSyncConfigFlag(noBaseURL, true, false)
if err != nil {
t.Fatalf("parseDBSyncConfigFlag update without base_url = %v, want nil", err)
}
source, ok := cfg["source"].(map[string]interface{})
if !ok {
t.Fatalf("source = %T, want map", cfg["source"])
}
if _, exists := source["base_url"]; exists {
t.Fatalf("source.base_url present %v, want absent (CLI must not inject default)", source["base_url"])
}
})
}
func TestDBSyncEnvironmentHelpDefaultsToOnline(t *testing.T) {
for _, shortcut := range []common.Shortcut{AppsDBSyncCreate, AppsDBSyncList, AppsDBSyncUpdate} {
t.Run(shortcut.Command, func(t *testing.T) {
for _, flag := range shortcut.Flags {
if flag.Name != "environment" {
continue
}
if !strings.Contains(flag.Desc, "leave unset to use online") {
t.Fatalf("%s --environment description = %q, want online default", shortcut.Command, flag.Desc)
}
if strings.Contains(flag.Desc, "auto-select") || strings.Contains(flag.Desc, "multi-env app uses dev") {
t.Fatalf("%s --environment description still claims auto-select: %q", shortcut.Command, flag.Desc)
}
return
}
t.Fatalf("%s missing --environment flag", shortcut.Command)
})
}
}
func TestDBSyncHint(t *testing.T) {
t.Run("nil and untyped errors pass through", func(t *testing.T) {
if got := withDBSyncHint(nil, "fallback"); got != nil {
t.Fatalf("withDBSyncHint(nil) = %v", got)
}
plain := errors.New("plain")
if got := withDBSyncHint(plain, "fallback"); got != plain {
t.Fatalf("withDBSyncHint(plain) = %v, want original", got)
}
})
t.Run("known code gets mapped hint", func(t *testing.T) {
in := errs.NewAPIError(errs.SubtypeNotFound, "target missing").WithCode(400002483).WithLogID("log_x")
out := withDBSyncHint(in, "fallback")
p, ok := errs.ProblemOf(out)
if !ok {
t.Fatalf("withDBSyncHint returned untyped error: %T", out)
}
if !strings.Contains(p.Hint, "target.table.action") {
t.Fatalf("hint = %q, want target-table recovery", p.Hint)
}
if p.Code != 400002483 || p.LogID != "log_x" || p.Subtype != errs.SubtypeNotFound {
t.Fatalf("problem metadata mutated: %+v", p)
}
})
t.Run("existing hint is preserved", func(t *testing.T) {
in := errs.NewAPIError(errs.SubtypeInvalidParameters, "bad mapping").WithCode(400002477).WithHint("server hint")
out := withDBSyncHint(in, "fallback")
p, _ := errs.ProblemOf(out)
if p.Hint != "server hint" {
t.Fatalf("hint = %q, want existing hint", p.Hint)
}
})
t.Run("record-id mapping code guides adding a unique column", func(t *testing.T) {
in := errs.NewAPIError(errs.SubtypeInvalidParameters, "mapping must include Base 表记录 ID").WithCode(400002477)
out := withDBSyncHint(in, "fallback")
p, ok := errs.ProblemOf(out)
if !ok {
t.Fatalf("withDBSyncHint returned untyped error: %T", out)
}
if !strings.Contains(p.Hint, "+db-execute") || !strings.Contains(p.Hint, "base_record_id") {
t.Fatalf("hint = %q, want +db-execute add-column guidance", p.Hint)
}
if p.Code != 400002477 || p.Subtype != errs.SubtypeInvalidParameters {
t.Fatalf("problem metadata mutated: %+v", p)
}
})
t.Run("unknown code uses fallback", func(t *testing.T) {
in := errs.NewAPIError(errs.SubtypeUnknown, "unknown").WithCode(42)
out := withDBSyncHint(in, "fallback")
p, _ := errs.ProblemOf(out)
if p.Hint != "fallback" {
t.Fatalf("hint = %q, want fallback", p.Hint)
}
})
t.Run("online DDL subcode gets multi-env dev hint", func(t *testing.T) {
in := errs.NewAPIError(errs.SubtypeUnknown, "k_dl_4000001:forbid ddl/dcl operation in online env").
WithCode(500002776).WithLogID("log_ddl")
out := withDBSyncHint(in, "fallback")
p, ok := errs.ProblemOf(out)
if !ok {
t.Fatalf("withDBSyncHint returned untyped error: %T", out)
}
if !strings.Contains(p.Hint, "multi-env") || !strings.Contains(p.Hint, "--environment dev") {
t.Fatalf("hint = %q, want multi-env online-DDL guidance", p.Hint)
}
if p.Code != 500002776 || p.LogID != "log_ddl" || p.Subtype != errs.SubtypeUnknown {
t.Fatalf("problem metadata mutated: %+v", p)
}
})
t.Run("generic 500002776 without subcode does not get dev hint", func(t *testing.T) {
in := errs.NewAPIError(errs.SubtypeUnknown, "some other 500002776 failure").WithCode(500002776)
out := withDBSyncHint(in, "fallback")
p, _ := errs.ProblemOf(out)
if strings.Contains(p.Hint, "multi-env") {
t.Fatalf("hint = %q, must not attach online-DDL guidance without subcode", p.Hint)
}
if p.Hint != "fallback" {
t.Fatalf("hint = %q, want fallback for unmapped 500002776", p.Hint)
}
})
t.Run("online DDL subcode does not override server hint", func(t *testing.T) {
in := errs.NewAPIError(errs.SubtypeUnknown, "k_dl_4000001:forbid ddl/dcl operation in online env").
WithCode(500002776).WithHint("server hint")
out := withDBSyncHint(in, "fallback")
p, _ := errs.ProblemOf(out)
if p.Hint != "server hint" {
t.Fatalf("hint = %q, want preserved server hint", p.Hint)
}
})
}
func assertDBSyncConfigValidation(t *testing.T, raw string, requireFieldMaps, allowAutoMatch bool, wantText string) {
t.Helper()
_, err := parseDBSyncConfigFlag(raw, requireFieldMaps, allowAutoMatch)
if err == nil {
t.Fatalf("parseDBSyncConfigFlag(%s) = nil, want validation error", raw)
}
var validationErr *errs.ValidationError
if !errors.As(err, &validationErr) {
t.Fatalf("error = %T %v, want ValidationError", err, err)
}
if validationErr.Param != "--config" {
t.Fatalf("Param = %q, want --config", validationErr.Param)
}
if !strings.Contains(validationErr.Message, wantText) && !strings.Contains(validationErr.Hint, wantText) {
t.Fatalf("message=%q hint=%q, want text %q", validationErr.Message, validationErr.Hint, wantText)
}
if strings.Contains(validationErr.Message, `"target"`) || strings.Contains(validationErr.Message, `"source"`) {
t.Fatalf("error message echoes config: %q", validationErr.Message)
}
}