Compare commits

..

6 Commits

Author SHA1 Message Date
copilot-swe-agent[bot] f43e67ecd7 Addressing PR comments
Co-authored-by: kevwan <1918356+kevwan@users.noreply.github.com>
2025-10-07 04:00:47 +00:00
copilot-swe-agent[bot] 6efaf92471 Restore jwt.api again - it was deleted by an automated commit
Co-authored-by: kevwan <1918356+kevwan@users.noreply.github.com>
2025-10-07 03:59:59 +00:00
copilot-swe-agent[bot] d9304b24ec Addressing PR comments
Co-authored-by: kevwan <1918356+kevwan@users.noreply.github.com>
2025-10-07 03:45:26 +00:00
copilot-swe-agent[bot] ff7258bc52 Restore jwt.api file that was accidentally deleted
Co-authored-by: kevwan <1918356+kevwan@users.noreply.github.com>
2025-10-07 03:44:26 +00:00
copilot-swe-agent[bot] 42b5713621 Fix SSE handler generation to use correct template file
The issue was that SSE handlers were incorrectly using handler.tpl instead of sse_handler.tpl, causing the wrong handler pattern to be generated when custom templates were present. This fix ensures SSE handlers and logic use their dedicated template files (sse_handler.tpl and sse_logic.tpl) when SSE annotation is enabled.

Co-authored-by: kevwan <1918356+kevwan@users.noreply.github.com>
2025-10-07 03:31:50 +00:00
copilot-swe-agent[bot] 59e9e54a21 Initial plan 2025-10-07 03:15:19 +00:00
80 changed files with 612 additions and 3330 deletions
+27 -71
View File
@@ -8,16 +8,14 @@ go-zero is a web and RPC framework with lots of built-in engineering practices d
### Key Architecture Components
- **REST API framework** (`rest/`) - HTTP service framework with middleware chain support
- **RPC framework** (`zrpc/`) - gRPC-based RPC framework with etcd service discovery and p2c_ewma load balancing
- **Gateway** (`gateway/`) - API gateway supporting both HTTP and gRPC upstreams with proto-based routing
- **MCP Server** (`mcp/`) - Model Context Protocol server for AI agent integration via SSE
- **Core utilities** (`core/`) - Production-grade components:
- Resilience: circuit breakers (`breaker/`), rate limiters (`limit/`), adaptive load shedding (`load/`)
- Storage: SQL with cache (`stores/sqlc/`), Redis (`stores/redis/`), MongoDB (`stores/mongo/`)
- Concurrency: MapReduce (`mr/`), worker pools (`executors/`), sync primitives (`syncx/`)
- Observability: metrics (`metric/`), tracing (`trace/`), structured logging (`logx/`)
- **Code generation tool** (`tools/goctl/`) - CLI tool for generating Go code from `.api` and `.proto` files
- **REST API framework** (`rest/`) - HTTP service framework with middleware support
- **RPC framework** (`zrpc/`) - gRPC-based RPC framework with service discovery
- **Core utilities** (`core/`) - Foundational components including:
- Circuit breakers, rate limiters, load shedding
- Caching, stores (Redis, MongoDB, SQL)
- Concurrency control, metrics, tracing
- Configuration management
- **Code generation tool** (`tools/goctl/`) - CLI tool for generating code from API files
## Coding Standards and Conventions
@@ -27,22 +25,18 @@ go-zero is a web and RPC framework with lots of built-in engineering practices d
2. **Package naming**: Use lowercase, single-word package names when possible
3. **Error handling**: Always handle errors explicitly, use `errorx.BatchError` for multiple errors
4. **Context propagation**: Always pass `context.Context` as the first parameter for functions that may block
5. **Configuration structures**: Use struct tags with JSON annotations, defaults, and validation
5. **Configuration structures**: Use struct tags with JSON annotations and default values
**Pattern**: All service configs embed `service.ServiceConf` for common fields (Name, Log, Mode, Telemetry)
Example configuration pattern:
```go
type Config struct {
service.ServiceConf // Always embed for services
Host string `json:",default=0.0.0.0"`
Port int // Required field (no default)
Timeout int64 `json:",default=3000"` // Timeouts in milliseconds
Optional string `json:",optional"` // Optional field
Mode string `json:",default=pro,options=dev|test|rt|pre|pro"` // Validated options
Port int `json:",default=8080"`
Timeout int `json:",default=3000"`
Optional string `json:",optional"`
}
```
**Service modes**: `dev`/`test`/`rt` disable load shedding and stats; `pre`/`pro` enable all resilience features
### Interface Design
1. **Small interfaces**: Follow Go's preference for small, focused interfaces
@@ -100,33 +94,25 @@ func TestSomething(t *testing.T) {
### REST API Development
1. **API Definition**: Use `.api` files to define REST APIs with goctl codegen
2. **Handler pattern**: Separate business logic into logic packages (handlers call logic layer)
3. **Middleware chain**: Middlewares wrap via `chain.Chain` interface - use `Append()` or `Prepend()` to control order
- Built-in middlewares (all in `rest/handler/`): tracing, logging, metrics, recovery, breaker, shedding, timeout, maxconns, maxbytes, gunzip
- Custom middleware: `func(http.Handler) http.Handler` - call `next.ServeHTTP(w, r)` to continue chain
4. **Response handling**: Use `httpx.WriteJson(w, code, v)` for JSON responses
5. **Error handling**: Use `httpx.Error(w, err)` or `httpx.ErrorCtx(ctx, w, err)` for HTTP error responses
6. **Route registration**: Routes defined with `Method`, `Path`, and `Handler` - wildcards use `:param` syntax
1. **API Definition**: Use `.api` files to define REST APIs
2. **Handler pattern**: Separate business logic into logic packages
3. **Middleware**: Use built-in middlewares (tracing, logging, metrics, recovery)
4. **Response handling**: Use `httpx.WriteJson` for JSON responses
5. **Error handling**: Use `httpx.Error` for HTTP error responses
### RPC Development
1. **Protocol Buffers**: Use protobuf for service definitions, generate code with goctl
2. **Service discovery**: Use etcd for dynamic service registration/discovery, or direct endpoints for static routing
3. **Load balancing**: Default is `p2c_ewma` (power of 2 choices with EWMA), configurable via `BalancerName`
4. **Client configuration**: Support `Etcd`, `Endpoints`, or `Target` - use `BuildTarget()` to construct connection string
5. **Interceptors**: Implement gRPC interceptors for cross-cutting concerns (auth, logging, metrics)
6. **Health checks**: gRPC health checks enabled by default (`Health: true`)
1. **Protocol Buffers**: Use protobuf for service definitions
2. **Service discovery**: Integrate with etcd for service registration
3. **Load balancing**: Use built-in load balancing strategies
4. **Interceptors**: Implement interceptors for cross-cutting concerns
### Database Operations
1. **SQL operations**: Use `sqlx.SqlConn` interface - methods always end with `Ctx` for context support
2. **Caching pattern**: `stores/sqlc` provides `CachedConn` for automatic cache-aside pattern
- `QueryRowCtx`: Query with cache key, auto-populate on cache miss
- `ExecCtx`: Execute and delete cache keys
3. **Transactions**: Use `sqlx.SqlConn.TransactCtx()` to get transaction session
4. **Connection pooling**: Managed automatically (64 max idle/open, 1min lifetime)
5. **Test helpers**: Use `redistest.CreateRedis(t)` for Redis, SQL mocks for DB testing
1. **SQL operations**: Use `sqlx` package for database operations
2. **Caching**: Implement caching patterns with `cache` package
3. **Transactions**: Use proper transaction handling
4. **Connection pooling**: Configure appropriate connection pools
Example cache pattern:
```go
@@ -206,36 +192,6 @@ Always implement proper resource cleanup using defer and context cancellation.
- Test: `go test ./...`
- Test with race detection: `go test -race ./...`
- Format: `gofmt -w .`
- Code generation:
- REST API: `goctl api go -api *.api -dir .`
- RPC: `goctl rpc protoc *.proto --go_out=. --go-grpc_out=. --zrpc_out=.`
- Model from SQL: `goctl model mysql datasource -url="user:pass@tcp(host:port)/db" -table="*" -dir="./model"`
## Critical Architecture Patterns
### Resilience Design Philosophy
go-zero implements defense-in-depth with multiple protection layers:
1. **Circuit Breaker** (`core/breaker`): Google SRE breaker - tracks success/failure, opens on error threshold
2. **Adaptive Load Shedding** (`core/load`): CPU-based auto-rejection when system overloaded (disabled in dev/test/rt modes)
3. **Rate Limiting** (`core/limit`): Token bucket (Redis-based) and period limiters
4. **Timeout Control**: Cascading timeouts via context - set at multiple levels (client, server, handler)
### Middleware Chain Architecture
`rest/chain` provides middleware composition:
```go
// Middleware signature
type Middleware func(http.Handler) http.Handler
// Chain operations
chain := chain.New(m1, m2)
chain.Append(m3) // Adds to end: m1 -> m2 -> m3
chain.Prepend(m0) // Adds to start: m0 -> m1 -> m2 -> m3
handler := chain.Then(finalHandler)
```
### Concurrency Patterns
- **MapReduce** (`core/mr`): Parallel processing with worker pools - use for batch operations
- **Executors** (`core/executors`): Bulk/period executors for batching operations
- **SingleFlight** (`core/syncx`): Deduplicates concurrent identical requests
- Generate code: `goctl api go -api *.api -dir .`
Remember to run tests and ensure all checks pass before submitting changes. The project emphasizes high quality, performance, and reliability, so these should be primary considerations in all development work.
+4 -4
View File
@@ -35,11 +35,11 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v6
uses: actions/checkout@v5
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v4
uses: github/codeql-action/init@v3
with:
languages: ${{ matrix.language }}
# If you wish to specify custom queries, you can do so here or in a config file.
@@ -50,7 +50,7 @@ jobs:
# Autobuild attempts to build any compiled languages (C/C++, C#, or Java).
# If this step fails, then you should remove it and run the build manually (see below)
- name: Autobuild
uses: github/codeql-action/autobuild@v4
uses: github/codeql-action/autobuild@v3
# Command-line programs to run using the OS shell.
# 📚 https://git.io/JvXDl
@@ -64,4 +64,4 @@ jobs:
# make release
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v4
uses: github/codeql-action/analyze@v3
+2 -2
View File
@@ -12,7 +12,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Check out code into the Go module directory
uses: actions/checkout@v6
uses: actions/checkout@v5
- name: Set up Go 1.x
uses: actions/setup-go@v6
@@ -52,7 +52,7 @@ jobs:
runs-on: windows-latest
steps:
- name: Checkout codebase
uses: actions/checkout@v6
uses: actions/checkout@v5
- name: Set up Go 1.x
uses: actions/setup-go@v6
+1 -1
View File
@@ -16,7 +16,7 @@ jobs:
- goarch: "386"
goos: darwin
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v5
- uses: zeromicro/go-zero-release-action@master
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
+1 -6
View File
@@ -5,12 +5,7 @@ jobs:
name: runner / staticcheck
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: actions/setup-go@v6
with:
go-version-file: go.mod
check-latest: true
cache: true
- uses: actions/checkout@v5
- uses: reviewdog/action-staticcheck@v1
with:
github_token: ${{ secrets.github_token }}
+1 -1
View File
@@ -10,7 +10,7 @@ jobs:
version-check:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v5
- name: Set up Go
uses: actions/setup-go@v6
+1 -1
View File
@@ -40,7 +40,7 @@ type (
}
)
// New creates a Filter, store is the backed redis, key is the key for the bloom filter,
// New create a Filter, store is the backed redis, key is the key for the bloom filter,
// bits is how many bits will be used, maps is how many hashes for each addition.
// best practices:
// elements - means how many actual elements
-4
View File
@@ -81,10 +81,6 @@ func (c *Cache) Del(key string) {
delete(c.data, key)
c.lruCache.remove(key)
c.lock.Unlock()
// RemoveTimer is called outside the lock to avoid performance impact from this
// potentially time-consuming operation. Data integrity is maintained by lruCache,
// which will eventually evict any remaining entries when capacity is exceeded.
c.timingWheel.RemoveTimer(key)
}
-3
View File
@@ -164,7 +164,6 @@ func (tw *TimingWheel) Stop() {
func (tw *TimingWheel) drainAll(fn func(key, value any)) {
runner := threading.NewTaskRunner(drainWorkers)
for _, slot := range tw.slots {
for e := slot.Front(); e != nil; {
task := e.Value.(*timingEntry)
@@ -178,8 +177,6 @@ func (tw *TimingWheel) drainAll(fn func(key, value any)) {
}
}
}
runner.Wait()
}
func (tw *TimingWheel) getPositionAndCircle(d time.Duration) (pos, circle int) {
-151
View File
@@ -629,157 +629,6 @@ func TestMoveAndRemoveTask(t *testing.T) {
assert.Equal(t, 0, len(keys))
}
// TestTimingWheel_DrainClosureBug tests the closure capture bug in drainAll
// Issue: https://github.com/zeromicro/go-zero/issues/5314
func TestTimingWheel_DrainClosureBug(t *testing.T) {
ticker := timex.NewFakeTicker()
tw, _ := NewTimingWheelWithTicker(testStep, 10, func(k, v any) {}, ticker)
defer tw.Stop()
// Set multiple timers with different values
for i := 0; i < 10; i++ {
tw.SetTimer(i, i*10, testStep*5)
}
// Give time for timers to be set
time.Sleep(time.Millisecond * 100)
var mu sync.Mutex
received := make(map[int]int)
var wg sync.WaitGroup
wg.Add(10)
tw.Drain(func(key, value any) {
mu.Lock()
defer mu.Unlock()
k := key.(int)
v := value.(int)
received[k] = v
wg.Done()
})
wg.Wait()
// Check if all values match their keys
for k, v := range received {
expected := k * 10
assert.Equal(t, expected, v, "key %d should have value %d, got %d", k, expected, v)
}
}
// TestTimingWheel_RunTasksClosureBug tests the closure capture bug in runTasks
// Issue: https://github.com/zeromicro/go-zero/issues/5314
func TestTimingWheel_RunTasksClosureBug(t *testing.T) {
ticker := timex.NewFakeTicker()
var mu sync.Mutex
executed := make(map[int]int)
var wg sync.WaitGroup
tw, _ := NewTimingWheelWithTicker(testStep, 10, func(k, v any) {
mu.Lock()
defer mu.Unlock()
key := k.(int)
val := v.(int)
executed[key] = val
wg.Done()
}, ticker)
defer tw.Stop()
// Set multiple timers that should fire in the same tick
count := 10
wg.Add(count)
for i := 0; i < count; i++ {
tw.SetTimer(i, i*10, testStep)
}
// Advance ticker to trigger tasks
ticker.Tick()
// Wait for execution with timeout
done := make(chan struct{})
go func() {
wg.Wait()
close(done)
}()
select {
case <-done:
// Success
case <-time.After(2 * time.Second):
t.Fatal("timeout waiting for tasks to execute")
}
// Verify all tasks executed with correct values
assert.Equal(t, count, len(executed), "should have executed all tasks")
for k, v := range executed {
expected := k * 10
assert.Equal(t, expected, v, "key %d should have value %d, got %d", k, expected, v)
}
}
// TestTimingWheel_RunTasksRaceCondition tests for race conditions in runTasks
// This test specifically targets the loop variable capture bug
func TestTimingWheel_RunTasksRaceCondition(t *testing.T) {
// Run multiple times to increase likelihood of catching the bug
for attempt := 0; attempt < 10; attempt++ {
t.Run("", func(t *testing.T) {
ticker := timex.NewFakeTicker()
var mu sync.Mutex
keyValues := make(map[int][]int)
var wg sync.WaitGroup
tw, _ := NewTimingWheelWithTicker(testStep, 10, func(k, v any) {
// Add small delay to increase chance of race
time.Sleep(time.Microsecond)
mu.Lock()
defer mu.Unlock()
key := k.(int)
val := v.(int)
keyValues[key] = append(keyValues[key], val)
wg.Done()
}, ticker)
defer tw.Stop()
// Set many timers rapidly to increase chance of race
count := 50
wg.Add(count)
for i := 0; i < count; i++ {
tw.SetTimer(i, i*100, testStep)
}
ticker.Tick()
done := make(chan struct{})
go func() {
wg.Wait()
close(done)
}()
select {
case <-done:
case <-time.After(5 * time.Second):
t.Fatal("timeout waiting for tasks")
}
// Check for duplicates or wrong values
wrongCount := 0
for key, values := range keyValues {
assert.Equal(t, 1, len(values), "key %d should only execute once, got %v", key, values)
if len(values) > 0 {
expected := key * 100
if values[0] != expected {
wrongCount++
t.Logf("BUG DETECTED: key %d should have value %d, got %d", key, expected, values[0])
}
}
}
if wrongCount > 0 {
t.Errorf("Found %d tasks with wrong values due to closure bug", wrongCount)
}
})
}
}
func BenchmarkTimingWheel(b *testing.B) {
b.ReportAllocs()
+1 -1
View File
@@ -368,5 +368,5 @@ func getFullName(parent, child string) string {
return child
}
return parent + "." + child
return strings.Join([]string{parent, child}, ".")
}
-20
View File
@@ -1377,23 +1377,3 @@ func (m mockConfig) Validate() error {
return nil
}
func TestGetFullName(t *testing.T) {
tests := []struct {
parent string
child string
want string
}{
{"", "child", "child"},
{"parent", "child", "parent.child"},
{"a.b", "c", "a.b.c"},
{"root", "nested.field", "root.nested.field"},
}
for _, tt := range tests {
t.Run(tt.parent+"."+tt.child, func(t *testing.T) {
got := getFullName(tt.parent, tt.child)
assert.Equal(t, tt.want, got)
})
}
}
-48
View File
@@ -1,9 +1,6 @@
package subscriber
import (
"sync"
"sync/atomic"
"github.com/zeromicro/go-zero/core/discov"
"github.com/zeromicro/go-zero/core/logx"
)
@@ -40,7 +37,6 @@ func NewEtcdSubscriber(conf EtcdConf) (Subscriber, error) {
func buildSubOptions(conf EtcdConf) []discov.SubOption {
opts := []discov.SubOption{
discov.WithExactMatch(),
discov.WithContainer(newContainer()),
}
if len(conf.User) > 0 {
@@ -69,47 +65,3 @@ func (s *etcdSubscriber) Value() (string, error) {
return "", nil
}
type container struct {
value atomic.Value
listeners []func()
lock sync.Mutex
}
func newContainer() *container {
return &container{}
}
func (c *container) OnAdd(kv discov.KV) {
c.value.Store([]string{kv.Val})
c.notifyChange()
}
func (c *container) OnDelete(_ discov.KV) {
c.value.Store([]string(nil))
c.notifyChange()
}
func (c *container) AddListener(listener func()) {
c.lock.Lock()
c.listeners = append(c.listeners, listener)
c.lock.Unlock()
}
func (c *container) GetValues() []string {
if vals, ok := c.value.Load().([]string); ok {
return vals
}
return []string(nil)
}
func (c *container) notifyChange() {
c.lock.Lock()
listeners := append(([]func())(nil), c.listeners...)
c.lock.Unlock()
for _, listener := range listeners {
listener()
}
}
-186
View File
@@ -1,186 +0,0 @@
package subscriber
import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/zeromicro/go-zero/core/discov"
)
const (
actionAdd = iota
actionDel
)
func TestConfigCenterContainer(t *testing.T) {
type action struct {
act int
key string
val string
}
tests := []struct {
name string
do []action
expect []string
}{
{
name: "add one",
do: []action{
{
act: actionAdd,
key: "first",
val: "a",
},
},
expect: []string{
"a",
},
},
{
name: "add two",
do: []action{
{
act: actionAdd,
key: "first",
val: "a",
},
{
act: actionAdd,
key: "second",
val: "b",
},
},
expect: []string{
"b",
},
},
{
name: "add two, delete one",
do: []action{
{
act: actionAdd,
key: "first",
val: "a",
},
{
act: actionAdd,
key: "second",
val: "b",
},
{
act: actionDel,
key: "first",
},
},
expect: []string(nil),
},
{
name: "add two, delete two",
do: []action{
{
act: actionAdd,
key: "first",
val: "a",
},
{
act: actionAdd,
key: "second",
val: "b",
},
{
act: actionDel,
key: "first",
},
{
act: actionDel,
key: "second",
},
},
expect: []string(nil),
},
{
name: "add two, dup values",
do: []action{
{
act: actionAdd,
key: "first",
val: "a",
},
{
act: actionAdd,
key: "second",
val: "b",
},
{
act: actionAdd,
key: "third",
val: "a",
},
},
expect: []string{"a"},
},
{
name: "add three, dup values, delete two, add one",
do: []action{
{
act: actionAdd,
key: "first",
val: "a",
},
{
act: actionAdd,
key: "second",
val: "b",
},
{
act: actionAdd,
key: "third",
val: "a",
},
{
act: actionDel,
key: "first",
},
{
act: actionDel,
key: "second",
},
{
act: actionAdd,
key: "forth",
val: "c",
},
},
expect: []string{"c"},
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
var changed bool
c := newContainer()
c.AddListener(func() {
changed = true
})
assert.Nil(t, c.GetValues())
assert.False(t, changed)
for _, order := range test.do {
if order.act == actionAdd {
c.OnAdd(discov.KV{
Key: order.key,
Val: order.val,
})
} else {
c.OnDelete(discov.KV{
Key: order.key,
Val: order.val,
})
}
}
assert.True(t, changed)
assert.ElementsMatch(t, test.expect, c.GetValues())
})
}
}
+1 -2
View File
@@ -386,9 +386,8 @@ func (c *cluster) watch(cli EtcdClient, key watchKey, rev int64) {
rev = c.load(cli, key)
}
// log the error and retry with cooldown to prevent CPU/disk exhaustion
// log the error and retry
logc.Error(cli.Ctx(), err)
time.Sleep(coolDownUnstable.AroundDuration(coolDownInterval))
}
}
+14 -33
View File
@@ -19,9 +19,8 @@ type (
exclusive bool
key string
exactMatch bool
items Container
items *container
}
KV = internal.KV
)
// NewSubscriber returns a Subscriber.
@@ -36,9 +35,7 @@ func NewSubscriber(endpoints []string, key string, opts ...SubOption) (*Subscrib
for _, opt := range opts {
opt(sub)
}
if sub.items == nil {
sub.items = newContainer(sub.exclusive)
}
sub.items = newContainer(sub.exclusive)
if err := internal.GetRegistry().Monitor(endpoints, key, sub.exactMatch, sub.items); err != nil {
return nil, err
@@ -49,7 +46,7 @@ func NewSubscriber(endpoints []string, key string, opts ...SubOption) (*Subscrib
// AddListener adds listener to s.
func (s *Subscriber) AddListener(listener func()) {
s.items.AddListener(listener)
s.items.addListener(listener)
}
// Close closes the subscriber.
@@ -59,7 +56,7 @@ func (s *Subscriber) Close() {
// Values returns all the subscription values.
func (s *Subscriber) Values() []string {
return s.items.GetValues()
return s.items.getValues()
}
// Exclusive means that key value can only be 1:1,
@@ -91,32 +88,16 @@ func WithSubEtcdTLS(certFile, certKeyFile, caFile string, insecureSkipVerify boo
}
}
// WithContainer provides a custom container to the subscriber.
func WithContainer(container Container) SubOption {
return func(sub *Subscriber) {
sub.items = container
}
type container struct {
exclusive bool
values map[string][]string
mapping map[string]string
snapshot atomic.Value
dirty *syncx.AtomicBool
listeners []func()
lock sync.Mutex
}
type (
Container interface {
OnAdd(kv internal.KV)
OnDelete(kv internal.KV)
AddListener(listener func())
GetValues() []string
}
container struct {
exclusive bool
values map[string][]string
mapping map[string]string
snapshot atomic.Value
dirty *syncx.AtomicBool
listeners []func()
lock sync.Mutex
}
)
func newContainer(exclusive bool) *container {
return &container{
exclusive: exclusive,
@@ -160,7 +141,7 @@ func (c *container) addKv(key, value string) ([]string, bool) {
return nil, false
}
func (c *container) AddListener(listener func()) {
func (c *container) addListener(listener func()) {
c.lock.Lock()
c.listeners = append(c.listeners, listener)
c.lock.Unlock()
@@ -189,7 +170,7 @@ func (c *container) doRemoveKey(key string) {
}
}
func (c *container) GetValues() []string {
func (c *container) getValues() []string {
if !c.dirty.True() {
return c.snapshot.Load().([]string)
}
+8 -11
View File
@@ -171,10 +171,10 @@ func TestContainer(t *testing.T) {
t.Run(test.name, func(t *testing.T) {
var changed bool
c := newContainer(exclusive)
c.AddListener(func() {
c.addListener(func() {
changed = true
})
assert.Nil(t, c.GetValues())
assert.Nil(t, c.getValues())
assert.False(t, changed)
for _, order := range test.do {
@@ -193,9 +193,9 @@ func TestContainer(t *testing.T) {
assert.True(t, changed)
assert.True(t, c.dirty.True())
assert.ElementsMatch(t, test.expect, c.GetValues())
assert.ElementsMatch(t, test.expect, c.getValues())
assert.False(t, c.dirty.True())
assert.ElementsMatch(t, test.expect, c.GetValues())
assert.ElementsMatch(t, test.expect, c.getValues())
})
}
}
@@ -204,14 +204,12 @@ func TestContainer(t *testing.T) {
func TestSubscriber(t *testing.T) {
sub := new(Subscriber)
Exclusive()(sub)
c := newContainer(sub.exclusive)
WithContainer(c)(sub)
sub.items = c
sub.items = newContainer(sub.exclusive)
var count int32
sub.AddListener(func() {
atomic.AddInt32(&count, 1)
})
c.notifyChange()
sub.items.notifyChange()
assert.Empty(t, sub.Values())
assert.Equal(t, int32(1), atomic.LoadInt32(&count))
}
@@ -231,13 +229,12 @@ func TestWithSubEtcdAccount(t *testing.T) {
func TestWithExactMatch(t *testing.T) {
sub := new(Subscriber)
WithExactMatch()(sub)
c := newContainer(sub.exclusive)
sub.items = c
sub.items = newContainer(sub.exclusive)
var count int32
sub.AddListener(func() {
atomic.AddInt32(&count, 1)
})
c.notifyChange()
sub.items.notifyChange()
assert.Empty(t, sub.Values())
assert.Equal(t, int32(1), atomic.LoadInt32(&count))
}
+2 -2
View File
@@ -168,7 +168,7 @@ func (s Stream) Count() (count int) {
return
}
// Distinct removes the duplicated items based on the given KeyFunc.
// Distinct removes the duplicated items base on the given KeyFunc.
func (s Stream) Distinct(fn KeyFunc) Stream {
source := make(chan any)
@@ -459,7 +459,7 @@ func (s Stream) Tail(n int64) Stream {
return Range(source)
}
// Walk lets the callers handle each item, the caller may write zero, one or more items based on the given item.
// Walk lets the callers handle each item, the caller may write zero, one or more items base on the given item.
func (s Stream) Walk(fn WalkFunc, opts ...Option) Stream {
option := buildOptions(opts...)
if option.unlimitedWorkers {
+3 -2
View File
@@ -1,6 +1,8 @@
package fx
import (
"io"
"log"
"math/rand"
"reflect"
"runtime"
@@ -11,7 +13,6 @@ import (
"time"
"github.com/stretchr/testify/assert"
"github.com/zeromicro/go-zero/core/logx/logtest"
"github.com/zeromicro/go-zero/core/stringx"
"go.uber.org/goleak"
)
@@ -237,7 +238,7 @@ func TestLast(t *testing.T) {
func TestMap(t *testing.T) {
runCheckedTest(t, func(t *testing.T) {
logtest.Discard(t)
log.SetOutput(io.Discard)
tests := []struct {
mapper MapFunc
+1 -1
View File
@@ -96,7 +96,7 @@ func (h *ConsistentHash) AddWithWeight(node any, weight int) {
h.AddWithReplicas(node, replicas)
}
// Get returns the corresponding node from h based on the given v.
// Get returns the corresponding node from h base on the given v.
func (h *ConsistentHash) Get(v any) (any, bool) {
h.lock.RLock()
defer h.lock.RUnlock()
+1 -1
View File
@@ -66,7 +66,7 @@ type (
gzip bool
}
// SizeLimitRotateRule a rotation rule that makes the log file rotated based on size
// SizeLimitRotateRule a rotation rule that make the log file rotated base on size
SizeLimitRotateRule struct {
DailyRotateRule
maxSize int64
-2
View File
@@ -444,8 +444,6 @@ func wrapLevelWithColor(level string) string {
colour = color.FgRed
case levelError:
colour = color.FgRed
case levelSevere:
colour = color.FgRed
case levelFatal:
colour = color.FgRed
case levelInfo:
+5 -4
View File
@@ -104,13 +104,14 @@ func convertToString(val any, fullName string) (string, error) {
func convertTypeFromString(kind reflect.Kind, str string) (any, error) {
switch kind {
case reflect.Bool:
if str == "1" || strings.EqualFold(str, "true") {
switch strings.ToLower(str) {
case "1", "true":
return true, nil
}
if str == "0" || strings.EqualFold(str, "false") {
case "0", "false":
return false, nil
default:
return false, errTypeMismatch
}
return false, errTypeMismatch
case reflect.Int:
return strconv.ParseInt(str, 10, intSize)
case reflect.Int8:
-40
View File
@@ -334,43 +334,3 @@ func TestValidateValueRange(t *testing.T) {
func TestSetMatchedPrimitiveValue(t *testing.T) {
assert.Error(t, setMatchedPrimitiveValue(reflect.Func, reflect.ValueOf(2), "1"))
}
func TestConvertTypeFromString_Bool(t *testing.T) {
tests := []struct {
name string
input string
want bool
wantErr bool
}{
// true cases
{name: "1", input: "1", want: true, wantErr: false},
{name: "true lowercase", input: "true", want: true, wantErr: false},
{name: "True mixed", input: "True", want: true, wantErr: false},
{name: "TRUE uppercase", input: "TRUE", want: true, wantErr: false},
{name: "TrUe mixed", input: "TrUe", want: true, wantErr: false},
// false cases
{name: "0", input: "0", want: false, wantErr: false},
{name: "false lowercase", input: "false", want: false, wantErr: false},
{name: "False mixed", input: "False", want: false, wantErr: false},
{name: "FALSE uppercase", input: "FALSE", want: false, wantErr: false},
{name: "FaLsE mixed", input: "FaLsE", want: false, wantErr: false},
// error cases
{name: "invalid yes", input: "yes", want: false, wantErr: true},
{name: "invalid no", input: "no", want: false, wantErr: true},
{name: "invalid empty", input: "", want: false, wantErr: true},
{name: "invalid 2", input: "2", want: false, wantErr: true},
{name: "invalid truee", input: "truee", want: false, wantErr: true},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, err := convertTypeFromString(reflect.Bool, tt.input)
if tt.wantErr {
assert.Error(t, err)
} else {
assert.NoError(t, err)
assert.Equal(t, tt.want, got)
}
})
}
}
+1 -1
View File
@@ -6,7 +6,7 @@ import (
"time"
)
// An Unstable is used to generate random value around the mean value based on given deviation.
// An Unstable is used to generate random value around the mean value base on given deviation.
type Unstable struct {
deviation float64
r *rand.Rand
+5
View File
@@ -4,6 +4,8 @@ import (
"context"
"errors"
"fmt"
"io"
"log"
"runtime"
"sync/atomic"
"testing"
@@ -15,6 +17,9 @@ import (
var errDummy = errors.New("dummy")
func init() {
log.SetOutput(io.Discard)
}
func TestFinish(t *testing.T) {
defer goleak.VerifyNone(t)
+1 -1
View File
@@ -532,7 +532,7 @@ func createModel(t *testing.T, coll mon.Collection) *Model {
}
}
// mustNewTestModel returns a test Model with the given cache.
// mustNewTestModel returns a test Model with the given cache.
func mustNewTestModel(collection mon.Collection, c cache.CacheConf, opts ...cache.Option) *Model {
return &Model{
Model: &mon.Model{
-83
View File
@@ -259,34 +259,12 @@ func (s *Redis) BitPosCtx(ctx context.Context, key string, bit, start, end int64
}
// Blpop uses passed in redis connection to execute blocking queries.
//
// For blocking operations, you must create a dedicated RedisNode using CreateBlockingNode to avoid
// exhausting the connection pool. Blocking commands hold connections for extended periods and should
// not share the regular connection pool.
//
// Example usage:
//
// node, err := redis.CreateBlockingNode(rds)
// if err != nil {
// // handle error
// }
// defer node.Close()
//
// value, err := rds.Blpop(node, "mylist")
// if err != nil {
// // handle error
// }
//
// Doesn't benefit from pooling redis connections of blocking queries
func (s *Redis) Blpop(node RedisNode, key string) (string, error) {
return s.BlpopCtx(context.Background(), node, key)
}
// BlpopCtx uses passed in redis connection to execute blocking queries.
//
// For blocking operations, you must create a dedicated RedisNode using CreateBlockingNode.
// See Blpop for usage examples.
//
// Doesn't benefit from pooling redis connections of blocking queries
func (s *Redis) BlpopCtx(ctx context.Context, node RedisNode, key string) (string, error) {
return s.BlpopWithTimeoutCtx(ctx, node, blockingQueryTimeout, key)
@@ -294,18 +272,12 @@ func (s *Redis) BlpopCtx(ctx context.Context, node RedisNode, key string) (strin
// BlpopEx uses passed in redis connection to execute blpop command.
// The difference against Blpop is that this method returns a bool to indicate success.
//
// For blocking operations, you must create a dedicated RedisNode using CreateBlockingNode.
// See Blpop for usage examples.
func (s *Redis) BlpopEx(node RedisNode, key string) (string, bool, error) {
return s.BlpopExCtx(context.Background(), node, key)
}
// BlpopExCtx uses passed in redis connection to execute blpop command.
// The difference against Blpop is that this method returns a bool to indicate success.
//
// For blocking operations, you must create a dedicated RedisNode using CreateBlockingNode.
// See Blpop for usage examples.
func (s *Redis) BlpopExCtx(ctx context.Context, node RedisNode, key string) (string, bool, error) {
if node == nil {
return "", false, ErrNilNode
@@ -325,18 +297,12 @@ func (s *Redis) BlpopExCtx(ctx context.Context, node RedisNode, key string) (str
// BlpopWithTimeout uses passed in redis connection to execute blpop command.
// Control blocking query timeout
//
// For blocking operations, you must create a dedicated RedisNode using CreateBlockingNode.
// See Blpop for usage examples.
func (s *Redis) BlpopWithTimeout(node RedisNode, timeout time.Duration, key string) (string, error) {
return s.BlpopWithTimeoutCtx(context.Background(), node, timeout, key)
}
// BlpopWithTimeoutCtx uses passed in redis connection to execute blpop command.
// Control blocking query timeout
//
// For blocking operations, you must create a dedicated RedisNode using CreateBlockingNode.
// See Blpop for usage examples.
func (s *Redis) BlpopWithTimeoutCtx(ctx context.Context, node RedisNode, timeout time.Duration,
key string) (string, error) {
if node == nil {
@@ -664,28 +630,6 @@ func (s *Redis) GetDelCtx(ctx context.Context, key string) (string, error) {
return val, err
}
// GetEx is the implementation of redis getex command.
// Available since: redis version 6.2.0
func (s *Redis) GetEx(key string, seconds int) (string, error) {
return s.GetExCtx(context.Background(), key, seconds)
}
// GetExCtx is the implementation of redis getex command.
// Available since: redis version 6.2.0
func (s *Redis) GetExCtx(ctx context.Context, key string, seconds int) (string, error) {
conn, err := getRedis(s)
if err != nil {
return "", err
}
val, err := conn.GetEx(ctx, key, time.Duration(seconds)*time.Second).Result()
if errors.Is(err, red.Nil) {
return "", nil
}
return val, err
}
// GetSet is the implementation of redis getset command.
func (s *Redis) GetSet(key, value string) (string, error) {
return s.GetSetCtx(context.Background(), key, value)
@@ -1896,29 +1840,6 @@ func (s *Redis) XInfoStreamCtx(ctx context.Context, stream string) (*red.XInfoSt
// XReadGroup reads messages from Redis streams as part of a consumer group.
// It allows for distributed processing of stream messages with automatic message delivery semantics.
//
// For blocking operations, you must create a dedicated RedisNode using CreateBlockingNode to avoid
// exhausting the connection pool. Blocking commands hold connections for extended periods and should
// not share the regular connection pool.
//
// Example usage:
//
// node, err := redis.CreateBlockingNode(rds)
// if err != nil {
// // handle error
// }
// defer node.Close()
//
// streams, err := rds.XReadGroup(
// node, // RedisNode created with CreateBlockingNode
// "mygroup", // consumer group name
// "consumer1", // consumer ID
// 10, // max number of messages to read
// 5*time.Second, // block duration
// false, // noAck flag
// "mystream", // stream name
// )
//
// Doesn't benefit from pooling redis connections of blocking queries.
func (s *Redis) XReadGroup(node RedisNode, group string, consumerId string, count int64,
block time.Duration, noAck bool, streams ...string) ([]red.XStream, error) {
@@ -1926,10 +1847,6 @@ func (s *Redis) XReadGroup(node RedisNode, group string, consumerId string, coun
}
// XReadGroupCtx is the context-aware version of XReadGroup.
//
// For blocking operations, you must create a dedicated RedisNode using CreateBlockingNode to avoid
// exhausting the connection pool. See XReadGroup for usage examples.
//
// Doesn't benefit from pooling redis connections of blocking queries.
func (s *Redis) XReadGroupCtx(ctx context.Context, node RedisNode, group string, consumerId string,
count int64, block time.Duration, noAck bool, streams ...string) ([]red.XStream, error) {
-39
View File
@@ -1104,45 +1104,6 @@ func TestRedis_GetDel(t *testing.T) {
})
}
func TestRedis_GetEx(t *testing.T) {
t.Run("get_ex", func(t *testing.T) {
runOnRedis(t, func(client *Redis) {
val, err := client.GetEx("getex_key", 10)
assert.Equal(t, "", val)
assert.Nil(t, err)
err = client.Set("getex_key", "getex_value")
assert.Nil(t, err)
val, err = client.GetEx("getex_key", 10)
assert.Nil(t, err)
assert.Equal(t, "getex_value", val)
val, err = client.Get("getex_key")
assert.Nil(t, err)
assert.Equal(t, "getex_value", val)
ttl, err := client.Ttl("getex_key")
assert.Nil(t, err)
assert.True(t, ttl > 0 && ttl <= 10)
val, err = client.GetEx("getex_key", 5)
assert.Nil(t, err)
assert.Equal(t, "getex_value", val)
ttl, err = client.Ttl("getex_key")
assert.Nil(t, err)
assert.True(t, ttl > 0 && ttl <= 5)
})
})
t.Run("get_ex_with_error", func(t *testing.T) {
runOnRedisWithError(t, func(client *Redis) {
_, err := newRedis(client.Addr, badType()).GetEx("hello", 10)
assert.Error(t, err)
})
})
}
func TestRedis_GetSet(t *testing.T) {
t.Run("set_get", func(t *testing.T) {
runOnRedis(t, func(client *Redis) {
+1 -31
View File
@@ -13,37 +13,7 @@ type ClosableNode interface {
Close()
}
// CreateBlockingNode creates a dedicated RedisNode for blocking operations.
//
// Blocking Redis commands (like BLPOP, BRPOP, XREADGROUP with block parameter) hold connections
// for extended periods while waiting for data. Using them with the regular Redis connection pool
// can exhaust all available connections, causing other operations to fail or timeout.
//
// CreateBlockingNode creates a separate Redis client with a minimal connection pool (size 1) that
// is dedicated to blocking operations. This ensures blocking commands don't interfere with regular
// Redis operations.
//
// Example usage:
//
// rds := redis.MustNewRedis(redis.RedisConf{
// Host: "localhost:6379",
// Type: redis.NodeType,
// })
//
// // Create a dedicated node for blocking operations
// node, err := redis.CreateBlockingNode(rds)
// if err != nil {
// // handle error
// }
// defer node.Close() // Important: close the node when done
//
// // Use the node for blocking operations
// value, err := rds.Blpop(node, "mylist")
// if err != nil {
// // handle error
// }
//
// The returned ClosableNode must be closed when no longer needed to release resources.
// CreateBlockingNode returns a ClosableNode.
func CreateBlockingNode(r *Redis) (ClosableNode, error) {
timeout := readWriteTimeout + blockingQueryTimeout
+17 -8
View File
@@ -70,16 +70,25 @@ func getTaggedFieldValueMap(v reflect.Value) (map[string]any, error) {
}
func getValueInterface(value reflect.Value) (any, error) {
if !value.CanAddr() || !value.Addr().CanInterface() {
return nil, ErrNotReadableValue
}
switch value.Kind() {
case reflect.Ptr:
if !value.CanInterface() {
return nil, ErrNotReadableValue
}
if value.Kind() == reflect.Pointer && value.IsNil() {
baseValueType := mapping.Deref(value.Type())
value.Set(reflect.New(baseValueType))
}
if value.IsNil() {
baseValueType := mapping.Deref(value.Type())
value.Set(reflect.New(baseValueType))
}
return value.Addr().Interface(), nil
return value.Interface(), nil
default:
if !value.CanAddr() || !value.Addr().CanInterface() {
return nil, ErrNotReadableValue
}
return value.Addr().Interface(), nil
}
}
func isScanFailed(err error) bool {
-778
View File
@@ -4,9 +4,7 @@ import (
"context"
"database/sql"
"errors"
"reflect"
"testing"
"time"
"github.com/DATA-DOG/go-sqlmock"
"github.com/stretchr/testify/assert"
@@ -1577,782 +1575,6 @@ func TestAnonymousStructPrError(t *testing.T) {
})
}
func TestUnmarshalRowsZeroValueStructPtr(t *testing.T) {
secondNamePtr := "second_ptr"
secondAgePtr := int64(30)
thirdNamePtr := "third_ptr"
thirdAgePtr := int64(0)
expect := []struct {
Name string
NamePtr *string
Age int64
AgePtr *int64
}{
{
Name: "first",
NamePtr: nil,
Age: 2,
AgePtr: nil,
},
{
Name: "second",
NamePtr: &secondNamePtr,
Age: 3,
AgePtr: &secondAgePtr,
},
{
Name: "",
NamePtr: &thirdNamePtr,
Age: 0,
AgePtr: &thirdAgePtr,
},
}
var value []struct {
Age int64 `db:"age"`
AgePtr *int64 `db:"age_ptr"`
Name string `db:"name"`
NamePtr *string `db:"name_ptr"`
}
dbtest.RunTest(t, func(db *sql.DB, mock sqlmock.Sqlmock) {
rs := sqlmock.NewRows([]string{"name", "name_ptr", "age", "age_ptr"}).
AddRow("first", nil, 2, nil).
AddRow("second", "second_ptr", 3, 30).
AddRow("", "third_ptr", 0, 0)
mock.ExpectQuery("select (.+) from users where user=?").
WithArgs("anyone").WillReturnRows(rs)
assert.Nil(t, query(context.Background(), db, func(rows *sql.Rows) error {
return unmarshalRows(&value, rows, true)
}, "select name, name_ptr, age, age_ptr from users where user=?", "anyone"))
assert.Equal(t, 3, len(value), "应该返回3行数据")
for i, each := range expect {
assert.Equal(t, each.Name, value[i].Name)
assert.Equal(t, each.Age, value[i].Age)
if each.NamePtr == nil {
assert.Nil(t, value[i].NamePtr)
} else {
assert.NotNil(t, value[i].NamePtr)
assert.Equal(t, *each.NamePtr, *value[i].NamePtr)
}
if each.AgePtr == nil {
assert.Nil(t, value[i].AgePtr)
} else {
assert.NotNil(t, value[i].AgePtr)
assert.Equal(t, *each.AgePtr, *value[i].AgePtr)
}
}
})
}
func TestUnmarshalRowsAllNullStructPtrFields(t *testing.T) {
expect := []struct {
NamePtr *string
AgePtr *int64
}{
{
NamePtr: nil,
AgePtr: nil,
},
{
NamePtr: stringPtr("second"),
AgePtr: int64Ptr(30),
},
{
NamePtr: nil,
AgePtr: nil,
},
}
var value []struct {
AgePtr *int64 `db:"age_ptr"`
NamePtr *string `db:"name_ptr"`
}
dbtest.RunTest(t, func(db *sql.DB, mock sqlmock.Sqlmock) {
rs := sqlmock.NewRows([]string{"name_ptr", "age_ptr"}).
AddRow(nil, nil).
AddRow("second", 30).
AddRow(nil, nil)
mock.ExpectQuery("select (.+) from users where user=?").
WithArgs("anyone").WillReturnRows(rs)
assert.Nil(t, query(context.Background(), db, func(rows *sql.Rows) error {
return unmarshalRows(&value, rows, true)
}, "select name_ptr, age_ptr from users where user=?", "anyone"))
assert.Equal(t, 3, len(value))
for i, each := range expect {
if each.NamePtr == nil {
assert.Nil(t, value[i].NamePtr)
} else {
assert.NotNil(t, value[i].NamePtr)
assert.Equal(t, *each.NamePtr, *value[i].NamePtr)
}
if each.AgePtr == nil {
assert.Nil(t, value[i].AgePtr)
} else {
assert.NotNil(t, value[i].AgePtr)
assert.Equal(t, *each.AgePtr, *value[i].AgePtr)
}
}
})
}
func TestUnmarshalRowsWithSqlNullTypes(t *testing.T) {
expect := []struct {
Name string
NullName sql.NullString
Age int64
NullAge sql.NullInt64
Score float64
NullScore sql.NullFloat64
Active bool
NullActive sql.NullBool
}{
{
Name: "first",
NullName: sql.NullString{
String: "",
Valid: false,
},
Age: 20,
NullAge: sql.NullInt64{
Int64: 0,
Valid: false,
},
Score: 85.5,
NullScore: sql.NullFloat64{
Float64: 0,
Valid: false,
},
Active: true,
NullActive: sql.NullBool{
Bool: false,
Valid: false,
},
},
{
Name: "second",
NullName: sql.NullString{
String: "not_null_name",
Valid: true,
},
Age: 25,
NullAge: sql.NullInt64{
Int64: 30,
Valid: true,
},
Score: 90.0,
NullScore: sql.NullFloat64{
Float64: 95.5,
Valid: true,
},
Active: false,
NullActive: sql.NullBool{
Bool: true,
Valid: true,
},
},
{
Name: "third",
NullName: sql.NullString{
String: "",
Valid: false,
},
Age: 0,
NullAge: sql.NullInt64{
Int64: 0,
Valid: false,
},
Score: 0,
NullScore: sql.NullFloat64{
Float64: 0,
Valid: false,
},
Active: false,
NullActive: sql.NullBool{
Bool: false,
Valid: false,
},
},
}
var value []struct {
Name string `db:"name"`
NullName sql.NullString `db:"null_name"`
Age int64 `db:"age"`
NullAge sql.NullInt64 `db:"null_age"`
Score float64 `db:"score"`
NullScore sql.NullFloat64 `db:"null_score"`
Active bool `db:"active"`
NullActive sql.NullBool `db:"null_active"`
}
dbtest.RunTest(t, func(db *sql.DB, mock sqlmock.Sqlmock) {
rs := sqlmock.NewRows([]string{
"name", "null_name", "age", "null_age", "score", "null_score", "active", "null_active",
}).
AddRow("first", nil, 20, nil, 85.5, nil, true, nil).
AddRow("second", "not_null_name", 25, 30, 90.0, 95.5, false, true).
AddRow("third", nil, 0, nil, 0, nil, false, nil)
mock.ExpectQuery("select (.+) from users where type=?").
WithArgs("test").WillReturnRows(rs)
assert.Nil(t, query(context.Background(), db, func(rows *sql.Rows) error {
return unmarshalRows(&value, rows, true)
}, "select name, null_name, age, null_age, score, null_score, active, null_active from users where type=?", "test"))
assert.Equal(t, 3, len(value))
for i, each := range expect {
assert.Equal(t, each.Name, value[i].Name)
assert.Equal(t, each.Age, value[i].Age)
assert.Equal(t, each.Score, value[i].Score)
assert.Equal(t, each.Active, value[i].Active)
assert.Equal(t, each.NullName.Valid, value[i].NullName.Valid)
if each.NullName.Valid {
assert.Equal(t, each.NullName.String, value[i].NullName.String)
}
assert.Equal(t, each.NullAge.Valid, value[i].NullAge.Valid)
if each.NullAge.Valid {
assert.Equal(t, each.NullAge.Int64, value[i].NullAge.Int64)
}
assert.Equal(t, each.NullScore.Valid, value[i].NullScore.Valid)
if each.NullScore.Valid {
assert.Equal(t, each.NullScore.Float64, value[i].NullScore.Float64)
}
assert.Equal(t, each.NullActive.Valid, value[i].NullActive.Valid)
if each.NullActive.Valid {
assert.Equal(t, each.NullActive.Bool, value[i].NullActive.Bool)
}
}
})
}
func TestUnmarshalRowsSqlNullWithMixedData(t *testing.T) {
expect := []struct {
Name string
NullName sql.NullString
Age int64
NullAge sql.NullInt64
IsStudent bool
NullActive sql.NullBool
}{
{
Name: "student1",
NullName: sql.NullString{
String: "",
Valid: false,
},
Age: 18,
NullAge: sql.NullInt64{
Int64: 0,
Valid: false,
},
IsStudent: true,
NullActive: sql.NullBool{
Bool: false,
Valid: false,
},
},
{
Name: "student2",
NullName: sql.NullString{
String: "has_nickname",
Valid: true,
},
Age: 20,
NullAge: sql.NullInt64{
Int64: 22,
Valid: true,
},
IsStudent: false,
NullActive: sql.NullBool{
Bool: true,
Valid: true,
},
},
}
var value []struct {
Name string `db:"name"`
NullName sql.NullString `db:"null_name"`
Age int64 `db:"age"`
NullAge sql.NullInt64 `db:"null_age"`
IsStudent bool `db:"is_student"`
NullActive sql.NullBool `db:"null_active"`
}
dbtest.RunTest(t, func(db *sql.DB, mock sqlmock.Sqlmock) {
rs := sqlmock.NewRows([]string{"name", "null_name", "age", "null_age", "is_student", "null_active"}).
AddRow("student1", nil, 18, nil, true, nil).
AddRow("student2", "has_nickname", 20, 22, false, true)
mock.ExpectQuery("select (.+) from students where class=?").
WithArgs("A").WillReturnRows(rs)
assert.Nil(t, query(context.Background(), db, func(rows *sql.Rows) error {
return unmarshalRows(&value, rows, true)
}, "select name, null_name, age, null_age, is_student, null_active from students where class=?", "A"))
assert.Equal(t, 2, len(value))
for i, each := range expect {
assert.Equal(t, each.Name, value[i].Name)
assert.Equal(t, each.Age, value[i].Age)
assert.Equal(t, each.IsStudent, value[i].IsStudent)
assert.Equal(t, each.NullName.Valid, value[i].NullName.Valid)
if each.NullName.Valid {
assert.Equal(t, each.NullName.String, value[i].NullName.String)
}
assert.Equal(t, each.NullAge.Valid, value[i].NullAge.Valid)
if each.NullAge.Valid {
assert.Equal(t, each.NullAge.Int64, value[i].NullAge.Int64)
}
assert.Equal(t, each.NullActive.Valid, value[i].NullActive.Valid)
if each.NullActive.Valid {
assert.Equal(t, each.NullActive.Bool, value[i].NullActive.Bool)
}
}
})
}
func TestUnmarshalRowsSqlNullTime(t *testing.T) {
now := time.Now()
futureTime := now.AddDate(1, 0, 0)
expect := []struct {
Name string
BirthDate sql.NullTime
LastLogin sql.NullTime
}{
{
Name: "user1",
BirthDate: sql.NullTime{
Time: time.Time{},
Valid: false,
},
LastLogin: sql.NullTime{
Time: now,
Valid: true,
},
},
{
Name: "user2",
BirthDate: sql.NullTime{
Time: futureTime,
Valid: true,
},
LastLogin: sql.NullTime{
Time: time.Time{},
Valid: false,
},
},
}
var value []struct {
Name string `db:"name"`
BirthDate sql.NullTime `db:"birth_date"`
LastLogin sql.NullTime `db:"last_login"`
}
dbtest.RunTest(t, func(db *sql.DB, mock sqlmock.Sqlmock) {
rs := sqlmock.NewRows([]string{"name", "birth_date", "last_login"}).
AddRow("user1", nil, now).
AddRow("user2", futureTime, nil)
mock.ExpectQuery("select (.+) from users").
WillReturnRows(rs)
assert.Nil(t, query(context.Background(), db, func(rows *sql.Rows) error {
return unmarshalRows(&value, rows, true)
}, "select name, birth_date, last_login from users"))
assert.Equal(t, 2, len(value))
for i, each := range expect {
assert.Equal(t, each.Name, value[i].Name)
assert.Equal(t, each.BirthDate.Valid, value[i].BirthDate.Valid)
if each.BirthDate.Valid {
assert.WithinDuration(t, each.BirthDate.Time, value[i].BirthDate.Time, time.Second)
}
assert.Equal(t, each.LastLogin.Valid, value[i].LastLogin.Valid)
if each.LastLogin.Valid {
assert.WithinDuration(t, each.LastLogin.Time, value[i].LastLogin.Time, time.Second)
}
}
})
}
func TestUnmarshalRowsSqlNullWithEmptyValues(t *testing.T) {
expect := []struct {
Name string
NullString sql.NullString
NullInt sql.NullInt64
NullFloat sql.NullFloat64
NullBool sql.NullBool
}{
{
Name: "empty_values",
NullString: sql.NullString{
String: "",
Valid: true,
},
NullInt: sql.NullInt64{
Int64: 0,
Valid: true,
},
NullFloat: sql.NullFloat64{
Float64: 0.0,
Valid: true,
},
NullBool: sql.NullBool{
Bool: false,
Valid: true,
},
},
{
Name: "null_values",
NullString: sql.NullString{
String: "",
Valid: false,
},
NullInt: sql.NullInt64{
Int64: 0,
Valid: false,
},
NullFloat: sql.NullFloat64{
Float64: 0.0,
Valid: false,
},
NullBool: sql.NullBool{
Bool: false,
Valid: false,
},
},
{
Name: "mixed_values",
NullString: sql.NullString{
String: "actual_value",
Valid: true,
},
NullInt: sql.NullInt64{
Int64: 0,
Valid: true,
},
NullFloat: sql.NullFloat64{
Float64: 0.0,
Valid: false,
},
NullBool: sql.NullBool{
Bool: true,
Valid: true,
},
},
}
var value []struct {
Name string `db:"name"`
NullString sql.NullString `db:"null_string"`
NullInt sql.NullInt64 `db:"null_int"`
NullFloat sql.NullFloat64 `db:"null_float"`
NullBool sql.NullBool `db:"null_bool"`
}
dbtest.RunTest(t, func(db *sql.DB, mock sqlmock.Sqlmock) {
rs := sqlmock.NewRows([]string{"name", "null_string", "null_int", "null_float", "null_bool"}).
AddRow("empty_values", "", 0, 0.0, false).
AddRow("null_values", nil, nil, nil, nil).
AddRow("mixed_values", "actual_value", 0, nil, true)
mock.ExpectQuery("select (.+) from test_table").
WillReturnRows(rs)
assert.Nil(t, query(context.Background(), db, func(rows *sql.Rows) error {
return unmarshalRows(&value, rows, true)
}, "select name, null_string, null_int, null_float, null_bool from test_table"))
assert.Equal(t, 3, len(value))
for i, each := range expect {
assert.Equal(t, each.Name, value[i].Name)
assert.Equal(t, each.NullString.Valid, value[i].NullString.Valid)
if each.NullString.Valid {
assert.Equal(t, each.NullString.String, value[i].NullString.String)
} else {
assert.Equal(t, "", value[i].NullString.String)
}
assert.Equal(t, each.NullInt.Valid, value[i].NullInt.Valid)
if each.NullInt.Valid {
assert.Equal(t, each.NullInt.Int64, value[i].NullInt.Int64)
} else {
assert.Equal(t, int64(0), value[i].NullInt.Int64)
}
assert.Equal(t, each.NullFloat.Valid, value[i].NullFloat.Valid)
if each.NullFloat.Valid {
assert.Equal(t, each.NullFloat.Float64, value[i].NullFloat.Float64)
} else {
assert.Equal(t, 0.0, value[i].NullFloat.Float64)
}
assert.Equal(t, each.NullBool.Valid, value[i].NullBool.Valid)
if each.NullBool.Valid {
assert.Equal(t, each.NullBool.Bool, value[i].NullBool.Bool)
} else {
assert.Equal(t, false, value[i].NullBool.Bool)
}
}
})
}
func TestUnmarshalRowsSqlNullStringEmptyVsNull(t *testing.T) {
expect := []struct {
Name string
EmptyString sql.NullString
NullString sql.NullString
NormalString sql.NullString
}{
{
Name: "row1",
EmptyString: sql.NullString{
String: "",
Valid: true,
},
NullString: sql.NullString{
String: "",
Valid: false,
},
NormalString: sql.NullString{
String: "hello",
Valid: true,
},
},
{
Name: "row2",
EmptyString: sql.NullString{
String: " ",
Valid: true,
},
NullString: sql.NullString{
String: "",
Valid: false,
},
NormalString: sql.NullString{
String: "",
Valid: true,
},
},
}
var value []struct {
Name string `db:"name"`
EmptyString sql.NullString `db:"empty_string"`
NullString sql.NullString `db:"null_string"`
NormalString sql.NullString `db:"normal_string"`
}
dbtest.RunTest(t, func(db *sql.DB, mock sqlmock.Sqlmock) {
rs := sqlmock.NewRows([]string{"name", "empty_string", "null_string", "normal_string"}).
AddRow("row1", "", nil, "hello").
AddRow("row2", " ", nil, "")
mock.ExpectQuery("select (.+) from string_test").
WillReturnRows(rs)
assert.Nil(t, query(context.Background(), db, func(rows *sql.Rows) error {
return unmarshalRows(&value, rows, true)
}, "select name, empty_string, null_string, normal_string from string_test"))
assert.Equal(t, 2, len(value))
for i, each := range expect {
assert.True(t, value[i].EmptyString.Valid)
assert.Equal(t, each.EmptyString.String, value[i].EmptyString.String)
assert.False(t, value[i].NullString.Valid)
assert.Equal(t, "", value[i].NullString.String)
assert.Equal(t, each.NormalString.Valid, value[i].NormalString.Valid)
if each.NormalString.Valid {
assert.Equal(t, each.NormalString.String, value[i].NormalString.String)
}
}
})
}
func TestGetValueInterface(t *testing.T) {
t.Run("non_pointer_field", func(t *testing.T) {
type testStruct struct {
Name string
Age int
}
s := testStruct{}
v := reflect.ValueOf(&s).Elem()
nameField := v.Field(0)
result, err := getValueInterface(nameField)
assert.NoError(t, err)
assert.NotNil(t, result)
// Should return pointer to the field
ptr, ok := result.(*string)
assert.True(t, ok)
*ptr = "test"
assert.Equal(t, "test", s.Name)
})
t.Run("pointer_field_nil", func(t *testing.T) {
type testStruct struct {
NamePtr *string
AgePtr *int64
}
s := testStruct{}
v := reflect.ValueOf(&s).Elem()
// Test with nil pointer field
namePtrField := v.Field(0)
assert.True(t, namePtrField.IsNil(), "initial pointer should be nil")
result, err := getValueInterface(namePtrField)
assert.NoError(t, err)
assert.NotNil(t, result)
// Should have allocated the pointer
assert.False(t, namePtrField.IsNil(), "pointer should be allocated after getValueInterface")
// Should return pointer to pointer field
ptrPtr, ok := result.(**string)
assert.True(t, ok)
testValue := "initialized"
*ptrPtr = &testValue
assert.NotNil(t, s.NamePtr)
assert.Equal(t, "initialized", *s.NamePtr)
})
t.Run("pointer_field_already_allocated", func(t *testing.T) {
type testStruct struct {
NamePtr *string
}
initial := "existing"
s := testStruct{NamePtr: &initial}
v := reflect.ValueOf(&s).Elem()
namePtrField := v.Field(0)
assert.False(t, namePtrField.IsNil(), "pointer should not be nil initially")
result, err := getValueInterface(namePtrField)
assert.NoError(t, err)
assert.NotNil(t, result)
// Should return pointer to pointer field
ptrPtr, ok := result.(**string)
assert.True(t, ok)
// Verify it points to the existing value
assert.Equal(t, "existing", **ptrPtr)
// Modify through the returned pointer
newValue := "modified"
*ptrPtr = &newValue
assert.Equal(t, "modified", *s.NamePtr)
})
t.Run("pointer_field_zero_value", func(t *testing.T) {
type testStruct struct {
IntPtr *int
}
s := testStruct{}
v := reflect.ValueOf(&s).Elem()
intPtrField := v.Field(0)
result, err := getValueInterface(intPtrField)
assert.NoError(t, err)
// After calling getValueInterface, nil pointer should be allocated
assert.NotNil(t, s.IntPtr)
// Set zero value through returned interface
ptrPtr, ok := result.(**int)
assert.True(t, ok)
zero := 0
*ptrPtr = &zero
assert.Equal(t, 0, *s.IntPtr)
})
t.Run("not_addressable_value", func(t *testing.T) {
type testStruct struct {
Name string
}
s := testStruct{Name: "test"}
v := reflect.ValueOf(s) // Non-pointer, not addressable
nameField := v.Field(0)
result, err := getValueInterface(nameField)
assert.Error(t, err)
assert.Equal(t, ErrNotReadableValue, err)
assert.Nil(t, result)
})
t.Run("multiple_pointer_types", func(t *testing.T) {
type testStruct struct {
StringPtr *string
IntPtr *int
Int64Ptr *int64
FloatPtr *float64
BoolPtr *bool
}
s := testStruct{}
v := reflect.ValueOf(&s).Elem()
// Test each pointer type gets properly initialized
for i := 0; i < v.NumField(); i++ {
field := v.Field(i)
assert.True(t, field.IsNil(), "field %d should start as nil", i)
result, err := getValueInterface(field)
assert.NoError(t, err, "field %d should not error", i)
assert.NotNil(t, result, "field %d result should not be nil", i)
// After getValueInterface, pointer should be allocated
assert.False(t, field.IsNil(), "field %d should be allocated", i)
}
})
}
func stringPtr(s string) *string {
return &s
}
func int64Ptr(i int64) *int64 {
return &i
}
func BenchmarkIgnore(b *testing.B) {
db, mock, err := sqlmock.New()
if err != nil {
+3 -2
View File
@@ -1,12 +1,13 @@
package threading
import (
"io"
"log"
"sync"
"sync/atomic"
"testing"
"github.com/stretchr/testify/assert"
"github.com/zeromicro/go-zero/core/logx/logtest"
)
func TestRoutineGroupRun(t *testing.T) {
@@ -24,7 +25,7 @@ func TestRoutineGroupRun(t *testing.T) {
}
func TestRoutingGroupRunSafe(t *testing.T) {
logtest.Discard(t)
log.SetOutput(io.Discard)
var count int32
group := NewRoutineGroup()
+3 -2
View File
@@ -3,12 +3,13 @@ package threading
import (
"bytes"
"context"
"io"
"log"
"testing"
"github.com/stretchr/testify/assert"
"github.com/zeromicro/go-zero/core/lang"
"github.com/zeromicro/go-zero/core/logx"
"github.com/zeromicro/go-zero/core/logx/logtest"
)
func TestRoutineId(t *testing.T) {
@@ -16,7 +17,7 @@ func TestRoutineId(t *testing.T) {
}
func TestRunSafe(t *testing.T) {
logtest.Discard(t)
log.SetOutput(io.Discard)
i := 0
+25 -18
View File
@@ -7,6 +7,7 @@ import (
"os"
"sync"
"github.com/zeromicro/go-zero/core/lang"
"github.com/zeromicro/go-zero/core/logx"
"go.opentelemetry.io/otel"
"go.opentelemetry.io/otel/exporters/jaeger"
@@ -29,36 +30,42 @@ const (
)
var (
once sync.Once
tp *sdktrace.TracerProvider
shutdownOnceFn = sync.OnceFunc(func() {
if tp != nil {
_ = tp.Shutdown(context.Background())
}
})
agents = make(map[string]lang.PlaceholderType)
lock sync.Mutex
tp *sdktrace.TracerProvider
)
// StartAgent starts an opentelemetry agent.
// It uses sync.Once to ensure the agent is initialized only once,
// similar to prometheus.StartAgent and logx.SetUp.
// This prevents multiple ServiceConf.SetUp() calls from reinitializing
// the global tracer provider when running multiple servers (e.g., REST + RPC)
// in the same process.
func StartAgent(c Config) {
if c.Disabled {
return
}
once.Do(func() {
if err := startAgent(c); err != nil {
logx.Error(err)
}
})
lock.Lock()
defer lock.Unlock()
_, ok := agents[c.Endpoint]
if ok {
return
}
// if error happens, let later calls run.
if err := startAgent(c); err != nil {
return
}
agents[c.Endpoint] = lang.Placeholder
}
// StopAgent shuts down the span processors in the order they were registered.
func StopAgent() {
shutdownOnceFn()
lock.Lock()
defer lock.Unlock()
if tp != nil {
_ = tp.Shutdown(context.Background())
tp = nil
}
}
func createExporter(c Config) (sdktrace.SpanExporter, error) {
+19 -304
View File
@@ -1,13 +1,10 @@
package trace
import (
"context"
"errors"
"testing"
"github.com/stretchr/testify/assert"
"github.com/zeromicro/go-zero/core/logx"
"go.opentelemetry.io/otel"
)
func TestStartAgent(t *testing.T) {
@@ -92,305 +89,23 @@ func TestStartAgent(t *testing.T) {
StartAgent(c10)
defer StopAgent()
// With sync.Once, only the first non-disabled config (c1) takes effect.
// Subsequent calls are ignored, which is the desired behavior to prevent
// multiple servers (REST + RPC) from reinitializing the global tracer.
assert.NotNil(t, tp)
}
func TestCreateExporter_InvalidFilePath(t *testing.T) {
logx.Disable()
c := Config{
Name: "test-invalid-file",
Endpoint: "/non-existent-directory/trace.log",
Batcher: kindFile,
}
_, err := createExporter(c)
assert.Error(t, err)
assert.Contains(t, err.Error(), "file exporter endpoint error")
}
func TestCreateExporter_UnknownBatcher(t *testing.T) {
logx.Disable()
c := Config{
Name: "test-unknown",
Endpoint: "localhost:1234",
Batcher: "unknown-batcher-type",
}
_, err := createExporter(c)
assert.Error(t, err)
assert.Contains(t, err.Error(), "unknown exporter")
}
func TestCreateExporter_ValidExporters(t *testing.T) {
logx.Disable()
tests := []struct {
name string
config Config
wantErr bool
errMsg string
}{
{
name: "valid file exporter",
config: Config{
Name: "file-test",
Endpoint: "/tmp/trace-test.log",
Batcher: kindFile,
},
wantErr: false,
},
{
name: "invalid file path",
config: Config{
Name: "file-test-invalid",
Endpoint: "/invalid-path/that/does/not/exist/trace.log",
Batcher: kindFile,
},
wantErr: true,
errMsg: "file exporter endpoint error",
},
{
name: "unknown batcher",
config: Config{
Name: "unknown-test",
Endpoint: "localhost:1234",
Batcher: "invalid-batcher",
},
wantErr: true,
errMsg: "unknown exporter",
},
{
name: "jaeger http",
config: Config{
Name: "jaeger-http",
Endpoint: "http://localhost:14268/api/traces",
Batcher: kindJaeger,
},
wantErr: false,
},
{
name: "jaeger udp",
config: Config{
Name: "jaeger-udp",
Endpoint: "udp://localhost:6831",
Batcher: kindJaeger,
},
wantErr: false,
},
{
name: "zipkin",
config: Config{
Name: "zipkin",
Endpoint: "http://localhost:9411/api/v2/spans",
Batcher: kindZipkin,
},
wantErr: false,
},
{
name: "otlpgrpc",
config: Config{
Name: "otlpgrpc",
Endpoint: "localhost:4317",
Batcher: kindOtlpGrpc,
},
wantErr: false,
},
{
name: "otlpgrpc with headers",
config: Config{
Name: "otlpgrpc-headers",
Endpoint: "localhost:4317",
Batcher: kindOtlpGrpc,
OtlpHeaders: map[string]string{
"authorization": "Bearer token123",
"x-custom-key": "custom-value",
},
},
wantErr: false,
},
{
name: "otlphttp",
config: Config{
Name: "otlphttp",
Endpoint: "localhost:4318",
Batcher: kindOtlpHttp,
},
wantErr: false,
},
{
name: "otlphttp with headers",
config: Config{
Name: "otlphttp-headers",
Endpoint: "localhost:4318",
Batcher: kindOtlpHttp,
OtlpHeaders: map[string]string{
"authorization": "Bearer token456",
"x-api-key": "api-key-value",
},
},
wantErr: false,
},
{
name: "otlphttp with headers and path",
config: Config{
Name: "otlphttp-headers-path",
Endpoint: "localhost:4318",
Batcher: kindOtlpHttp,
OtlpHttpPath: "/v1/traces",
OtlpHeaders: map[string]string{
"authorization": "Bearer token789",
"x-custom-trace": "trace-id",
},
},
wantErr: false,
},
{
name: "otlphttp with secure connection",
config: Config{
Name: "otlphttp-secure",
Endpoint: "localhost:4318",
Batcher: kindOtlpHttp,
OtlpHttpSecure: true,
OtlpHeaders: map[string]string{
"authorization": "Bearer secure-token",
},
},
wantErr: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
exporter, err := createExporter(tt.config)
if tt.wantErr {
assert.Error(t, err)
if tt.errMsg != "" {
assert.Contains(t, err.Error(), tt.errMsg)
}
assert.Nil(t, exporter)
} else {
assert.NoError(t, err)
assert.NotNil(t, exporter)
// Clean up the exporter
if exporter != nil {
_ = exporter.Shutdown(context.Background())
}
}
})
}
}
func TestStopAgent(t *testing.T) {
logx.Disable()
// StopAgent should be idempotent and safe to call multiple times
assert.NotPanics(t, func() {
StopAgent()
StopAgent()
StopAgent()
})
}
func TestStartAgent_WithEndpoint(t *testing.T) {
logx.Disable()
tests := []struct {
name string
config Config
wantErr bool
}{
{
name: "empty endpoint - no exporter created",
config: Config{
Name: "test-no-endpoint",
Sampler: 1.0,
},
wantErr: false,
},
{
name: "valid endpoint with file exporter",
config: Config{
Name: "test-with-endpoint",
Endpoint: "/tmp/test-trace.log",
Batcher: kindFile,
Sampler: 1.0,
},
wantErr: false,
},
{
name: "endpoint with invalid exporter type",
config: Config{
Name: "test-invalid-batcher",
Endpoint: "localhost:1234",
Batcher: "invalid-type",
Sampler: 1.0,
},
wantErr: true,
},
{
name: "endpoint with invalid file path",
config: Config{
Name: "test-invalid-path",
Endpoint: "/non/existent/path/trace.log",
Batcher: kindFile,
Sampler: 1.0,
},
wantErr: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
// Reset tp for each test
originalTp := tp
tp = nil
defer func() {
if tp != nil {
_ = tp.Shutdown(context.Background())
}
tp = originalTp
}()
err := startAgent(tt.config)
if tt.wantErr {
assert.Error(t, err)
} else {
assert.NoError(t, err)
assert.NotNil(t, tp, "TracerProvider should be created")
}
})
}
}
func TestStartAgent_ErrorHandler(t *testing.T) {
// Setup a tracer provider to test error handler
originalTp := tp
tp = nil
defer func() {
if tp != nil {
_ = tp.Shutdown(context.Background())
}
tp = originalTp
}()
// Call startAgent to set up the error handler
config := Config{
Name: "test-error-handler",
Sampler: 1.0,
}
err := startAgent(config)
assert.NoError(t, err)
assert.NotNil(t, tp)
// Verify the error handler was set and can be called without panicking
// We test this by calling otel.Handle which will invoke the registered error handler
testErr := errors.New("test otel error")
assert.NotPanics(t, func() {
otel.Handle(testErr)
}, "Error handler should handle errors without panicking")
lock.Lock()
defer lock.Unlock()
// because remotehost cannot be resolved
assert.Equal(t, 6, len(agents))
_, ok := agents[""]
assert.True(t, ok)
_, ok = agents[endpoint1]
assert.True(t, ok)
_, ok = agents[endpoint2]
assert.False(t, ok)
_, ok = agents[endpoint5]
assert.True(t, ok)
_, ok = agents[endpoint6]
assert.False(t, ok)
_, ok = agents[endpoint71]
assert.True(t, ok)
_, ok = agents[endpoint72]
assert.False(t, ok)
}
+1 -25
View File
@@ -11,40 +11,16 @@ const (
metadataPrefix = "gateway-"
)
// OpenTelemetry trace propagation headers that need to be forwarded to gRPC metadata.
// These headers are used by the W3C Trace Context standard for distributed tracing.
var traceHeaders = map[string]bool{
"traceparent": true,
"tracestate": true,
"baggage": true,
}
// ProcessHeaders builds the headers for the gateway from HTTP headers.
// It forwards both custom metadata headers (with Grpc-Metadata- prefix)
// and OpenTelemetry trace propagation headers (traceparent, tracestate, baggage)
// to ensure distributed tracing works correctly across the gateway.
func ProcessHeaders(header http.Header) []string {
var headers []string
for k, v := range header {
// Forward OpenTelemetry trace propagation headers
// These must be lowercase per gRPC metadata conventions
if lowerKey := strings.ToLower(k); traceHeaders[lowerKey] {
for _, vv := range v {
headers = append(headers, lowerKey+":"+vv)
}
continue
}
// Forward custom metadata headers with Grpc-Metadata- prefix
if !strings.HasPrefix(k, metadataHeaderPrefix) {
continue
}
// gRPC metadata keys are case-insensitive and stored as lowercase,
// so we lowercase the key to match gRPC conventions
trimmedKey := strings.TrimPrefix(k, metadataHeaderPrefix)
key := strings.ToLower(fmt.Sprintf("%s%s", metadataPrefix, trimmedKey))
key := fmt.Sprintf("%s%s", metadataPrefix, strings.TrimPrefix(k, metadataHeaderPrefix))
for _, vv := range v {
headers = append(headers, key+":"+vv)
}
+1 -89
View File
@@ -18,93 +18,5 @@ func TestBuildHeadersWithValues(t *testing.T) {
req := httptest.NewRequest("GET", "/", http.NoBody)
req.Header.Add("grpc-metadata-a", "b")
req.Header.Add("grpc-metadata-b", "b")
assert.ElementsMatch(t, []string{"gateway-a:b", "gateway-b:b"}, ProcessHeaders(req.Header))
}
func TestProcessHeadersWithTraceContext(t *testing.T) {
req := httptest.NewRequest("GET", "/", http.NoBody)
req.Header.Set("traceparent", "00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01")
req.Header.Set("tracestate", "key1=value1,key2=value2")
req.Header.Set("baggage", "userId=alice,serverNode=DF:28")
headers := ProcessHeaders(req.Header)
assert.Len(t, headers, 3)
assert.Contains(t, headers, "traceparent:00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01")
assert.Contains(t, headers, "tracestate:key1=value1,key2=value2")
assert.Contains(t, headers, "baggage:userId=alice,serverNode=DF:28")
}
func TestProcessHeadersWithMixedHeaders(t *testing.T) {
req := httptest.NewRequest("GET", "/", http.NoBody)
req.Header.Set("traceparent", "00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01")
req.Header.Set("grpc-metadata-custom", "value1")
req.Header.Set("content-type", "application/json")
req.Header.Set("tracestate", "key1=value1")
headers := ProcessHeaders(req.Header)
// Should include trace headers and grpc-metadata headers, but not regular headers
assert.Len(t, headers, 3)
assert.Contains(t, headers, "traceparent:00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01")
assert.Contains(t, headers, "tracestate:key1=value1")
assert.Contains(t, headers, "gateway-custom:value1")
}
func TestProcessHeadersTraceparentCaseInsensitive(t *testing.T) {
tests := []struct {
name string
headerKey string
headerVal string
expectedKey string
}{
{
name: "lowercase traceparent",
headerKey: "traceparent",
headerVal: "00-trace-span-01",
expectedKey: "traceparent",
},
{
name: "uppercase Traceparent",
headerKey: "Traceparent",
headerVal: "00-trace-span-01",
expectedKey: "traceparent",
},
{
name: "mixed case TraceParent",
headerKey: "TraceParent",
headerVal: "00-trace-span-01",
expectedKey: "traceparent",
},
{
name: "lowercase tracestate",
headerKey: "tracestate",
headerVal: "key=value",
expectedKey: "tracestate",
},
{
name: "mixed case TraceState",
headerKey: "TraceState",
headerVal: "key=value",
expectedKey: "tracestate",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
req := httptest.NewRequest("GET", "/", http.NoBody)
req.Header.Set(tt.headerKey, tt.headerVal)
headers := ProcessHeaders(req.Header)
assert.Len(t, headers, 1)
assert.Contains(t, headers, tt.expectedKey+":"+tt.headerVal)
})
}
}
func TestProcessHeadersEmptyHeaders(t *testing.T) {
req := httptest.NewRequest("GET", "/", http.NoBody)
headers := ProcessHeaders(req.Header)
assert.Empty(t, headers)
assert.ElementsMatch(t, []string{"gateway-A:b", "gateway-B:b"}, ProcessHeaders(req.Header))
}
+2 -2
View File
@@ -16,12 +16,12 @@ require (
github.com/jhump/protoreflect v1.17.0
github.com/pelletier/go-toml/v2 v2.2.2
github.com/prometheus/client_golang v1.21.1
github.com/redis/go-redis/v9 v9.17.2
github.com/redis/go-redis/v9 v9.15.0
github.com/spaolacci/murmur3 v1.1.0
github.com/stretchr/testify v1.11.1
go.etcd.io/etcd/api/v3 v3.5.15
go.etcd.io/etcd/client/v3 v3.5.15
go.mongodb.org/mongo-driver/v2 v2.4.1
go.mongodb.org/mongo-driver/v2 v2.3.0
go.opentelemetry.io/otel v1.24.0
go.opentelemetry.io/otel/exporters/jaeger v1.17.0
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.24.0
+4 -4
View File
@@ -154,8 +154,8 @@ github.com/prometheus/common v0.62.0 h1:xasJaQlnWAeyHdUBeGjXmutelfJHWMRr+Fg4QszZ
github.com/prometheus/common v0.62.0/go.mod h1:vyBcEuLSvWos9B1+CyL7JZ2up+uFzXhkqml0W5zIY1I=
github.com/prometheus/procfs v0.15.1 h1:YagwOFzUgYfKKHX6Dr+sHT7km/hxC76UB0learggepc=
github.com/prometheus/procfs v0.15.1/go.mod h1:fB45yRUv8NstnjriLhBQLuOUt+WW4BsoGhij/e3PBqk=
github.com/redis/go-redis/v9 v9.17.2 h1:P2EGsA4qVIM3Pp+aPocCJ7DguDHhqrXNhVcEp4ViluI=
github.com/redis/go-redis/v9 v9.17.2/go.mod h1:u410H11HMLoB+TP67dz8rL9s6QW2j76l0//kSOd3370=
github.com/redis/go-redis/v9 v9.15.0 h1:2jdes0xJxer4h3NUZrZ4OGSntGlXp4WbXju2nOTRXto=
github.com/redis/go-redis/v9 v9.15.0/go.mod h1:huWgSWd8mW6+m0VPhJjSSQ+d6Nh1VICQ6Q5lHuCH/Iw=
github.com/rivo/uniseg v0.2.0 h1:S1pD9weZBuJdFmowNwbpi7BJ8TNftyUImj/0WQi72jY=
github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
github.com/rogpeppe/go-internal v1.10.0 h1:TMyTOH3F/DB16zRVcYyreMH6GnZZrwQVAoYjRBZyWFQ=
@@ -197,8 +197,8 @@ go.etcd.io/etcd/client/pkg/v3 v3.5.15 h1:fo0HpWz/KlHGMCC+YejpiCmyWDEuIpnTDzpJLB5
go.etcd.io/etcd/client/pkg/v3 v3.5.15/go.mod h1:mXDI4NAOwEiszrHCb0aqfAYNCrZP4e9hRca3d1YK8EU=
go.etcd.io/etcd/client/v3 v3.5.15 h1:23M0eY4Fd/inNv1ZfU3AxrbbOdW79r9V9Rl62Nm6ip4=
go.etcd.io/etcd/client/v3 v3.5.15/go.mod h1:CLSJxrYjvLtHsrPKsy7LmZEE+DK2ktfd2bN4RhBMwlU=
go.mongodb.org/mongo-driver/v2 v2.4.1 h1:hGDMngUao03OVQ6sgV5csk+RWOIkF+CuLsTPobNMGNI=
go.mongodb.org/mongo-driver/v2 v2.4.1/go.mod h1:jHeEDJHJq7tm6ZF45Issun9dbogjfnPySb1vXA7EeAI=
go.mongodb.org/mongo-driver/v2 v2.3.0 h1:sh55yOXA2vUjW1QYw/2tRlHSQViwDyPnW61AwpZ4rtU=
go.mongodb.org/mongo-driver/v2 v2.3.0/go.mod h1:jHeEDJHJq7tm6ZF45Issun9dbogjfnPySb1vXA7EeAI=
go.opentelemetry.io/otel v1.24.0 h1:0LAOdjNmQeSTzGBzduGe/rU4tZhMwL5rWgtp9Ku5Jfo=
go.opentelemetry.io/otel v1.24.0/go.mod h1:W7b9Ozg4nkF5tWI5zsXkaKKDjdVjpD4oAt9Qi/MArHo=
go.opentelemetry.io/otel/exporters/jaeger v1.17.0 h1:D7UpUy2Xc2wsi1Ras6V40q806WM07rqoCWzXu7Sqy+4=
+1 -1
View File
@@ -43,7 +43,7 @@ func AddProbe(probe Probe) {
defaultHealthManager.addProbe(probe)
}
// CreateHttpHandler creates a health http handler based on the given probe.
// CreateHttpHandler create health http handler base on given probe.
func CreateHttpHandler(healthResponse string) http.HandlerFunc {
return func(w http.ResponseWriter, _ *http.Request) {
if defaultHealthManager.IsReady() {
+71 -95
View File
@@ -17,7 +17,7 @@
<a href="https://trendshift.io/repositories/3263" target="_blank"><img src="https://trendshift.io/api/badge/repositories/3263" alt="zeromicro%2Fgo-zero | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
<a href="https://www.producthunt.com/posts/go-zero?utm_source=badge-featured&utm_medium=badge&utm_souce=badge-go&#0045;zero" target="_blank"><img src="https://api.producthunt.com/widgets/embed-image/v1/featured.svg?post_id=334030&theme=light" alt="go&#0045;zero - A&#0032;web&#0032;&#0038;&#0032;rpc&#0032;framework&#0032;written&#0032;in&#0032;Go&#0046; | Product Hunt" style="width: 250px; height: 54px;" width="250" height="54" /></a>
## go-zero 介绍
## 0. go-zero 介绍
go-zero收录于 CNCF 云原生技术全景图:[https://landscape.cncf.io/?selected=go-zero](https://landscape.cncf.io/?selected=go-zero))是一个集成了各种工程实践的 web 和 rpc 框架。通过弹性设计保障了大并发服务端的稳定性,经受了充分的实战检验。
@@ -25,50 +25,72 @@ go-zero 包含极简的 API 定义和生成工具 goctl可以根据定义的
使用 go-zero 的好处:
* 经过千万日活服务验证的稳定性
* 内建弹性保护:级联超时、限流、熔断、降载(无需配置)
* 极简 API 语法生成多端代码
* 自动参数校验和丰富的微服务工具包
* 轻松获得支撑千万日活服务的稳定性
* 内建级联超时控制、限流、自适应熔断、自适应降载等微服务治理能力,无需配置和额外代码
* 微服务治理中间件可无缝集成到其它现有框架使用
* 极简的 API 描述,一键生成各端代码
* 自动校验客户端请求参数合法性
* 大量微服务治理和并发工具包
![架构图](https://raw.githubusercontent.com/zeromicro/zero-doc/main/doc/images/architecture.png)
## go-zero 框架背景
## 1. go-zero 框架背景
18 年初,我们决定从 `Java+MongoDB` 的单体架构迁移到微服务架构,选择:
18 年初,我们决定从 `Java+MongoDB` 的单体架构迁移到微服务架构,经过仔细思考和对比,我们决定:
* **基于 Go 语言** - 高效性能、简洁语法、极致部署体验、极低资源成本
* **自研微服务框架** - 更快速的问题定位、更便捷的新特性增加
* 基于 Go 语言
* 高效的性能
* 简洁的语法
* 广泛验证的工程效率
* 极致的部署体验
* 极低的服务端资源成本
* 自研微服务框架
* 有过很多微服务框架自研经验
* 需要有更快速的问题定位能力
* 更便捷的增加新特性
## go-zero 框架设计思考
## 2. go-zero 框架设计思考
go-zero 遵循以下核心设计准则:
对于微服务框架的设计,我们期望保障微服务稳定性的同时,也要特别注重研发效率。所以设计之初,我们就有如下一些准则
* **保持简单** - 简单是第一原则
* **高可用** - 高并发、易扩展
* **弹性设计** - 面向故障编程
* **工具驱动** - 工具大于约定和文档
* **业务友好** - 封装复杂度、一事一法
* 保持简单第一原则
* 弹性设计,面向故障编程
* 工具大于约定和文档
* 高可用、高并发、易扩展
* 对业务开发友好,封装复杂度
* 约束做一件事只有一种方式
## go-zero 项目实现和特点
我们经历不到半年时间,彻底完成了从 `Java+MongoDB``Golang+MySQL` 为主的微服务体系迁移,并于 18 年 8 月底完全上线,稳定保障了业务后续迅速增长,确保了整个服务的高可用。
go-zero 集成各种工程实践,主要特点:
## 3. go-zero 项目实现和特点
* **强大工具支持** - 尽可能少的代码编写
* **极简接口** - 完全兼容 net/http
* **高性能** - 优化的速度和效率
* **弹性设计** - 内建限流、熔断、降载,自动触发、自动恢复
* **服务治理** - 内建服务发现、负载均衡、链路跟踪
* **开发工具** - API 参数自动校验、超时级联控制、自动缓存控制
go-zero 是一个集成了各种工程实践的包含 web 和 rpc 框架,有如下主要特点:
* 强大的工具支持,尽可能少的代码编写
* 极简的接口
* 完全兼容 net/http
* 支持中间件,方便扩展
* 高性能
* 面向故障编程,弹性设计
* 内建服务发现、负载均衡
* 内建限流、熔断、降载,且自动触发,自动恢复
* API 参数自动校验
* 超时级联控制
* 自动缓存控制
* 链路跟踪、统计报警等
* 高并发支撑,稳定保障了疫情期间每天的流量洪峰
如下图,我们从多个层面保障了整体服务的高可用:
![弹性设计](https://raw.githubusercontent.com/zeromicro/zero-doc/main/doc/images/resilience.jpg)
## 我们使用 go-zero 的基本架构图
## 4. 我们使用 go-zero 的基本架构图
<img width="1067" alt="image" src="https://user-images.githubusercontent.com/1918356/171880582-11a86658-41c3-466c-95e7-7b1220eecc52.png">
觉得不错的话,别忘 **star** 👏
## Installation
## 5. Installation
在项目目录下通过如下命令安装:
@@ -76,57 +98,7 @@ go-zero 集成各种工程实践,主要特点:
GO111MODULE=on GOPROXY=https://goproxy.cn/,direct go get -u github.com/zeromicro/go-zero
```
## AI 原生开发
go-zero 团队构建了完整的 AI 工具生态,让 Claude、GitHub Copilot、Cursor 生成符合 go-zero 规范的代码。
### 三大核心项目
**[ai-context](https://github.com/zeromicro/ai-context)** - AI 的工作流程指南
**[zero-skills](https://github.com/zeromicro/zero-skills)** - 模式库和示例
**[mcp-zero](https://github.com/zeromicro/mcp-zero)** - 基于 MCP 的代码生成工具
### 快速配置
#### GitHub Copilot
```bash
git submodule add https://github.com/zeromicro/ai-context.git .github/ai-context
ln -s ai-context/00-instructions.md .github/copilot-instructions.md # macOS/Linux
# Windows: mklink .github\copilot-instructions.md .github\ai-context\00-instructions.md
git submodule update --remote .github/ai-context # 更新
```
#### Cursor
```bash
git submodule add https://github.com/zeromicro/ai-context.git .cursorrules
git submodule update --remote .cursorrules # 更新
```
#### Windsurf
```bash
git submodule add https://github.com/zeromicro/ai-context.git .windsurfrules
git submodule update --remote .windsurfrules # 更新
```
#### Claude Desktop
```bash
git clone https://github.com/zeromicro/mcp-zero.git && cd mcp-zero && go build
# 配置: ~/Library/Application Support/Claude/claude_desktop_config.json
# 或: claude mcp add --transport stdio mcp-zero --env GOCTL_PATH=/path/to/goctl -- /path/to/mcp-zero
```
### 协同工作原理
AI 助手通过三个工具协同配合:
1. **ai-context** - 工作流程指导
2. **zero-skills** - 实现模式
3. **mcp-zero** - 实时代码生成
**示例**:创建新的 REST API → AI 读取 **ai-context** 了解工作流 → 调用 **mcp-zero** 生成代码 → 参考 **zero-skills** 实现模式 → 生成符合规范的代码 ✅
## Quick Start
## 6. Quick Start
0. 完整示例请查看
@@ -136,22 +108,23 @@ AI 助手通过三个工具协同配合:
1. 安装 goctl 工具
`goctl` 读作 `go control`,不要读成 `go C-T-L``goctl` 的意思是不要被代码控制,而是要去控制它。其中的 `go` 不是指 `golang`。在设计 `goctl` 之初,我就希望通过 `工具` 来解放我们的双手👈
```shell
# Go
GOPROXY=https://goproxy.cn/,direct go install github.com/zeromicro/go-zero/tools/goctl@latest
# For Mac
brew install goctl
# docker for all platforms
docker pull kevinwan/goctl
# run goctl
docker run --rm -it -v `pwd`:/app kevinwan/goctl --help
```
确保 goctl 可执行在 $PATH 环境变量里。
确保 goctl 可执行,并且在 $PATH 环境变量里。
2. 快速生成 api 服务
```shell
@@ -184,7 +157,7 @@ AI 助手通过三个工具协同配合:
* 可以在 `servicecontext.go` 里面传递依赖给 logic比如 mysql, redis 等
* 在 api 定义的 `get/post/put/delete` 等请求对应的 logic 里增加业务处理逻辑
3. 生成多语言客户端代码
3. 可以根据 api 文件生成前端需要的 Java, TypeScript, Dart, JavaScript 代码
```shell
goctl api java -api greet.api -dir greet
@@ -192,17 +165,17 @@ AI 助手通过三个工具协同配合:
...
```
## Benchmark
## 7. Benchmark
![benchmark](https://raw.githubusercontent.com/zeromicro/zero-doc/main/doc/images/benchmark.png)
[测试代码见这里](https://github.com/smallnest/go-web-framework-benchmark)
## 文档
## 8. 文档
* API 文档
[https://go-zero.dev](https://go-zero.dev)
[https://go-zero.dev/cn/](https://go-zero.dev/cn/)
* awesome 系列(更多文章见『微服务实践』公众号)
@@ -219,9 +192,9 @@ AI 助手通过三个工具协同配合:
| [goctl-android](https://github.com/zeromicro/goctl-android) | 生成 `java (android)``http client` 请求代码 |
| [goctl-go-compact](https://github.com/zeromicro/goctl-go-compact) | 合并 `api` 里同一个 `group` 里的 `handler` 到一个 `go` 文件 |
## go-zero 用户
## 9. go-zero 用户
go-zero 已被多公司用于生产部署,场景涵盖在线教育、电商、游戏、区块链等目前使用 go-zero 的公司包括但不限于:
go-zero 已被多公司用于生产部署,接入场景如在线教育、电商业务、游戏、区块链等目前为止,已使用 go-zero 的公司包括但不限于
>1. 好未来
>2. 上海晓信信息科技有限公司(晓黑板)
@@ -332,13 +305,10 @@ go-zero 已被众多公司用于生产部署,场景涵盖在线教育、电商
>107. 深圳市聚货通信息科技有限公司
>108. 浙江银盾云科技有限公司
>109. 南京造世网络科技有限公司
>110. 温州飞儿云信息技术有限公司
>111. 统信软件
>112. 深圳坐标软件集团有限公司
如果贵公司也已使用 go-zero欢迎在 [登记地址](https://github.com/zeromicro/go-zero/issues/602) 登记,仅仅为了推广,不做其它用途。
## CNCF 云原生技术全景图
## 10. CNCF 云原生技术全景图
<p float="left">
<img src="https://raw.githubusercontent.com/zeromicro/zero-doc/main/doc/images/cncf-logo.svg" width="200"/>&nbsp;&nbsp;&nbsp;
@@ -347,13 +317,13 @@ go-zero 已被众多公司用于生产部署,场景涵盖在线教育、电商
go-zero 收录在 [CNCF Cloud Native 云原生技术全景图](https://landscape.cncf.io/?selected=go-zero)。
## 微信公众号
## 11. 微信公众号
`go-zero` 相关文章和视频都会在 `微服务实践` 公众号整理呈现,欢迎扫码关注 👏
<img src="https://raw.githubusercontent.com/zeromicro/zero-doc/main/doc/images/zeromicro.jpg" alt="wechat" width="600" />
## 微信交流群
## 12. 微信交流群
如果文档中未能覆盖的任何疑问,欢迎您在群里提出,我们会尽快答复。
@@ -363,4 +333,10 @@ go-zero 收录在 [CNCF Cloud Native 云原生技术全景图](https://landscape
加群之前有劳点一下 ***star***,一个小小的 ***star*** 是作者们回答海量问题的动力!🤝
<img src="https://raw.githubusercontent.com/zeromicro/zero-doc/main/doc/images/wechat.jpg" alt="wechat" width="300" />
<img src="https://raw.githubusercontent.com/zeromicro/zero-doc/main/doc/images/wechat.jpg" alt="wechat" width="300" />
## 13. 知识星球
官方团队运营的知识星球
<img src="https://raw.githubusercontent.com/zeromicro/zero-doc/main/doc/images/zsxq.jpg" alt="知识星球" width="300" />
+63 -89
View File
@@ -42,39 +42,61 @@ go-zero contains simple API description syntax and code generation tool called `
## Backgrounds of go-zero
In early 2018, we transitioned from a Java+MongoDB monolithic architecture to microservices, choosing:
In early 2018, we embarked on a transformative journey to redesign our system, transitioning from a monolithic architecture built with Java and MongoDB to a microservices architecture. After careful research and comparison, we made a deliberate choice to:
* **Golang** - High performance, simple syntax, excellent deployment experience, and low resource consumption
* **Self-designed microservice framework** - Better problem isolation, easier feature extension, and faster issue resolution
* Go Beyond with Golang
* Great performance
* Simple syntax
* Proven engineering efficiency
* Extreme deployment experience
* Less server resource consumption
* Self-Design Our Microservice Architecture
* Microservice architecture facilitates the creation of scalable, flexible, and maintainable software systems with independent, reusable components.
* Easy to locate the problems within microservices.
* Easy to extend the features by adding or modifying specific microservices without impacting the entire system.
## Design considerations on go-zero
go-zero follows these core design principles:
By designing the microservice architecture, we expected to ensure stability, as well as productivity. And from just the beginning, we have the following design principles:
* **Simplicity** - Keep it simple, first principle
* **High availability** - Stable under high concurrency
* **Resilience** - Failure-oriented programming with adaptive protection
* **Developer friendly** - Encapsulate complexity, one way to do one thing
* **Easy to extend** - Flexible architecture for growth
* Keep it simple
* High availability
* Stable on high concurrency
* Easy to extend
* Resilience design, failure-oriented programming
* Try best to be friendly to the business logic development, encapsulate the complexity
* One thing, one way
After almost half a year, we finished the transfer from a monolithic system to microservice system and deployed on August 2018. The new system guaranteed business growth and system stability.
## The implementation and features of go-zero
go-zero integrates engineering best practices:
go-zero is a web and rpc framework that integrates lots of engineering practices. The features are mainly listed below:
* **Code generation** - Powerful tools to minimize boilerplate
* **Simple API** - Clean interfaces, fully compatible with net/http
* **High performance** - Optimized for speed and efficiency
* **Resilience** - Built-in circuit breaker, rate limiting, load shedding, timeout control
* **Service mesh** - Service discovery, load balancing, call tracing
* **Developer tools** - Auto parameter validation, cache management, metrics and monitoring
* Powerful tool included, less code to write
* Simple interfaces
* Fully compatible with net/http
* Middlewares are supported, easy to extend
* High performance
* Failure-oriented programming, resilience design
* Builtin service discovery, load balancing
* Builtin concurrency control, adaptive circuit breaker, adaptive load shedding, auto-trigger, auto recover
* Auto validation of API request parameters
* Chained timeout control
* Auto management of data caching
* Call tracing, metrics, and monitoring
* High concurrency protected
As below, go-zero protects the system with a couple of layers and mechanisms:
![Resilience](https://raw.githubusercontent.com/zeromicro/zero-doc/main/doc/images/resilience-en.png)
## Architecture with go-zero
## The simplified architecture that we use with go-zero
<img width="1067" alt="image" src="https://user-images.githubusercontent.com/1918356/171880372-5010d846-e8b1-4942-8fe2-e2bbb584f762.png">
## Installation
## Installation
Run the following command under your project:
@@ -82,59 +104,9 @@ Run the following command under your project:
go get -u github.com/zeromicro/go-zero
```
## AI-Native Development
## Quick Start
The go-zero team provides AI tooling for Claude, GitHub Copilot, Cursor to generate framework-compliant code.
### Three Core Projects
**[ai-context](https://github.com/zeromicro/ai-context)** - Workflow guide for AI assistants
**[zero-skills](https://github.com/zeromicro/zero-skills)** - Pattern library with examples
**[mcp-zero](https://github.com/zeromicro/mcp-zero)** - Code generation tools via Model Context Protocol
### Quick Setup
#### GitHub Copilot
```bash
git submodule add https://github.com/zeromicro/ai-context.git .github/ai-context
ln -s ai-context/00-instructions.md .github/copilot-instructions.md # macOS/Linux
# Windows: mklink .github\copilot-instructions.md .github\ai-context\00-instructions.md
git submodule update --remote .github/ai-context # Update
```
#### Cursor
```bash
git submodule add https://github.com/zeromicro/ai-context.git .cursorrules
git submodule update --remote .cursorrules # Update
```
#### Windsurf
```bash
git submodule add https://github.com/zeromicro/ai-context.git .windsurfrules
git submodule update --remote .windsurfrules # Update
```
#### Claude Desktop
```bash
git clone https://github.com/zeromicro/mcp-zero.git && cd mcp-zero && go build
# Configure: ~/Library/Application Support/Claude/claude_desktop_config.json
# Or: claude mcp add --transport stdio mcp-zero --env GOCTL_PATH=/path/to/goctl -- /path/to/mcp-zero
```
### How It Works
AI assistants use these tools together:
1. **ai-context** - workflow guidance
2. **zero-skills** - implementation patterns
3. **mcp-zero** - real-time code generation
**Example**: Creating a REST API → AI reads **ai-context** for workflow → calls **mcp-zero** to generate code → references **zero-skills** for patterns → produces production-ready code ✅
## Quick Start
1. Full examples:
1. Full examples can be checked out from below:
[Rapid development of microservice systems](https://github.com/zeromicro/zero-doc/blob/main/doc/shorturl-en.md)
@@ -142,22 +114,24 @@ AI assistants use these tools together:
2. Install goctl
`goctl`can be read as `go control`. `goctl` means not to be controlled by code, instead, we control it. The inside `go` is not `golang`. At the very beginning, I was expecting it to help us improve productivity, and make our lives easier.
```shell
# for Go
go install github.com/zeromicro/go-zero/tools/goctl@latest
# For Mac
brew install goctl
# docker for all platforms
docker pull kevinwan/goctl
# run goctl
docker run --rm -it -v `pwd`:/app kevinwan/goctl --help
```
Ensure goctl is executable and in your $PATH.
3. Create the API file (greet.api):
make sure goctl is executable and in your $PATH.
3. Create the API file, like greet.api, you can install the plugin of goctl in vs code, api syntax is supported.
```go
type (
@@ -176,19 +150,19 @@ AI assistants use these tools together:
}
```
Generate .api template:
the .api files also can be generated by goctl, like below:
```shell
goctl api -o greet.api
```
4. Generate Go server code
4. Generate the go server-side code
```shell
goctl api go -api greet.api -dir greet
```
Generated structure:
the generated files look like:
```Plain Text
├── greet
@@ -210,7 +184,7 @@ AI assistants use these tools together:
└── greet.api // api description file
```
Run the service:
the generated code can be run directly:
```shell
cd greet
@@ -218,15 +192,15 @@ AI assistants use these tools together:
go run greet.go -f etc/greet-api.yaml
```
Default port: 8888 (configurable in etc/greet-api.yaml)
by default, its listening on port 8888, while it can be changed in the configuration file.
Test with curl:
you can check it by curl:
```shell
curl -i http://localhost:8888/greet/from/you
```
Response:
the response looks like below:
```http
HTTP/1.1 200 OK
@@ -234,12 +208,12 @@ AI assistants use these tools together:
Content-Length: 0
```
5. Write business logic
5. Write the business logic code
* Pass dependencies (mysql, redis, etc.) via servicecontext.go
* Add logic code in the logic package per .api definition
* the dependencies can be passed into the logic within servicecontext.go, like mysql, redis, etc.
* add the logic code in a logic package according to .api file
6. Generate client code for multiple languages
6. Generate code like Java, TypeScript, Dart, JavaScript, etc. just from the api file
```shell
goctl api java -api greet.api -dir greet
@@ -260,11 +234,11 @@ AI assistants use these tools together:
* [Rapid development of microservice systems - multiple RPCs](https://github.com/zeromicro/zero-doc/blob/main/docs/zero/bookstore-en.md)
* [Examples](https://github.com/zeromicro/zero-examples)
## Chat group
## Chat group
Join the chat via https://discord.gg/4JQvC5A4Fe
## Cloud Native Landscape
## Cloud Native Landscape
<p float="left">
<img src="https://raw.githubusercontent.com/zeromicro/zero-doc/main/doc/images/cncf-logo.svg" width="200"/>&nbsp;&nbsp;&nbsp;
+1 -1
View File
@@ -13,7 +13,7 @@ import (
)
type (
// TraceOption defines the method to customize a traceOptions.
// TraceOption defines the method to customize an traceOptions.
TraceOption func(options *traceOptions)
// traceOptions is TraceHandler options.
+1 -1
View File
@@ -19,7 +19,7 @@ const (
var (
// ErrInvalidMethod is an error that indicates not a valid http method.
ErrInvalidMethod = errors.New("not a valid http method")
// ErrInvalidPath is an error that indicates path does not start with /.
// ErrInvalidPath is an error that indicates path is not start with /.
ErrInvalidPath = errors.New("path must begin with '/'")
)
+4 -4
View File
@@ -32,7 +32,7 @@ import '../vars/vars.dart';
/// Send GET request.
///
/// ok: the function that will be called on success.
/// failthe function that will be called on failure.
/// failthe fuction that will be called on failure.
/// eventuallythe function that will be called regardless of success or failure.
Future apiGet(String path,
{Map<String, String> header,
@@ -47,7 +47,7 @@ Future apiGet(String path,
///
/// data: the data to post, it will be marshaled to json automatically.
/// ok: the function that will be called on success.
/// failthe function that will be called on failure.
/// failthe fuction that will be called on failure.
/// eventuallythe function that will be called regardless of success or failure.
Future apiPost(String path, dynamic data,
{Map<String, String> header,
@@ -132,7 +132,7 @@ Future _apiRequest(String method, String path, dynamic data,
/// data: any request class that will be converted to json automatically
/// ok: is called when request succeeds
/// fail: is called when request fails
/// eventually: is always called after the nearby function returns
/// eventually: is always called until the nearby functions returns
Future apiPost(String path, dynamic data,
{Map<String, String>? header,
Function(Map<String, dynamic>)? ok,
@@ -146,7 +146,7 @@ Future _apiRequest(String method, String path, dynamic data,
///
/// ok: is called when request succeeds
/// fail: is called when request fails
/// eventually: is always called after the nearby function returns
/// eventually: is always called until the nearby functions returns
Future apiGet(String path,
{Map<String, String>? header,
Function(Map<String, dynamic>)? ok,
+1 -1
View File
@@ -36,7 +36,7 @@ func DocCommand(_ *cobra.Command, _ []string) error {
}
if !pathx.FileExists(dir) {
return fmt.Errorf("dir %s not exist", dir)
return fmt.Errorf("dir %s not exsit", dir)
}
dir, err := filepath.Abs(dir)
+18 -12
View File
@@ -55,29 +55,32 @@ service Test {
// Verify SSE-specific patterns in handler
// Handler should call: err := l.Sse(&req, client)
assert.Contains(t, handlerStr, "err := l.Sse(&req, client)",
assert.Contains(t, handlerStr, "err := l.Sse(&req, client)",
"Handler should call logic with client channel parameter")
// Handler should NOT have the regular pattern: resp, err := l.Sse(&req)
assert.NotContains(t, handlerStr, "resp, err := l.Sse(&req)",
assert.NotContains(t, handlerStr, "resp, err := l.Sse(&req)",
"Handler should not use regular pattern with resp return")
// Handler should use threading.GoSafeCtx
assert.Contains(t, handlerStr, "threading.GoSafeCtx",
assert.Contains(t, handlerStr, "threading.GoSafeCtx",
"Handler should use threading.GoSafeCtx for SSE")
// Handler should create client channel
assert.Contains(t, handlerStr, "client := make(chan",
assert.Contains(t, handlerStr, "client := make(chan",
"Handler should create client channel")
// Verify SSE-specific patterns in logic
// Logic should have signature: Sse(req *types.SseReq, client chan<- *types.SseResp) error
assert.Contains(t, logicStr, "func (l *SseLogic) Sse(req *types.SseReq, client chan<- *types.SseResp) error",
assert.Contains(t, logicStr, "func (l *SseLogic) Sse(req *types.SseReq, client chan<- *types.SseResp) error",
"Logic should have SSE signature with client channel parameter")
// Logic should NOT have regular signature: Sse(req *types.SseReq) (resp *types.SseResp, err error)
assert.NotContains(t, logicStr, "(resp *types.SseResp, err error)",
assert.NotContains(t, logicStr, "(resp *types.SseResp, err error)",
"Logic should not have regular signature with resp return")
t.Logf("Handler content:\n%s", handlerStr)
t.Logf("Logic content:\n%s", logicStr)
}
func TestNonSSEGeneration(t *testing.T) {
@@ -123,20 +126,20 @@ service Test {
// Verify regular (non-SSE) patterns in handler
// Handler should call: resp, err := l.Sse(&req)
assert.Contains(t, handlerStr, "resp, err := l.Sse(&req)",
assert.Contains(t, handlerStr, "resp, err := l.Sse(&req)",
"Handler should use regular pattern with resp return")
// Handler should NOT have SSE pattern: err := l.Sse(&req, client)
assert.NotContains(t, handlerStr, "err := l.Sse(&req, client)",
assert.NotContains(t, handlerStr, "err := l.Sse(&req, client)",
"Handler should not use SSE pattern")
// Handler should NOT use threading.GoSafeCtx
assert.NotContains(t, handlerStr, "threading.GoSafeCtx",
assert.NotContains(t, handlerStr, "threading.GoSafeCtx",
"Handler should not use threading.GoSafeCtx for regular routes")
// Verify regular (non-SSE) patterns in logic
// Logic should have signature: Sse(req *types.SseReq) (resp *types.SseResp, err error)
assert.Contains(t, logicStr, "(resp *types.SseResp, err error)",
assert.Contains(t, logicStr, "(resp *types.SseResp, err error)",
"Logic should have regular signature with resp return")
// Logic should NOT have SSE signature with client parameter
@@ -148,6 +151,9 @@ service Test {
break
}
}
assert.False(t, hasSSESignature,
assert.False(t, hasSSESignature,
"Logic should not have SSE signature with client channel parameter")
t.Logf("Handler content:\n%s", handlerStr)
t.Logf("Logic content:\n%s", logicStr)
}
+14 -10
View File
@@ -31,16 +31,20 @@ func TestServerIntegration(t *testing.T) {
Port: 0, // Use random available port
},
}
server := rest.MustNewServer(c.RestConf)
defer server.Stop()
ctx := svc.NewServiceContext(c)
handler.RegisterHandlers(server, ctx)
// Create serverless wrapper for testing
serverless, err := rest.NewServerless(server)
require.NoError(t, err)
// Start server in background
go func() {
server.Start()
}()
// Wait for server to start
time.Sleep(100 * time.Millisecond)
tests := []struct {
name string
@@ -52,7 +56,7 @@ func TestServerIntegration(t *testing.T) {
}{
{
name: "health check",
method: http.MethodGet,
method: "GET",
path: "/health",
expectedStatus: http.StatusNotFound, // Adjust based on actual routes
setup: func() {},
@@ -68,7 +72,7 @@ func TestServerIntegration(t *testing.T) {
},
{{end}}{{end}}{
name: "not found route",
method: http.MethodGet,
method: "GET",
path: "/nonexistent",
expectedStatus: http.StatusNotFound,
setup: func() {},
@@ -83,10 +87,10 @@ func TestServerIntegration(t *testing.T) {
require.NoError(t, err)
rr := httptest.NewRecorder()
serverless.Serve(rr, req)
server.ServeHTTP(rr, req)
assert.Equal(t, tt.expectedStatus, rr.Code)
// TODO: Add response body assertions
t.Logf("Response: %s", rr.Body.String())
})
@@ -96,13 +100,13 @@ func TestServerIntegration(t *testing.T) {
func TestServerLifecycle(t *testing.T) {
c := config.Config{
RestConf: rest.RestConf{
Host: "127.0.0.1",
Host: "127.0.0.1",
Port: 0,
},
}
server := rest.MustNewServer(c.RestConf)
// Test server can start and stop without errors
ctx := svc.NewServiceContext(c)
handler.RegisterHandlers(server, ctx)
-17
View File
@@ -1,17 +0,0 @@
type Request {
Name string `path:"name,options=you|me"`
}
type Response {
Message string `json:"message"`
}
@server(
jwt: Auth
jwtTransition: Trans
middleware: TokenValidate
)
service A-api {
@handler GreetHandler
get /greet/from/:name(Request) returns (Response)
}
+1 -1
View File
@@ -28,7 +28,7 @@ type (
syntax *SyntaxExpr
}
// ParserOption defines a function with argument Parser
// ParserOption defines an function with argument Parser
ParserOption func(p *Parser)
)
+2 -2
View File
@@ -268,7 +268,7 @@ func (v *ApiVisitor) VisitReplybody(ctx *api.ReplybodyContext) any {
v.panic(lit.Expr(), fmt.Sprintf("expecting 'ID', but found golang keyword '%s'", lit.Expr().Text()))
}
default:
v.panic(dt.Expr(), fmt.Sprintf("unsupported %s", dt.Expr().Text()))
v.panic(dt.Expr(), fmt.Sprintf("unsupport %s", dt.Expr().Text()))
}
case *Literal:
lit := dataType.Literal.Text()
@@ -276,7 +276,7 @@ func (v *ApiVisitor) VisitReplybody(ctx *api.ReplybodyContext) any {
v.panic(dataType.Literal, fmt.Sprintf("expecting 'ID', but found golang keyword '%s'", lit))
}
default:
v.panic(dt.Expr(), fmt.Sprintf("unsupported %s", dt.Expr().Text()))
v.panic(dt.Expr(), fmt.Sprintf("unsupport %s", dt.Expr().Text()))
}
return &Body{
+1 -1
View File
@@ -190,7 +190,7 @@ func (v *ApiVisitor) VisitTypeBlockStruct(ctx *api.TypeBlockStructContext) any {
structExpr := v.newExprWithToken(ctx.GetStructToken())
structTokenText := ctx.GetStructToken().GetText()
if structTokenText != "struct" {
v.panic(structExpr, fmt.Sprintf("expecting 'struct', found input '%s'", structTokenText))
v.panic(structExpr, fmt.Sprintf("expecting 'struct', found imput '%s'", structTokenText))
}
if api.IsGolangKeyWord(structTokenText, "struct") {
+4 -4
View File
@@ -18,7 +18,7 @@ type parser struct {
}
// Parse parses the api file.
// Deprecated: use tools/goctl/pkg/parser/api/parser/parser.go:18 instead,
// Depreacted: use tools/goctl/pkg/parser/api/parser/parser.go:18 instead,
// it will be removed in the future.
func Parse(filename string) (*spec.ApiSpec, error) {
if env.UseExperimental() {
@@ -63,14 +63,14 @@ func parseContent(content string, skipCheckTypeDeclaration bool, filename ...str
return apiSpec, nil
}
// Deprecated: use tools/goctl/pkg/parser/api/parser/parser.go:18 instead,
// Depreacted: use tools/goctl/pkg/parser/api/parser/parser.go:18 instead,
// it will be removed in the future.
// ParseContent parses the api content
func ParseContent(content string, filename ...string) (*spec.ApiSpec, error) {
return parseContent(content, false, filename...)
}
// Deprecated: use tools/goctl/pkg/parser/api/parser/parser.go:18 instead,
// Depreacted: use tools/goctl/pkg/parser/api/parser/parser.go:18 instead,
// it will be removed in the future.
// ParseContentWithParserSkipCheckTypeDeclaration parses the api content with skip check type declaration
func ParseContentWithParserSkipCheckTypeDeclaration(content string, filename ...string) (*spec.ApiSpec, error) {
@@ -227,7 +227,7 @@ func (p parser) astTypeToSpec(in ast.DataType) spec.Type {
return spec.PointerType{RawName: v.PointerExpr.Text(), Type: spec.DefineStruct{RawName: raw}}
}
panic(fmt.Sprintf("unsupported type %+v", in))
panic(fmt.Sprintf("unspported type %+v", in))
}
func (p parser) stringExprs(docs []ast.Expr) []string {
+2 -7
View File
@@ -24,15 +24,10 @@ func getFirstUsableString(def ...string) string {
}
for _, val := range def {
// Try to unquote if it's a quoted string
if str, err := strconv.Unquote(val); err == nil && len(str) != 0 {
str, err := strconv.Unquote(val)
if err == nil && len(str) != 0 {
return str
}
// Otherwise, use the value as-is if it's not empty
if len(val) != 0 {
return val
}
}
return ""
-105
View File
@@ -89,108 +89,3 @@ func Test_getListFromInfoOrDefault(t *testing.T) {
assert.Equal(t, []string{"query"}, getListFromInfoOrDefault(unquotedProperties, "tags", []string{"default"}))
assert.Equal(t, []string{"default"}, getListFromInfoOrDefault(unquotedProperties, "empty", []string{"default"}))
}
func Test_getFirstUsableString(t *testing.T) {
t.Run("empty input", func(t *testing.T) {
result := getFirstUsableString()
assert.Equal(t, "", result, "should return empty string for no arguments")
})
t.Run("single plain string", func(t *testing.T) {
result := getFirstUsableString("Check server health status.")
assert.Equal(t, "Check server health status.", result)
})
t.Run("single quoted string", func(t *testing.T) {
// This is how Go would represent a quoted string literal
result := getFirstUsableString(`"Check server health status."`)
assert.Equal(t, "Check server health status.", result, "should unquote quoted strings")
})
t.Run("multiple plain strings", func(t *testing.T) {
result := getFirstUsableString("", "second", "third")
assert.Equal(t, "second", result, "should return first non-empty string")
})
t.Run("handler name fallback", func(t *testing.T) {
// Simulates the real use case: @doc text, handler name
result := getFirstUsableString("", "HealthCheck")
assert.Equal(t, "HealthCheck", result, "should fallback to handler name")
})
t.Run("doc text over handler name", func(t *testing.T) {
// Simulates the real use case with @doc text
result := getFirstUsableString("Check server health status.", "HealthCheck")
assert.Equal(t, "Check server health status.", result, "should use doc text over handler name")
})
t.Run("empty strings before valid", func(t *testing.T) {
result := getFirstUsableString("", "", "valid")
assert.Equal(t, "valid", result, "should skip empty strings")
})
t.Run("all empty strings", func(t *testing.T) {
result := getFirstUsableString("", "", "")
assert.Equal(t, "", result, "should return empty if all are empty")
})
t.Run("quoted then plain", func(t *testing.T) {
result := getFirstUsableString(`"quoted"`, "plain")
assert.Equal(t, "quoted", result, "should unquote first quoted string")
})
t.Run("plain then quoted", func(t *testing.T) {
result := getFirstUsableString("plain", `"quoted"`)
assert.Equal(t, "plain", result, "should use first plain string")
})
t.Run("invalid quoted string", func(t *testing.T) {
// String that looks quoted but isn't valid Go syntax
result := getFirstUsableString(`"incomplete`, "fallback")
assert.Equal(t, `"incomplete`, result, "should use as-is if unquote fails but not empty")
})
t.Run("whitespace only", func(t *testing.T) {
result := getFirstUsableString(" ", "fallback")
assert.Equal(t, " ", result, "should not trim whitespace, return as-is")
})
t.Run("real world API doc scenario", func(t *testing.T) {
// This is the actual bug scenario from issue #5229
atDocText := "Check server health status."
handlerName := "HealthCheck"
result := getFirstUsableString(atDocText, handlerName)
assert.Equal(t, "Check server health status.", result,
"should use @doc text for API summary")
})
t.Run("real world with empty doc", func(t *testing.T) {
// When @doc is empty, should fall back to handler name
atDocText := ""
handlerName := "HealthCheck"
result := getFirstUsableString(atDocText, handlerName)
assert.Equal(t, "HealthCheck", result,
"should fallback to handler name when @doc is empty")
})
t.Run("complex summary with special characters", func(t *testing.T) {
result := getFirstUsableString("Get user by ID: /users/{id}")
assert.Equal(t, "Get user by ID: /users/{id}", result,
"should handle special characters in plain strings")
})
t.Run("multiline string", func(t *testing.T) {
result := getFirstUsableString("Line 1\nLine 2")
assert.Equal(t, "Line 1\nLine 2", result,
"should handle multiline strings")
})
t.Run("unicode characters", func(t *testing.T) {
result := getFirstUsableString("健康检查", "HealthCheck")
assert.Equal(t, "健康检查", result,
"should handle unicode characters")
})
}
+11 -22
View File
@@ -8,37 +8,28 @@ import (
apiSpec "github.com/zeromicro/go-zero/tools/goctl/api/spec"
)
func isRequestBodyJson(ctx Context, method string, tp apiSpec.Type) (string, bool) {
// Support HTTP methods that commonly use request bodies with JSON
// POST, PUT, PATCH are standard methods with bodies
// DELETE can also have a body (though less common)
method = strings.ToUpper(method)
if method != http.MethodPost && method != http.MethodPut &&
method != http.MethodPatch && method != http.MethodDelete {
func isPostJson(ctx Context, method string, tp apiSpec.Type) (string, bool) {
if !strings.EqualFold(method, http.MethodPost) {
return "", false
}
structType, ok := tp.(apiSpec.DefineStruct)
if !ok {
return "", false
}
var hasJsonField bool
var isPostJson bool
rangeMemberAndDo(ctx, structType, func(tag *apiSpec.Tags, required bool, member apiSpec.Member) {
jsonTag, _ := tag.Get(tagJson)
if !hasJsonField {
hasJsonField = jsonTag != nil
if !isPostJson {
isPostJson = jsonTag != nil
}
})
return structType.RawName, hasJsonField
return structType.RawName, isPostJson
}
func parametersFromType(ctx Context, method string, tp apiSpec.Type) []spec.Parameter {
if tp == nil {
return []spec.Parameter{}
}
structType, ok := tp.(apiSpec.DefineStruct)
if !ok {
return []spec.Parameter{}
@@ -52,13 +43,15 @@ func parametersFromType(ctx Context, method string, tp apiSpec.Type) []spec.Para
rangeMemberAndDo(ctx, structType, func(tag *apiSpec.Tags, required bool, member apiSpec.Member) {
headerTag, _ := tag.Get(tagHeader)
hasHeader := headerTag != nil
pathParameterTag, _ := tag.Get(tagPath)
hasPathParameter := pathParameterTag != nil
formTag, _ := tag.Get(tagForm)
hasForm := formTag != nil
jsonTag, _ := tag.Get(tagJson)
hasJson := jsonTag != nil
if hasHeader {
minimum, maximum, exclusiveMinimum, exclusiveMaximum := rangeValueFromOptions(headerTag.Options)
resp = append(resp, spec.Parameter{
@@ -82,7 +75,6 @@ func parametersFromType(ctx Context, method string, tp apiSpec.Type) []spec.Para
},
})
}
if hasPathParameter {
minimum, maximum, exclusiveMinimum, exclusiveMaximum := rangeValueFromOptions(pathParameterTag.Options)
resp = append(resp, spec.Parameter{
@@ -106,7 +98,6 @@ func parametersFromType(ctx Context, method string, tp apiSpec.Type) []spec.Para
},
})
}
if hasForm {
minimum, maximum, exclusiveMinimum, exclusiveMaximum := rangeValueFromOptions(formTag.Options)
if strings.EqualFold(method, http.MethodGet) {
@@ -154,8 +145,8 @@ func parametersFromType(ctx Context, method string, tp apiSpec.Type) []spec.Para
},
})
}
}
}
if hasJson {
minimum, maximum, exclusiveMinimum, exclusiveMaximum := rangeValueFromOptions(jsonTag.Options)
if required {
@@ -188,10 +179,9 @@ func parametersFromType(ctx Context, method string, tp apiSpec.Type) []spec.Para
properties[jsonTag.Name] = schema
}
})
if len(properties) > 0 {
if ctx.UseDefinitions {
structName, ok := isRequestBodyJson(ctx, method, tp)
structName, ok := isPostJson(ctx, method, tp)
if ok {
resp = append(resp, spec.Parameter{
ParamProps: spec.ParamProps{
@@ -223,6 +213,5 @@ func parametersFromType(ctx Context, method string, tp apiSpec.Type) []spec.Para
})
}
}
return resp
}
+3 -14
View File
@@ -8,7 +8,7 @@ import (
apiSpec "github.com/zeromicro/go-zero/tools/goctl/api/spec"
)
func TestIsRequestBodyJson(t *testing.T) {
func TestIsPostJson(t *testing.T) {
tests := []struct {
name string
method string
@@ -18,18 +18,13 @@ func TestIsRequestBodyJson(t *testing.T) {
{"POST with JSON", http.MethodPost, true, true},
{"POST without JSON", http.MethodPost, false, false},
{"GET with JSON", http.MethodGet, true, false},
{"PUT with JSON", http.MethodPut, true, true},
{"PUT without JSON", http.MethodPut, false, false},
{"PATCH with JSON", http.MethodPatch, true, true},
{"PATCH without JSON", http.MethodPatch, false, false},
{"DELETE with JSON", http.MethodDelete, true, true},
{"DELETE without JSON", http.MethodDelete, false, false},
{"PUT with JSON", http.MethodPut, true, false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
testStruct := createTestStruct("TestStruct", tt.hasJson)
_, result := isRequestBodyJson(testingContext(t), tt.method, testStruct)
_, result := isPostJson(testingContext(t), tt.method, testStruct)
assert.Equal(t, tt.expected, result)
})
}
@@ -46,12 +41,6 @@ func TestParametersFromType(t *testing.T) {
}{
{"POST JSON with definitions", http.MethodPost, true, true, 1, true},
{"POST JSON without definitions", http.MethodPost, false, true, 1, true},
{"PUT JSON with definitions", http.MethodPut, true, true, 1, true},
{"PUT JSON without definitions", http.MethodPut, false, true, 1, true},
{"PATCH JSON with definitions", http.MethodPatch, true, true, 1, true},
{"PATCH JSON without definitions", http.MethodPatch, false, true, 1, true},
{"DELETE JSON with definitions", http.MethodDelete, true, true, 1, true},
{"DELETE JSON without definitions", http.MethodDelete, false, true, 1, true},
{"GET with form", http.MethodGet, false, false, 1, false},
{"POST with form", http.MethodPost, false, false, 1, false},
}
+1 -1
View File
@@ -11,7 +11,7 @@
## swagger
1. [bug fix] remove example generation when request body are `query`, `path` and `header`
- it not supported in api spec 2.0
- it will generate example when request body is json format.
- it's will generate example when request body is json format.
2. [features] swagger generation supported definitions
- supported response definitions
- supported json request body definitions
+1 -2
View File
@@ -73,7 +73,6 @@ func dockerCommand(_ *cobra.Command, _ []string) (err error) {
base := varStringBase
port := varIntPort
etcDir := filepath.Join(filepath.Dir(goFile), etcDir)
if _, err := os.Stat(etcDir); os.IsNotExist(err) {
return generateDockerfile(goFile, base, port, version, timezone)
}
@@ -171,7 +170,7 @@ func generateDockerfile(goFile, base string, port int, version, timezone string,
t := template.Must(template.New("dockerfile").Parse(text))
return t.Execute(out, Docker{
Chinese: env.InChina(),
GoMainFrom: path.Join(projPath, filepath.Base(goFile)),
GoMainFrom: path.Join(projPath, goFile),
GoRelPath: projPath,
GoFile: goFile,
ExeFile: exeName,
-376
View File
@@ -1,376 +0,0 @@
package docker
import (
"os"
"path/filepath"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestDockerCommand_EtcDirResolution(t *testing.T) {
// Create a temporary project structure
tempDir := t.TempDir()
// Create project structure: project/service/api/
serviceDir := filepath.Join(tempDir, "service", "api")
etcDir := filepath.Join(serviceDir, "etc")
require.NoError(t, os.MkdirAll(etcDir, 0755))
// Create a Go file
goFile := filepath.Join(serviceDir, "api.go")
require.NoError(t, os.WriteFile(goFile, []byte("package main\n\nfunc main() {}"), 0644))
// Create a config file
configFile := filepath.Join(etcDir, "config.yaml")
require.NoError(t, os.WriteFile(configFile, []byte("Name: test\n"), 0644))
// Create go.mod at the root
goModFile := filepath.Join(tempDir, "go.mod")
require.NoError(t, os.WriteFile(goModFile, []byte("module test\n\ngo 1.21\n"), 0644))
// Test: etc directory should be found relative to Go file, not CWD
t.Run("etc directory resolved relative to go file", func(t *testing.T) {
// Save and restore original working directory
originalWd, err := os.Getwd()
require.NoError(t, err)
defer func() {
require.NoError(t, os.Chdir(originalWd))
}()
// Change to temp directory (not service/api directory)
require.NoError(t, os.Chdir(tempDir))
// The relative path from tempDir to the go file
relGoFile := filepath.Join("service", "api", "api.go")
// Test the etc directory resolution logic
resolvedEtcDir := filepath.Join(filepath.Dir(relGoFile), "etc")
// Verify the resolved path exists
_, err = os.Stat(resolvedEtcDir)
assert.NoError(t, err, "etc directory should be found at service/api/etc")
// Verify it's the correct path (use EvalSymlinks to handle /private on macOS)
absResolvedEtc, err := filepath.Abs(resolvedEtcDir)
require.NoError(t, err)
absResolvedEtc, err = filepath.EvalSymlinks(absResolvedEtc)
require.NoError(t, err)
expectedEtc, err := filepath.EvalSymlinks(etcDir)
require.NoError(t, err)
assert.Equal(t, expectedEtc, absResolvedEtc)
})
t.Run("etc directory with empty goFile", func(t *testing.T) {
// When goFile is empty, should default to "./etc"
goFile := ""
resolvedEtcDir := filepath.Join(filepath.Dir(goFile), "etc")
// Should resolve to just "etc"
assert.Equal(t, "etc", resolvedEtcDir)
})
t.Run("etc directory with absolute path", func(t *testing.T) {
// When goFile is absolute path
absGoFile := filepath.Join(tempDir, "service", "api", "api.go")
resolvedEtcDir := filepath.Join(filepath.Dir(absGoFile), "etc")
// Should resolve correctly
_, err := os.Stat(resolvedEtcDir)
assert.NoError(t, err)
})
}
func TestGenerateDockerfile_GoMainFromPath(t *testing.T) {
tests := []struct {
name string
goFile string
projPath string
expectedPath string
}{
{
name: "relative path with subdirectory",
goFile: "service/api/api.go",
projPath: "service/api",
expectedPath: "service/api/api.go",
},
{
name: "simple filename",
goFile: "main.go",
projPath: ".",
expectedPath: "main.go",
},
{
name: "nested service path",
goFile: "internal/service/user/user.go",
projPath: "internal/service/user",
expectedPath: "internal/service/user/user.go",
},
{
name: "deep nested path",
goFile: "cmd/api/internal/handler/handler.go",
projPath: "cmd/api/internal/handler",
expectedPath: "cmd/api/internal/handler/handler.go",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
// Simulate the fix: using filepath.Base instead of full path
goMainFrom := filepath.Join(tt.projPath, filepath.Base(tt.goFile))
assert.Equal(t, tt.expectedPath, goMainFrom,
"GoMainFrom should not duplicate path segments")
// Verify the old buggy behavior would have been wrong
if tt.goFile != filepath.Base(tt.goFile) {
buggyPath := filepath.Join(tt.projPath, tt.goFile)
assert.NotEqual(t, tt.expectedPath, buggyPath,
"Old implementation would have created incorrect path")
}
})
}
}
func TestGenerateDockerfile_PathJoinBehavior(t *testing.T) {
t.Run("demonstrates the bug and fix", func(t *testing.T) {
projPath := "service/api"
goFile := "service/api/api.go"
// OLD (buggy) behavior: path duplication
buggyPath := filepath.Join(projPath, goFile)
assert.Equal(t, "service/api/service/api/api.go", buggyPath,
"Bug: path segments are duplicated")
// NEW (fixed) behavior: correct path
fixedPath := filepath.Join(projPath, filepath.Base(goFile))
assert.Equal(t, "service/api/api.go", fixedPath,
"Fix: using filepath.Base prevents duplication")
})
}
func TestFindConfig(t *testing.T) {
tempDir := t.TempDir()
etcDir := filepath.Join(tempDir, "etc")
require.NoError(t, os.MkdirAll(etcDir, 0755))
t.Run("finds config matching go file name", func(t *testing.T) {
// Create config files
require.NoError(t, os.WriteFile(
filepath.Join(etcDir, "api.yaml"), []byte("test"), 0644))
require.NoError(t, os.WriteFile(
filepath.Join(etcDir, "other.yaml"), []byte("test"), 0644))
cfg, err := findConfig("api.go", etcDir)
assert.NoError(t, err)
assert.Equal(t, "api.yaml", cfg)
})
t.Run("returns first config when no match", func(t *testing.T) {
etcDir2 := filepath.Join(tempDir, "etc2")
require.NoError(t, os.MkdirAll(etcDir2, 0755))
require.NoError(t, os.WriteFile(
filepath.Join(etcDir2, "config.yaml"), []byte("test"), 0644))
cfg, err := findConfig("main.go", etcDir2)
assert.NoError(t, err)
assert.Equal(t, "config.yaml", cfg)
})
t.Run("returns error when no yaml files", func(t *testing.T) {
emptyDir := filepath.Join(tempDir, "empty")
require.NoError(t, os.MkdirAll(emptyDir, 0755))
_, err := findConfig("api.go", emptyDir)
assert.Error(t, err)
assert.Contains(t, err.Error(), "no yaml file")
})
t.Run("handles path in go file name", func(t *testing.T) {
// Test with service/api/api.go - should extract just "api"
cfg, err := findConfig("service/api/api.go", etcDir)
assert.NoError(t, err)
assert.Equal(t, "api.yaml", cfg)
})
}
func TestGetFilePath(t *testing.T) {
// Create a temporary directory with go.mod
tempDir := t.TempDir()
require.NoError(t, os.WriteFile(
filepath.Join(tempDir, "go.mod"),
[]byte("module testproject\n\ngo 1.21\n"),
0644,
))
// Create subdirectories
serviceDir := filepath.Join(tempDir, "service", "api")
require.NoError(t, os.MkdirAll(serviceDir, 0755))
originalWd, err := os.Getwd()
require.NoError(t, err)
defer func() {
require.NoError(t, os.Chdir(originalWd))
}()
t.Run("returns relative path from go.mod", func(t *testing.T) {
require.NoError(t, os.Chdir(tempDir))
path, err := getFilePath("service/api")
assert.NoError(t, err)
assert.Equal(t, "service/api", path)
})
t.Run("handles current directory", func(t *testing.T) {
require.NoError(t, os.Chdir(tempDir))
path, err := getFilePath(".")
assert.NoError(t, err)
// Current directory returns empty string when at go.mod root
assert.True(t, path == "." || path == "")
})
}
// Integration test to verify the complete fix
func TestDockerCommandIntegration(t *testing.T) {
// Create a complete project structure
tempDir := t.TempDir()
// Setup: project/service/api/
serviceDir := filepath.Join(tempDir, "service", "api")
etcDir := filepath.Join(serviceDir, "etc")
require.NoError(t, os.MkdirAll(etcDir, 0755))
// Create files
goFile := filepath.Join(serviceDir, "api.go")
require.NoError(t, os.WriteFile(goFile, []byte("package main\n\nfunc main() {}"), 0644))
configFile := filepath.Join(etcDir, "api.yaml")
require.NoError(t, os.WriteFile(configFile, []byte("Name: test-api\n"), 0644))
goModFile := filepath.Join(tempDir, "go.mod")
require.NoError(t, os.WriteFile(goModFile, []byte("module testproject\n\ngo 1.21\n"), 0644))
goSumFile := filepath.Join(tempDir, "go.sum")
require.NoError(t, os.WriteFile(goSumFile, []byte(""), 0644))
originalWd, err := os.Getwd()
require.NoError(t, err)
defer func() {
require.NoError(t, os.Chdir(originalWd))
}()
t.Run("etc directory detected from different working directory", func(t *testing.T) {
// Change to project root (not service/api)
require.NoError(t, os.Chdir(tempDir))
// Relative path to Go file
relGoFile := filepath.Join("service", "api", "api.go")
// Apply the fix: resolve etc directory relative to go file
resolvedEtcDir := filepath.Join(filepath.Dir(relGoFile), "etc")
// Verify etc directory is found
stat, err := os.Stat(resolvedEtcDir)
assert.NoError(t, err)
assert.True(t, stat.IsDir())
// Verify config can be found
cfg, err := findConfig(relGoFile, resolvedEtcDir)
assert.NoError(t, err)
assert.Equal(t, "api.yaml", cfg)
})
t.Run("GoMainFrom path is correct", func(t *testing.T) {
require.NoError(t, os.Chdir(tempDir))
goFileRel := filepath.Join("service", "api", "api.go")
// Simulate getFilePath return value
projPath := "service/api"
// Apply the fix: use filepath.Base
goMainFrom := filepath.Join(projPath, filepath.Base(goFileRel))
assert.Equal(t, "service/api/api.go", goMainFrom)
// Verify no path duplication
assert.NotContains(t, goMainFrom, "service/api/service/api")
})
}
// Test that specifically validates the bug described in PR #4343
func TestPR4343_BugFixes(t *testing.T) {
t.Run("Bug 1: etc directory check uses correct base path", func(t *testing.T) {
// Setup: Create a project structure where etc is NOT in CWD but IS relative to Go file
tempDir := t.TempDir()
serviceDir := filepath.Join(tempDir, "service", "api")
etcDir := filepath.Join(serviceDir, "etc")
require.NoError(t, os.MkdirAll(etcDir, 0755))
// Create a config file
require.NoError(t, os.WriteFile(
filepath.Join(etcDir, "config.yaml"),
[]byte("Name: test\n"),
0644,
))
originalWd, err := os.Getwd()
require.NoError(t, err)
defer func() {
require.NoError(t, os.Chdir(originalWd))
}()
// Change to project root (CWD = tempDir)
require.NoError(t, os.Chdir(tempDir))
goFile := filepath.Join("service", "api", "api.go")
// OLD (buggy) behavior: checks for "etc" in CWD
_, errOld := os.Stat("etc")
assert.Error(t, errOld, "Bug: etc not found in CWD")
// NEW (fixed) behavior: checks for "etc" relative to go file
etcDirResolved := filepath.Join(filepath.Dir(goFile), "etc")
stat, errNew := os.Stat(etcDirResolved)
assert.NoError(t, errNew, "Fix: etc found relative to go file")
assert.True(t, stat.IsDir())
// Verify config is accessible
cfg, err := findConfig(goFile, etcDirResolved)
assert.NoError(t, err)
assert.Equal(t, "config.yaml", cfg)
})
t.Run("Bug 2: GoMainFrom path not duplicated", func(t *testing.T) {
// Test case from PR description
projPath := "service/api"
goFile := "service/api/api.go"
// OLD (buggy) behavior: duplicates path
buggyPath := filepath.Join(projPath, goFile)
assert.Equal(t, "service/api/service/api/api.go", buggyPath,
"Bug: path duplication occurs with old implementation")
// NEW (fixed) behavior: correct path using filepath.Base
fixedPath := filepath.Join(projPath, filepath.Base(goFile))
assert.Equal(t, "service/api/api.go", fixedPath,
"Fix: using filepath.Base() prevents path duplication")
// Verify the fix works for various scenarios
testCases := []struct {
projPath string
goFile string
expected string
}{
{"service/api", "service/api/api.go", "service/api/api.go"},
{"cmd/server", "cmd/server/main.go", "cmd/server/main.go"},
{"internal/handler", "internal/handler/handler.go", "internal/handler/handler.go"},
{".", "main.go", "main.go"},
}
for _, tc := range testCases {
result := filepath.Join(tc.projPath, filepath.Base(tc.goFile))
assert.Equal(t, tc.expected, result,
"Fix should work for projPath=%s, goFile=%s", tc.projPath, tc.goFile)
}
})
}
+3 -3
View File
@@ -10,13 +10,13 @@ require (
github.com/go-sql-driver/mysql v1.9.0
github.com/gookit/color v1.6.0
github.com/iancoleman/strcase v0.3.0
github.com/spf13/cobra v1.10.2
github.com/spf13/cobra v1.10.1
github.com/spf13/pflag v1.0.10
github.com/stretchr/testify v1.11.1
github.com/withfig/autocomplete-tools/integrations/cobra v1.2.1
github.com/zeromicro/antlr v0.0.1
github.com/zeromicro/ddl-parser v1.0.5
github.com/zeromicro/go-zero v1.9.3
github.com/zeromicro/go-zero v1.9.1
golang.org/x/text v0.22.0
google.golang.org/grpc v1.65.0
google.golang.org/protobuf v1.36.5
@@ -72,7 +72,7 @@ require (
github.com/prometheus/client_model v0.6.1 // indirect
github.com/prometheus/common v0.62.0 // indirect
github.com/prometheus/procfs v0.15.1 // indirect
github.com/redis/go-redis/v9 v9.16.0 // indirect
github.com/redis/go-redis/v9 v9.15.0 // indirect
github.com/spaolacci/murmur3 v1.1.0 // indirect
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e // indirect
github.com/yuin/gopher-lua v1.1.1 // indirect
+6 -7
View File
@@ -148,15 +148,15 @@ github.com/prometheus/common v0.62.0 h1:xasJaQlnWAeyHdUBeGjXmutelfJHWMRr+Fg4QszZ
github.com/prometheus/common v0.62.0/go.mod h1:vyBcEuLSvWos9B1+CyL7JZ2up+uFzXhkqml0W5zIY1I=
github.com/prometheus/procfs v0.15.1 h1:YagwOFzUgYfKKHX6Dr+sHT7km/hxC76UB0learggepc=
github.com/prometheus/procfs v0.15.1/go.mod h1:fB45yRUv8NstnjriLhBQLuOUt+WW4BsoGhij/e3PBqk=
github.com/redis/go-redis/v9 v9.16.0 h1:OotgqgLSRCmzfqChbQyG1PHC3tLNR89DG4jdOERSEP4=
github.com/redis/go-redis/v9 v9.16.0/go.mod h1:u410H11HMLoB+TP67dz8rL9s6QW2j76l0//kSOd3370=
github.com/redis/go-redis/v9 v9.15.0 h1:2jdes0xJxer4h3NUZrZ4OGSntGlXp4WbXju2nOTRXto=
github.com/redis/go-redis/v9 v9.15.0/go.mod h1:huWgSWd8mW6+m0VPhJjSSQ+d6Nh1VICQ6Q5lHuCH/Iw=
github.com/rogpeppe/go-internal v1.11.0 h1:cWPaGQEPrBb5/AsnsZesgZZ9yb1OQ+GOISoDNXVBh4M=
github.com/rogpeppe/go-internal v1.11.0/go.mod h1:ddIwULY96R17DhadqLgMfk9H9tvdUzkipdSkR5nkCZA=
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/spaolacci/murmur3 v1.1.0 h1:7c1g84S4BPRrfL5Xrdp6fOJ206sU9y293DDHaoy0bLI=
github.com/spaolacci/murmur3 v1.1.0/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA=
github.com/spf13/cobra v1.10.2 h1:DMTTonx5m65Ic0GOoRY2c16WCbHxOOw6xxezuLaBpcU=
github.com/spf13/cobra v1.10.2/go.mod h1:7C1pvHqHw5A4vrJfjNwvOdzYu0Gml16OCs2GRiTUUS4=
github.com/spf13/cobra v1.10.1 h1:lJeBwCfmrnXthfAupyUTzJ/J4Nc1RsHC/mSRU2dll/s=
github.com/spf13/cobra v1.10.1/go.mod h1:7SmJGaTHFVBY0jW4NXGluQoLvhqFQM+6XSKD+P4XaB0=
github.com/spf13/pflag v1.0.9/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/spf13/pflag v1.0.10 h1:4EBh2KAYBwaONj6b2Ye1GiHfwjqyROoF4RwYO+vPwFk=
github.com/spf13/pflag v1.0.10/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
@@ -185,8 +185,8 @@ github.com/zeromicro/antlr v0.0.1 h1:CQpIn/dc0pUjgGQ81y98s/NGOm2Hfru2NNio2I9mQgk
github.com/zeromicro/antlr v0.0.1/go.mod h1:nfpjEwFR6Q4xGDJMcZnCL9tEfQRgszMwu3rDz2Z+p5M=
github.com/zeromicro/ddl-parser v1.0.5 h1:LaVqHdzMTjasua1yYpIYaksxKqRzFrEukj2Wi2EbWaQ=
github.com/zeromicro/ddl-parser v1.0.5/go.mod h1:ISU/8NuPyEpl9pa17Py9TBPetMjtsiHrb9f5XGiYbo8=
github.com/zeromicro/go-zero v1.9.3 h1:dJ568uUoRJY0RUxo4aH4htSglbEUF60WiM1MZVkTK9A=
github.com/zeromicro/go-zero v1.9.3/go.mod h1:JBAtfXQvErk+V7pxzcySR0mW6m2I4KPhNQZGASltDRQ=
github.com/zeromicro/go-zero v1.9.1 h1:GZCl4jun/ZgZHnSvX3SSNDHf+tEGmEQ8x2Z23xjHa9g=
github.com/zeromicro/go-zero v1.9.1/go.mod h1:bHOl7Xr7EV/iHZWEqsUNJwFc/9WgAMrPpPagYvOaMtY=
go.etcd.io/etcd/api/v3 v3.5.15 h1:3KpLJir1ZEBrYuV2v+Twaa/e2MdDCEZ/70H+lzEiwsk=
go.etcd.io/etcd/api/v3 v3.5.15/go.mod h1:N9EhGzXq58WuMllgH9ZvnEr7SI9pS0k0+DHZezGp7jM=
go.etcd.io/etcd/client/pkg/v3 v3.5.15 h1:fo0HpWz/KlHGMCC+YejpiCmyWDEuIpnTDzpJLB5fWlA=
@@ -227,7 +227,6 @@ go.uber.org/multierr v1.9.0 h1:7fIwc/ZtS0q++VgcfqFDxSBZVv/Xo49/SYnDFupUwlI=
go.uber.org/multierr v1.9.0/go.mod h1:X2jQV1h+kxSjClGpnseKVIxpmcjrj7MNnI0bnlfKTVQ=
go.uber.org/zap v1.24.0 h1:FiJd5l1UOLj0wCgbSE0rwwXHzEdAZS6hiiSnxJN/D60=
go.uber.org/zap v1.24.0/go.mod h1:2kMP+WWQ8aoFoedH3T2sq6iJ2yDWpHbP0f6MQbS9Gkg=
go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
+1 -1
View File
@@ -6,7 +6,7 @@ import (
)
// BuildVersion is the version of goctl.
const BuildVersion = "1.9.2"
const BuildVersion = "1.9.1"
var tag = map[string]int{"pre-alpha": 0, "alpha": 1, "pre-beta": 2, "beta": 3, "released": 4, "": 5}
+2 -2
View File
@@ -99,12 +99,12 @@ func (conn *MockConn) RawDB() (*sql.DB, error) {
return conn.db, nil
}
// Transact is the implementation of sqlx.SqlConn, nothing to do
// Transact is the implemention of sqlx.SqlConn, nothing to do
func (conn *MockConn) Transact(func(session sqlx.Session) error) error {
return nil
}
// TransactCtx is the implementation of sqlx.SqlConn, nothing to do
// TransactCtx is the implemention of sqlx.SqlConn, nothing to do
func (conn *MockConn) TransactCtx(ctx context.Context, fn func(context.Context, sqlx.Session) error) error {
return nil
}
+12
View File
@@ -2,6 +2,7 @@ package util
import (
"slices"
"strconv"
"strings"
"github.com/zeromicro/go-zero/tools/goctl/util/console"
@@ -129,3 +130,14 @@ func FieldsAndTrimSpace(s string, f func(r rune) bool) []string {
}
return resp
}
//Deprecated: This function implementation is incomplete and does not properly handle exceptional input cases.
//We strongly recommend using the standard library's strconv.Unquote function instead,
//which provides robust error handling and comprehensive support for various input formats.
func Unquote(s string) string {
ns, err := strconv.Unquote(s)
if err != nil {
return ""
}
return ns
}
+35 -18
View File
@@ -76,40 +76,40 @@ func TestEscapeGoKeyword(t *testing.T) {
func TestFieldsAndTrimSpace(t *testing.T) {
testCases := []struct {
name string
input string
name string
input string
delimiter func(r rune) bool
expected []string
expected []string
}{
{
name: "Comma-separated values",
input: "a, b, c",
name: "Comma-separated values",
input: "a, b, c",
delimiter: func(r rune) bool { return r == ',' },
expected: []string{"a", " b", " c"},
expected: []string{"a", " b", " c"},
},
{
name: "Space-separated values",
input: "a b c",
name: "Space-separated values",
input: "a b c",
delimiter: unicode.IsSpace,
expected: []string{"a", "b", "c"},
expected: []string{"a", "b", "c"},
},
{
name: "Mixed whitespace",
input: "a\tb\nc",
name: "Mixed whitespace",
input: "a\tb\nc",
delimiter: unicode.IsSpace,
expected: []string{"a", "b", "c"},
expected: []string{"a", "b", "c"},
},
{
name: "Empty input",
input: "",
name: "Empty input",
input: "",
delimiter: unicode.IsSpace,
expected: []string(nil),
expected: []string(nil),
},
{
name: "Trailing and leading spaces",
input: " a , b , c ",
name: "Trailing and leading spaces",
input: " a , b , c ",
delimiter: func(r rune) bool { return r == ',' },
expected: []string{" a ", " b ", " c "},
expected: []string{" a ", " b ", " c "},
},
}
@@ -120,3 +120,20 @@ func TestFieldsAndTrimSpace(t *testing.T) {
})
}
}
func TestUnquote(t *testing.T) {
testCases := []struct {
input string
expected string
}{
{input: `"hello"`, expected: `hello`},
{input: "`world`", expected: `world`},
{input: `"foo'bar"`, expected: `foo'bar`},
{input: "", expected: ""},
}
for _, tc := range testCases {
result := Unquote(tc.input)
assert.Equal(t, tc.expected, result)
}
}
-25
View File
@@ -1,16 +1,12 @@
package zrpc
import (
"context"
"fmt"
"time"
"github.com/zeromicro/go-zero/core/conf"
"github.com/zeromicro/go-zero/core/logx"
"github.com/zeromicro/go-zero/zrpc/internal"
"github.com/zeromicro/go-zero/zrpc/internal/auth"
"github.com/zeromicro/go-zero/zrpc/internal/balancer/consistenthash"
"github.com/zeromicro/go-zero/zrpc/internal/balancer/p2c"
"github.com/zeromicro/go-zero/zrpc/internal/clientinterceptors"
"google.golang.org/grpc"
"google.golang.org/grpc/keepalive"
@@ -21,9 +17,6 @@ var (
WithDialOption = internal.WithDialOption
// WithNonBlock sets the dialing to be nonblock.
WithNonBlock = internal.WithNonBlock
// WithBlock sets the dialing to be blocking.
// Deprecated: blocking dials are not recommended by gRPC.
WithBlock = internal.WithBlock
// WithStreamClientInterceptor is an alias of internal.WithStreamClientInterceptor.
WithStreamClientInterceptor = internal.WithStreamClientInterceptor
// WithTimeout is an alias of internal.WithTimeout.
@@ -64,8 +57,6 @@ func NewClient(c RpcClientConf, options ...ClientOption) (Client, error) {
}
if c.NonBlock {
opts = append(opts, WithNonBlock())
} else {
opts = append(opts, WithBlock())
}
if c.Timeout > 0 {
opts = append(opts, WithTimeout(time.Duration(c.Timeout)*time.Millisecond))
@@ -76,9 +67,6 @@ func NewClient(c RpcClientConf, options ...ClientOption) (Client, error) {
})))
}
svcCfg := makeLBServiceConfig(c.BalancerName)
opts = append(opts, WithDialOption(grpc.WithDefaultServiceConfig(svcCfg)))
opts = append(opts, options...)
target, err := c.BuildTarget()
@@ -123,20 +111,7 @@ func SetClientSlowThreshold(threshold time.Duration) {
clientinterceptors.SetSlowThreshold(threshold)
}
// SetHashKey sets the hash key into context.
func SetHashKey(ctx context.Context, key string) context.Context {
return consistenthash.SetHashKey(ctx, key)
}
// WithCallTimeout return a call option with given timeout to make a method call.
func WithCallTimeout(timeout time.Duration) grpc.CallOption {
return clientinterceptors.WithCallTimeout(timeout)
}
func makeLBServiceConfig(balancerName string) string {
if len(balancerName) == 0 {
balancerName = p2c.Name
}
return fmt.Sprintf(`{"loadBalancingPolicy":"%s"}`, balancerName)
}
-41
View File
@@ -12,8 +12,6 @@ import (
"github.com/zeromicro/go-zero/core/discov"
"github.com/zeromicro/go-zero/core/logx"
"github.com/zeromicro/go-zero/internal/mock"
"github.com/zeromicro/go-zero/zrpc/internal/balancer/consistenthash"
"github.com/zeromicro/go-zero/zrpc/internal/balancer/p2c"
"google.golang.org/grpc"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/credentials/insecure"
@@ -247,42 +245,3 @@ func TestNewClientWithTarget(t *testing.T) {
assert.NotNil(t, err)
}
func TestMakeLBServiceConfig(t *testing.T) {
tests := []struct {
name string
input string
expected string
}{
{
name: "empty name uses default p2c",
input: "",
expected: fmt.Sprintf(`{"loadBalancingPolicy":"%s"}`, p2c.Name),
},
{
name: "custom balancer name",
input: "consistent_hash",
expected: `{"loadBalancingPolicy":"consistent_hash"}`,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := makeLBServiceConfig(tt.input)
if got != tt.expected {
t.Errorf("expected %q, got %q", tt.expected, got)
}
})
}
}
func TestSetHashKey(t *testing.T) {
ctx := context.Background()
key := "abc123"
ctx = SetHashKey(ctx, key)
got := consistenthash.GetHashKey(ctx)
assert.Equal(t, key, got)
assert.Empty(t, consistenthash.GetHashKey(context.Background()))
}
+1 -2
View File
@@ -27,11 +27,10 @@ type (
Target string `json:",optional"`
App string `json:",optional"`
Token string `json:",optional"`
NonBlock bool `json:",default=true"`
NonBlock bool `json:",optional"`
Timeout int64 `json:",default=2000"`
KeepaliveTime time.Duration `json:",optional"`
Middlewares ClientMiddlewaresConf
BalancerName string `json:",default=p2c_ewma"`
}
// A RpcServerConf is a rpc server config.
-9
View File
@@ -4,11 +4,9 @@ import (
"testing"
"github.com/stretchr/testify/assert"
zconf "github.com/zeromicro/go-zero/core/conf"
"github.com/zeromicro/go-zero/core/discov"
"github.com/zeromicro/go-zero/core/service"
"github.com/zeromicro/go-zero/core/stores/redis"
"github.com/zeromicro/go-zero/zrpc/internal/balancer/p2c"
)
func TestRpcClientConf(t *testing.T) {
@@ -41,13 +39,6 @@ func TestRpcClientConf(t *testing.T) {
_, err := conf.BuildTarget()
assert.Error(t, err)
})
t.Run("default balancer name", func(t *testing.T) {
var conf RpcClientConf
err := zconf.FillDefault(&conf)
assert.NoError(t, err)
assert.Equal(t, p2c.Name, conf.BalancerName)
})
}
func TestRpcServerConf(t *testing.T) {
@@ -1,97 +0,0 @@
package consistenthash
import (
"context"
"github.com/zeromicro/go-zero/core/hash"
"google.golang.org/grpc/balancer"
"google.golang.org/grpc/balancer/base"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
)
const (
Name = "consistent_hash"
defaultReplicaCount = 100
)
var emptyPickResult balancer.PickResult
func init() {
balancer.Register(newBuilder())
}
type (
// hashKey is the key type for consistent hash in context.
hashKey struct{}
// pickerBuilder is a builder for picker.
pickerBuilder struct{}
// picker is a picker that uses consistent hash to pick a sub connection.
picker struct {
hashRing *hash.ConsistentHash
conns map[string]balancer.SubConn
}
)
func (b *pickerBuilder) Build(info base.PickerBuildInfo) balancer.Picker {
readySCs := info.ReadySCs
if len(readySCs) == 0 {
return base.NewErrPicker(balancer.ErrNoSubConnAvailable)
}
conns := make(map[string]balancer.SubConn, len(readySCs))
hashRing := hash.NewCustomConsistentHash(defaultReplicaCount, hash.Hash)
for conn, connInfo := range readySCs {
addr := connInfo.Address.Addr
conns[addr] = conn
hashRing.Add(addr)
}
return &picker{
hashRing: hashRing,
conns: conns,
}
}
func newBuilder() balancer.Builder {
return base.NewBalancerBuilder(Name, &pickerBuilder{}, base.Config{HealthCheck: true})
}
func (p *picker) Pick(info balancer.PickInfo) (balancer.PickResult, error) {
hashKey := GetHashKey(info.Ctx)
if len(hashKey) == 0 {
return emptyPickResult, status.Error(codes.InvalidArgument,
"[consistent_hash] missing hash key in context")
}
if addrAny, ok := p.hashRing.Get(hashKey); ok {
addr, ok := addrAny.(string)
if !ok {
return emptyPickResult, status.Error(codes.Internal,
"[consistent_hash] invalid addr type in consistent hash")
}
subConn, ok := p.conns[addr]
if !ok {
return emptyPickResult, status.Errorf(codes.Internal,
"[consistent_hash] no subConn for addr: %s", addr)
}
return balancer.PickResult{SubConn: subConn}, nil
}
return emptyPickResult, status.Errorf(codes.Unavailable,
"[consistent_hash] no matching conn for hashKey: %s", hashKey)
}
// SetHashKey sets the hash key into context.
func SetHashKey(ctx context.Context, key string) context.Context {
return context.WithValue(ctx, hashKey{}, key)
}
// GetHashKey gets the hash key from context.
func GetHashKey(ctx context.Context) string {
v, _ := ctx.Value(hashKey{}).(string)
return v
}
@@ -1,175 +0,0 @@
package consistenthash
import (
"context"
"testing"
"github.com/stretchr/testify/assert"
"github.com/zeromicro/go-zero/core/hash"
"google.golang.org/grpc/balancer"
"google.golang.org/grpc/balancer/base"
"google.golang.org/grpc/resolver"
)
type fakeSubConn struct{ id int }
func (f *fakeSubConn) Connect() {}
func (f *fakeSubConn) UpdateAddresses(_ []resolver.Address) {}
func (f *fakeSubConn) Shutdown() {}
func (f *fakeSubConn) GetOrBuildProducer(b balancer.ProducerBuilder) (balancer.Producer, func()) {
return nil, func() {}
}
func TestPickerBuilder_EmptyReadySCs(t *testing.T) {
b := &pickerBuilder{}
p := b.Build(base.PickerBuildInfo{ReadySCs: map[balancer.SubConn]base.SubConnInfo{}})
_, err := p.Pick(balancer.PickInfo{})
assert.Equal(t, balancer.ErrNoSubConnAvailable, err)
}
func TestPickerBuilder_BuildAndRing(t *testing.T) {
subConn1 := &fakeSubConn{id: 1}
subConn2 := &fakeSubConn{id: 2}
addr1 := "127.0.0.1:8080"
addr2 := "127.0.0.1:8081"
b := &pickerBuilder{}
info := base.PickerBuildInfo{
ReadySCs: map[balancer.SubConn]base.SubConnInfo{
subConn1: {Address: resolver.Address{Addr: addr1}},
subConn2: {Address: resolver.Address{Addr: addr2}},
},
}
p := b.Build(info).(*picker)
assert.NotNil(t, p.hashRing)
assert.Len(t, p.conns, 2)
}
func TestPicker_HashConsistency(t *testing.T) {
subConn1 := &fakeSubConn{id: 1}
subConn2 := &fakeSubConn{id: 2}
pb := &pickerBuilder{}
info := base.PickerBuildInfo{
ReadySCs: map[balancer.SubConn]base.SubConnInfo{
subConn1: {Address: resolver.Address{Addr: "127.0.0.1:8080"}},
subConn2: {Address: resolver.Address{Addr: "127.0.0.1:8081"}},
},
}
p := pb.Build(info).(*picker)
ctx := SetHashKey(context.Background(), "user_123")
res1, err := p.Pick(balancer.PickInfo{Ctx: ctx})
assert.NoError(t, err)
assert.NotNil(t, res1.SubConn)
// Multiple requests with the same key remain consistent
for i := 0; i < 5; i++ {
resN, err := p.Pick(balancer.PickInfo{Ctx: ctx})
assert.NoError(t, err)
assert.Equal(t, res1.SubConn, resN.SubConn)
}
}
func TestPicker_MissingKey(t *testing.T) {
subConn := &fakeSubConn{id: 1}
pb := &pickerBuilder{}
info := base.PickerBuildInfo{
ReadySCs: map[balancer.SubConn]base.SubConnInfo{
subConn: {Address: resolver.Address{Addr: "127.0.0.1:8080"}},
},
}
p := pb.Build(info).(*picker)
// No hash key in context
_, err := p.Pick(balancer.PickInfo{Ctx: context.Background()})
assert.Error(t, err)
assert.Contains(t, err.Error(), "[consistent_hash] missing hash key in context")
}
func TestPicker_NoMatchingConn(t *testing.T) {
emptyRing := newCustomRingForTest()
p := &picker{
hashRing: emptyRing,
conns: map[string]balancer.SubConn{},
}
_, err := p.Pick(balancer.PickInfo{Ctx: SetHashKey(context.Background(), "someone")})
assert.Error(t, err)
assert.Contains(t, err.Error(), "[consistent_hash] no matching conn for hashKey: someone")
}
func TestPicker_InvalidAddrType(t *testing.T) {
ring := newCustomRingForTest()
ring.Add(12345)
subConn := &fakeSubConn{id: 1}
p := &picker{
hashRing: ring,
conns: map[string]balancer.SubConn{
"12345": subConn,
},
}
_, err := p.Pick(balancer.PickInfo{Ctx: SetHashKey(context.Background(), "anykey")})
assert.Error(t, err)
assert.Contains(t, err.Error(), "[consistent_hash] invalid addr type in consistent hash")
}
func TestPicker_NoSubConnForAddr(t *testing.T) {
ring := newCustomRingForTest()
ring.Add("ghost:9999")
exist := &fakeSubConn{id: 1}
p := &picker{
hashRing: ring,
conns: map[string]balancer.SubConn{
"real:8080": exist,
},
}
_, err := p.Pick(balancer.PickInfo{Ctx: SetHashKey(context.Background(), "anykey")})
assert.Error(t, err)
assert.Contains(t, err.Error(), "[consistent_hash] no subConn for addr: ghost:9999")
}
func TestSetAndGetHashKey(t *testing.T) {
ctx := context.Background()
key := "abc123"
ctx = SetHashKey(ctx, key)
got := GetHashKey(ctx)
assert.Equal(t, key, got)
assert.Empty(t, GetHashKey(context.Background()))
}
func BenchmarkPicker_HashConsistency(b *testing.B) {
subConn1 := &fakeSubConn{id: 1}
subConn2 := &fakeSubConn{id: 2}
pb := &pickerBuilder{}
info := base.PickerBuildInfo{
ReadySCs: map[balancer.SubConn]base.SubConnInfo{
subConn1: {Address: resolver.Address{Addr: "127.0.0.1:8080"}},
subConn2: {Address: resolver.Address{Addr: "127.0.0.1:8081"}},
},
}
p := pb.Build(info).(*picker)
ctx := SetHashKey(context.Background(), "hot_user_123")
b.ResetTimer()
for i := 0; i < b.N; i++ {
res, err := p.Pick(balancer.PickInfo{Ctx: ctx})
if err != nil || res.SubConn == nil {
b.Fatalf("unexpected result: res=%v err=%v", res.SubConn, err)
}
}
}
func newCustomRingForTest() *hash.ConsistentHash {
return hash.NewCustomConsistentHash(defaultReplicaCount, hash.Hash)
}
+4 -9
View File
@@ -7,6 +7,7 @@ import (
"strings"
"time"
"github.com/zeromicro/go-zero/zrpc/internal/balancer/p2c"
"github.com/zeromicro/go-zero/zrpc/internal/clientinterceptors"
"github.com/zeromicro/go-zero/zrpc/resolver"
"google.golang.org/grpc"
@@ -52,6 +53,9 @@ func NewClient(target string, middlewares ClientMiddlewaresConf, opts ...ClientO
middlewares: middlewares,
}
svcCfg := fmt.Sprintf(`{"loadBalancingPolicy":"%s"}`, p2c.Name)
balancerOpt := WithDialOption(grpc.WithDefaultServiceConfig(svcCfg))
opts = append([]ClientOption{balancerOpt}, opts...)
if err := cli.dial(target, opts...); err != nil {
return nil, err
}
@@ -141,15 +145,6 @@ func (c *client) dial(server string, opts ...ClientOption) error {
return nil
}
// WithBlock sets the dialing to be blocking.
// Deprecated: blocking dials are not recommended by gRPC.
// See https://github.com/grpc/grpc-go/blob/master/Documentation/anti-patterns.md
func WithBlock() ClientOption {
return func(options *ClientOptions) {
options.NonBlock = false
}
}
// WithDialOption returns a func to customize a ClientOptions with given dial option.
func WithDialOption(opt grpc.DialOption) ClientOption {
return func(options *ClientOptions) {
-7
View File
@@ -34,13 +34,6 @@ func TestWithNonBlock(t *testing.T) {
assert.True(t, options.NonBlock)
}
func TestWithBlock(t *testing.T) {
var options ClientOptions
opt := WithBlock()
opt(&options)
assert.False(t, options.NonBlock)
}
func TestWithStreamClientInterceptor(t *testing.T) {
var options ClientOptions
opt := WithStreamClientInterceptor(func(ctx context.Context, desc *grpc.StreamDesc,
@@ -63,12 +63,8 @@ func UnaryStatInterceptor(metrics *stat.Metrics, conf StatConf) grpc.UnaryServer
}
func isSlow(duration, durationThreshold time.Duration) bool {
// Prioritize explicit config over global setting
if durationThreshold > 0 {
return duration > durationThreshold
}
return duration > slowThreshold.Load()
return duration > slowThreshold.Load() ||
(durationThreshold > 0 && duration > durationThreshold)
}
func logDuration(ctx context.Context, method string, req any, duration time.Duration,
@@ -252,15 +252,6 @@ func Test_isSlow(t *testing.T) {
SetSlowThreshold(time.Millisecond * 100)
},
},
{
"config_priority_fix",
args{
duration: time.Millisecond * 600,
staticSlowThreshold: time.Millisecond * 1000,
},
false,
nil,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
+23 -23
View File
@@ -5,7 +5,7 @@ import (
"github.com/zeromicro/go-zero/core/lang"
"github.com/zeromicro/go-zero/core/logx"
"k8s.io/api/discovery/v1"
"k8s.io/api/core/v1"
"k8s.io/client-go/tools/cache"
)
@@ -28,9 +28,9 @@ func NewEventHandler(update func([]string)) *EventHandler {
// OnAdd handles the endpoints add events.
func (h *EventHandler) OnAdd(obj any, _ bool) {
endpoints, ok := obj.(*v1.EndpointSlice)
endpoints, ok := obj.(*v1.Endpoints)
if !ok {
logx.Errorf("%v is not an object with type *v1.EndpointSlice", obj)
logx.Errorf("%v is not an object with type *v1.Endpoints", obj)
return
}
@@ -38,10 +38,10 @@ func (h *EventHandler) OnAdd(obj any, _ bool) {
defer h.lock.Unlock()
var changed bool
for _, point := range endpoints.Endpoints {
for _, address := range point.Addresses {
if _, ok := h.endpoints[address]; !ok {
h.endpoints[address] = lang.Placeholder
for _, sub := range endpoints.Subsets {
for _, point := range sub.Addresses {
if _, ok := h.endpoints[point.IP]; !ok {
h.endpoints[point.IP] = lang.Placeholder
changed = true
}
}
@@ -54,9 +54,9 @@ func (h *EventHandler) OnAdd(obj any, _ bool) {
// OnDelete handles the endpoints delete events.
func (h *EventHandler) OnDelete(obj any) {
endpoints, ok := obj.(*v1.EndpointSlice)
endpoints, ok := obj.(*v1.Endpoints)
if !ok {
logx.Errorf("%v is not an object with type *v1.EndpointSlice", obj)
logx.Errorf("%v is not an object with type *v1.Endpoints", obj)
return
}
@@ -64,10 +64,10 @@ func (h *EventHandler) OnDelete(obj any) {
defer h.lock.Unlock()
var changed bool
for _, point := range endpoints.Endpoints {
for _, address := range point.Addresses {
if _, ok := h.endpoints[address]; ok {
delete(h.endpoints, address)
for _, sub := range endpoints.Subsets {
for _, point := range sub.Addresses {
if _, ok := h.endpoints[point.IP]; ok {
delete(h.endpoints, point.IP)
changed = true
}
}
@@ -80,35 +80,35 @@ func (h *EventHandler) OnDelete(obj any) {
// OnUpdate handles the endpoints update events.
func (h *EventHandler) OnUpdate(oldObj, newObj any) {
oldEndpointSlices, ok := oldObj.(*v1.EndpointSlice)
oldEndpoints, ok := oldObj.(*v1.Endpoints)
if !ok {
logx.Errorf("%v is not an object with type *v1.EndpointSlice", oldObj)
logx.Errorf("%v is not an object with type *v1.Endpoints", oldObj)
return
}
newEndpointSlices, ok := newObj.(*v1.EndpointSlice)
newEndpoints, ok := newObj.(*v1.Endpoints)
if !ok {
logx.Errorf("%v is not an object with type *v1.EndpointSlice", newObj)
logx.Errorf("%v is not an object with type *v1.Endpoints", newObj)
return
}
if oldEndpointSlices.ResourceVersion == newEndpointSlices.ResourceVersion {
if oldEndpoints.ResourceVersion == newEndpoints.ResourceVersion {
return
}
h.Update(newEndpointSlices)
h.Update(newEndpoints)
}
// Update updates the endpoints.
func (h *EventHandler) Update(endpoints *v1.EndpointSlice) {
func (h *EventHandler) Update(endpoints *v1.Endpoints) {
h.lock.Lock()
defer h.lock.Unlock()
old := h.endpoints
h.endpoints = make(map[string]lang.PlaceholderType)
for _, point := range endpoints.Endpoints {
for _, address := range point.Addresses {
h.endpoints[address] = lang.Placeholder
for _, sub := range endpoints.Subsets {
for _, point := range sub.Addresses {
h.endpoints[point.IP] = lang.Placeholder
}
}
+154 -108
View File
@@ -4,7 +4,7 @@ import (
"testing"
"github.com/stretchr/testify/assert"
discoveryv1 "k8s.io/api/discovery/v1"
v1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
@@ -14,19 +14,21 @@ func TestAdd(t *testing.T) {
endpoints = change
})
h.OnAdd("bad", false)
h.OnAdd(&discoveryv1.EndpointSlice{
Endpoints: []discoveryv1.Endpoint{
{
Addresses: []string{"0.0.0.1"},
},
{
Addresses: []string{"0.0.0.2"},
},
{
Addresses: []string{"0.0.0.3"},
h.OnAdd(&v1.Endpoints{Subsets: []v1.EndpointSubset{
{
Addresses: []v1.EndpointAddress{
{
IP: "0.0.0.1",
},
{
IP: "0.0.0.2",
},
{
IP: "0.0.0.3",
},
},
},
}, false)
}}, false)
assert.ElementsMatch(t, []string{"0.0.0.1", "0.0.0.2", "0.0.0.3"}, endpoints)
}
@@ -35,30 +37,34 @@ func TestDelete(t *testing.T) {
h := NewEventHandler(func(change []string) {
endpoints = change
})
h.OnAdd(&discoveryv1.EndpointSlice{
Endpoints: []discoveryv1.Endpoint{
{
Addresses: []string{"0.0.0.1"},
},
{
Addresses: []string{"0.0.0.2"},
},
{
Addresses: []string{"0.0.0.3"},
h.OnAdd(&v1.Endpoints{Subsets: []v1.EndpointSubset{
{
Addresses: []v1.EndpointAddress{
{
IP: "0.0.0.1",
},
{
IP: "0.0.0.2",
},
{
IP: "0.0.0.3",
},
},
},
}, false)
}}, false)
h.OnDelete("bad")
h.OnDelete(&discoveryv1.EndpointSlice{
Endpoints: []discoveryv1.Endpoint{
{
Addresses: []string{"0.0.0.1"},
},
{
Addresses: []string{"0.0.0.2"},
h.OnDelete(&v1.Endpoints{Subsets: []v1.EndpointSubset{
{
Addresses: []v1.EndpointAddress{
{
IP: "0.0.0.1",
},
{
IP: "0.0.0.2",
},
},
},
})
}})
assert.ElementsMatch(t, []string{"0.0.0.3"}, endpoints)
}
@@ -67,28 +73,36 @@ func TestUpdate(t *testing.T) {
h := NewEventHandler(func(change []string) {
endpoints = change
})
h.OnUpdate(&discoveryv1.EndpointSlice{
Endpoints: []discoveryv1.Endpoint{
h.OnUpdate(&v1.Endpoints{
Subsets: []v1.EndpointSubset{
{
Addresses: []string{"0.0.0.1"},
},
{
Addresses: []string{"0.0.0.2"},
Addresses: []v1.EndpointAddress{
{
IP: "0.0.0.1",
},
{
IP: "0.0.0.2",
},
},
},
},
ObjectMeta: metav1.ObjectMeta{
ResourceVersion: "1",
},
}, &discoveryv1.EndpointSlice{
Endpoints: []discoveryv1.Endpoint{
}, &v1.Endpoints{
Subsets: []v1.EndpointSubset{
{
Addresses: []string{"0.0.0.1"},
},
{
Addresses: []string{"0.0.0.2"},
},
{
Addresses: []string{"0.0.0.3"},
Addresses: []v1.EndpointAddress{
{
IP: "0.0.0.1",
},
{
IP: "0.0.0.2",
},
{
IP: "0.0.0.3",
},
},
},
},
ObjectMeta: metav1.ObjectMeta{
@@ -102,25 +116,33 @@ func TestUpdateNoChange(t *testing.T) {
h := NewEventHandler(func(change []string) {
assert.Fail(t, "should not called")
})
h.OnUpdate(&discoveryv1.EndpointSlice{
Endpoints: []discoveryv1.Endpoint{
h.OnUpdate(&v1.Endpoints{
Subsets: []v1.EndpointSubset{
{
Addresses: []string{"0.0.0.1"},
},
{
Addresses: []string{"0.0.0.2"},
Addresses: []v1.EndpointAddress{
{
IP: "0.0.0.1",
},
{
IP: "0.0.0.2",
},
},
},
},
ObjectMeta: metav1.ObjectMeta{
ResourceVersion: "1",
},
}, &discoveryv1.EndpointSlice{
Endpoints: []discoveryv1.Endpoint{
}, &v1.Endpoints{
Subsets: []v1.EndpointSubset{
{
Addresses: []string{"0.0.0.1"},
},
{
Addresses: []string{"0.0.0.2"},
Addresses: []v1.EndpointAddress{
{
IP: "0.0.0.1",
},
{
IP: "0.0.0.2",
},
},
},
},
ObjectMeta: metav1.ObjectMeta{
@@ -134,35 +156,45 @@ func TestUpdateChangeWithDifferentVersion(t *testing.T) {
h := NewEventHandler(func(change []string) {
endpoints = change
})
h.OnAdd(&discoveryv1.EndpointSlice{
Endpoints: []discoveryv1.Endpoint{
{
Addresses: []string{"0.0.0.1"},
},
{
Addresses: []string{"0.0.0.3"},
h.OnAdd(&v1.Endpoints{Subsets: []v1.EndpointSubset{
{
Addresses: []v1.EndpointAddress{
{
IP: "0.0.0.1",
},
{
IP: "0.0.0.3",
},
},
},
}, false)
h.OnUpdate(&discoveryv1.EndpointSlice{
Endpoints: []discoveryv1.Endpoint{
}}, false)
h.OnUpdate(&v1.Endpoints{
Subsets: []v1.EndpointSubset{
{
Addresses: []string{"0.0.0.1"},
},
{
Addresses: []string{"0.0.0.3"},
Addresses: []v1.EndpointAddress{
{
IP: "0.0.0.1",
},
{
IP: "0.0.0.3",
},
},
},
},
ObjectMeta: metav1.ObjectMeta{
ResourceVersion: "1",
},
}, &discoveryv1.EndpointSlice{
Endpoints: []discoveryv1.Endpoint{
}, &v1.Endpoints{
Subsets: []v1.EndpointSubset{
{
Addresses: []string{"0.0.0.1"},
},
{
Addresses: []string{"0.0.0.2"},
Addresses: []v1.EndpointAddress{
{
IP: "0.0.0.1",
},
{
IP: "0.0.0.2",
},
},
},
},
ObjectMeta: metav1.ObjectMeta{
@@ -177,49 +209,63 @@ func TestUpdateNoChangeWithDifferentVersion(t *testing.T) {
h := NewEventHandler(func(change []string) {
endpoints = change
})
h.OnAdd(&discoveryv1.EndpointSlice{
Endpoints: []discoveryv1.Endpoint{
{
Addresses: []string{"0.0.0.1"},
},
{
Addresses: []string{"0.0.0.2"},
h.OnAdd(&v1.Endpoints{Subsets: []v1.EndpointSubset{
{
Addresses: []v1.EndpointAddress{
{
IP: "0.0.0.1",
},
{
IP: "0.0.0.2",
},
},
},
}, false)
h.OnUpdate("bad", &discoveryv1.EndpointSlice{
Endpoints: []discoveryv1.Endpoint{
{
Addresses: []string{"0.0.0.1"},
}}, false)
h.OnUpdate("bad", &v1.Endpoints{Subsets: []v1.EndpointSubset{
{
Addresses: []v1.EndpointAddress{
{
IP: "0.0.0.1",
},
},
},
})
h.OnUpdate(&discoveryv1.EndpointSlice{
Endpoints: []discoveryv1.Endpoint{
{
Addresses: []string{"0.0.0.1"},
}})
h.OnUpdate(&v1.Endpoints{Subsets: []v1.EndpointSubset{
{
Addresses: []v1.EndpointAddress{
{
IP: "0.0.0.1",
},
},
},
}, "bad")
h.OnUpdate(&discoveryv1.EndpointSlice{
Endpoints: []discoveryv1.Endpoint{
}}, "bad")
h.OnUpdate(&v1.Endpoints{
Subsets: []v1.EndpointSubset{
{
Addresses: []string{"0.0.0.1"},
},
{
Addresses: []string{"0.0.0.2"},
Addresses: []v1.EndpointAddress{
{
IP: "0.0.0.1",
},
{
IP: "0.0.0.2",
},
},
},
},
ObjectMeta: metav1.ObjectMeta{
ResourceVersion: "1",
},
}, &discoveryv1.EndpointSlice{
Endpoints: []discoveryv1.Endpoint{
}, &v1.Endpoints{
Subsets: []v1.EndpointSubset{
{
Addresses: []string{"0.0.0.1"},
},
{
Addresses: []string{"0.0.0.2"},
Addresses: []v1.EndpointAddress{
{
IP: "0.0.0.1",
},
{
IP: "0.0.0.2",
},
},
},
},
ObjectMeta: metav1.ObjectMeta{
+13 -38
View File
@@ -18,8 +18,8 @@ import (
)
const (
resyncInterval = 5 * time.Minute
serviceSelector = "kubernetes.io/service-name="
resyncInterval = 5 * time.Minute
nameSelector = "metadata.name="
)
type kubeResolver struct {
@@ -60,33 +60,14 @@ func (b *kubeBuilder) Build(target resolver.Target, cc resolver.ClientConn,
}
if svc.Port == 0 {
endpointSlices, err := cs.DiscoveryV1().EndpointSlices(svc.Namespace).List(context.Background(),
v1.ListOptions{
LabelSelector: serviceSelector + svc.Name,
})
// getting endpoints is only to get the port
endpoints, err := cs.CoreV1().Endpoints(svc.Namespace).Get(
context.Background(), svc.Name, v1.GetOptions{})
if err != nil {
return nil, err
}
if len(endpointSlices.Items) == 0 {
return nil, fmt.Errorf("no endpoint slices found for service %s in namespace %s",
svc.Name, svc.Namespace)
}
// Find the first slice with a valid port.
// Since this resolver is used for in-cluster service discovery,
// we expect at least one port to be available.
var foundPort bool
for _, slice := range endpointSlices.Items {
if len(slice.Ports) > 0 && slice.Ports[0].Port != nil {
svc.Port = int(*slice.Ports[0].Port)
foundPort = true
break
}
}
if !foundPort {
return nil, fmt.Errorf("no valid port found in endpoint slices for service %s in namespace %s",
svc.Name, svc.Namespace)
}
svc.Port = int(endpoints.Subsets[0].Ports[0].Port)
}
handler := kube.NewEventHandler(func(endpoints []string) {
@@ -107,29 +88,23 @@ func (b *kubeBuilder) Build(target resolver.Target, cc resolver.ClientConn,
inf := informers.NewSharedInformerFactoryWithOptions(cs, resyncInterval,
informers.WithNamespace(svc.Namespace),
informers.WithTweakListOptions(func(options *v1.ListOptions) {
options.LabelSelector = serviceSelector + svc.Name
options.FieldSelector = nameSelector + svc.Name
}))
in := inf.Discovery().V1().EndpointSlices()
in := inf.Core().V1().Endpoints()
_, err = in.Informer().AddEventHandler(handler)
if err != nil {
return nil, err
}
// get the initial endpoint slices, cannot use the previous endpoint slices,
// because the endpoint slices may be updated before/after the informer is started.
endpointSlices, err := cs.DiscoveryV1().EndpointSlices(svc.Namespace).List(
context.Background(), v1.ListOptions{
LabelSelector: serviceSelector + svc.Name,
})
// get the initial endpoints, cannot use the previous endpoints,
// because the endpoints may be updated before/after the informer is started.
endpoints, err := cs.CoreV1().Endpoints(svc.Namespace).Get(
context.Background(), svc.Name, v1.GetOptions{})
if err != nil {
return nil, err
}
// Aggregate endpoints from all EndpointSlices.
// Use OnAdd (not Update) to accumulate addresses across multiple slices.
for _, endpointSlice := range endpointSlices.Items {
handler.OnAdd(&endpointSlice, false)
}
handler.Update(endpoints)
r := &kubeResolver{
cc: cc,