Compare commits

..

16 Commits

Author SHA1 Message Date
copilot-swe-agent[bot]
c37121ff87 Add DestroyAll() method to syncx.Pool with comprehensive tests
Co-authored-by: kevwan <1918356+kevwan@users.noreply.github.com>
2025-09-26 14:21:07 +00:00
copilot-swe-agent[bot]
f08d9329a8 Initial plan 2025-09-26 14:15:49 +00:00
Remember
988fb9d9bf fix: SSE handler blocking (#5181)
Co-authored-by: Kevin Wan <wanjunfeng@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-09-26 13:53:42 +00:00
Copilot
d212c81bca Add GitHub Copilot instructions for go-zero project (#5178)
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: kevwan <1918356+kevwan@users.noreply.github.com>
2025-09-20 13:40:43 +08:00
Kevin Wan
bc43df2641 optimize: mapreduce panic stacktrace (#5168) 2025-09-14 19:33:09 +08:00
dependabot[bot]
351b8cb37b chore(deps): bump github.com/redis/go-redis/v9 from 9.13.0 to 9.14.0 (#5169)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-09-11 22:07:13 +08:00
wanwu
0d681a2e29 opt: optimization of machine performance data reading (#5174)
Co-authored-by: sam.yang <sam.yang@yijinin.com>
2025-09-11 13:56:07 +00:00
dependabot[bot]
5ea027c5de chore(deps): bump actions/setup-go from 5 to 6 (#5156)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-09-09 21:07:44 +08:00
dependabot[bot]
5de6112dcd chore(deps): bump actions/stale from 9 to 10 (#5157)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-09-09 18:55:45 +08:00
Kevin Wan
4fb51723b7 Add company to the list (#5153) 2025-09-07 22:53:49 +08:00
me-cs
06502d1115 update:optimize slice find and Unquote func (#5108) 2025-09-07 00:41:45 +00:00
kesonan
3854d6dd00 fix array type generation error (#5142) 2025-09-04 13:41:15 +00:00
dependabot[bot]
895854913a chore(deps): bump github.com/spf13/pflag from 1.0.7 to 1.0.10 in /tools/goctl (#5141)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-09-04 21:35:50 +08:00
dependabot[bot]
ef753b8857 chore(deps): bump github.com/spf13/cobra from 1.9.1 to 1.10.1 in /tools/goctl (#5147)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-09-04 21:28:40 +08:00
dependabot[bot]
9c16fede73 chore(deps): bump github.com/redis/go-redis/v9 from 9.12.1 to 9.13.0 (#5149)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-09-04 21:17:33 +08:00
Kevin Wan
ce11adb5e4 feat: add code generation headers in safe to edit files (#5136) 2025-09-01 21:27:30 +08:00
38 changed files with 745 additions and 77 deletions

197
.github/copilot-instructions.md vendored Normal file
View File

@@ -0,0 +1,197 @@
# GitHub Copilot Instructions for go-zero
This document provides guidelines for GitHub Copilot when assisting with development in the go-zero project.
## Project Overview
go-zero is a web and RPC framework with lots of built-in engineering practices designed to ensure the stability of busy services with resilience design. It has been serving sites with tens of millions of users for years.
### Key Architecture Components
- **REST API framework** (`rest/`) - HTTP service framework with middleware support
- **RPC framework** (`zrpc/`) - gRPC-based RPC framework with service discovery
- **Core utilities** (`core/`) - Foundational components including:
- Circuit breakers, rate limiters, load shedding
- Caching, stores (Redis, MongoDB, SQL)
- Concurrency control, metrics, tracing
- Configuration management
- **Code generation tool** (`tools/goctl/`) - CLI tool for generating code from API files
## Coding Standards and Conventions
### Code Style
1. **Follow Go conventions**: Use `gofmt` for formatting, follow effective Go practices
2. **Package naming**: Use lowercase, single-word package names when possible
3. **Error handling**: Always handle errors explicitly, use `errorx.BatchError` for multiple errors
4. **Context propagation**: Always pass `context.Context` as the first parameter for functions that may block
5. **Configuration structures**: Use struct tags with JSON annotations and default values
Example configuration pattern:
```go
type Config struct {
Host string `json:",default=0.0.0.0"`
Port int `json:",default=8080"`
Timeout int `json:",default=3000"`
Optional string `json:",optional"`
}
```
### Interface Design
1. **Small interfaces**: Follow Go's preference for small, focused interfaces
2. **Context methods**: Provide both context and non-context versions of methods
3. **Options pattern**: Use functional options for complex configuration
Example:
```go
func (c *Client) Get(key string, val any) error {
return c.GetCtx(context.Background(), key, val)
}
func (c *Client) GetCtx(ctx context.Context, key string, val any) error {
// implementation
}
```
### Testing Patterns
1. **Test file naming**: Use `*_test.go` suffix
2. **Test function naming**: Use `TestFunctionName` pattern
3. **Use testify/assert**: Prefer `assert` package for assertions
4. **Table-driven tests**: Use table-driven tests for multiple scenarios
5. **Mock interfaces**: Use `go.uber.org/mock` for mocking
6. **Test helpers**: Use `redistest`, `mongtest` helpers for database testing
Example test pattern:
```go
func TestSomething(t *testing.T) {
tests := []struct {
name string
input string
expected string
wantErr bool
}{
{"valid case", "input", "output", false},
{"error case", "bad", "", true},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
result, err := SomeFunction(tt.input)
if tt.wantErr {
assert.Error(t, err)
return
}
assert.NoError(t, err)
assert.Equal(t, tt.expected, result)
})
}
}
```
## Framework-Specific Guidelines
### REST API Development
1. **API Definition**: Use `.api` files to define REST APIs
2. **Handler pattern**: Separate business logic into logic packages
3. **Middleware**: Use built-in middlewares (tracing, logging, metrics, recovery)
4. **Response handling**: Use `httpx.WriteJson` for JSON responses
5. **Error handling**: Use `httpx.Error` for HTTP error responses
### RPC Development
1. **Protocol Buffers**: Use protobuf for service definitions
2. **Service discovery**: Integrate with etcd for service registration
3. **Load balancing**: Use built-in load balancing strategies
4. **Interceptors**: Implement interceptors for cross-cutting concerns
### Database Operations
1. **SQL operations**: Use `sqlx` package for database operations
2. **Caching**: Implement caching patterns with `cache` package
3. **Transactions**: Use proper transaction handling
4. **Connection pooling**: Configure appropriate connection pools
Example cache pattern:
```go
err := c.QueryRowCtx(ctx, &dest, key, func(ctx context.Context, conn sqlx.SqlConn) error {
return conn.QueryRowCtx(ctx, &dest, query, args...)
})
```
### Configuration Management
1. **YAML configuration**: Use YAML for configuration files
2. **Environment variables**: Support environment variable overrides
3. **Validation**: Include proper validation for configuration parameters
4. **Sensible defaults**: Provide reasonable default values
## Error Handling Best Practices
1. **Wrap errors**: Use `fmt.Errorf` with `%w` verb to wrap errors
2. **Custom errors**: Define custom error types when needed
3. **Error logging**: Log errors appropriately with context
4. **Graceful degradation**: Implement fallback mechanisms
## Performance Considerations
1. **Resource pools**: Use connection pools and worker pools
2. **Circuit breakers**: Implement circuit breaker patterns for external calls
3. **Rate limiting**: Apply rate limiting to protect services
4. **Load shedding**: Implement adaptive load shedding
5. **Metrics**: Add appropriate metrics and monitoring
## Security Guidelines
1. **Input validation**: Validate all input parameters
2. **SQL injection prevention**: Use parameterized queries
3. **Authentication**: Implement proper JWT token handling
4. **HTTPS**: Support TLS/HTTPS configurations
5. **CORS**: Configure CORS appropriately for web APIs
## Documentation Standards
1. **Package documentation**: Include package-level documentation
2. **Function documentation**: Document exported functions with examples
3. **API documentation**: Maintain API documentation in sync
4. **README updates**: Update README for significant changes
## Common Patterns to Follow
### Service Configuration
```go
type ServiceConf struct {
Name string
Log logx.LogConf
Mode string `json:",default=pro,options=[dev,test,pre,pro]"`
// ... other common fields
}
```
### Middleware Implementation
```go
func SomeMiddleware() rest.Middleware {
return func(next http.HandlerFunc) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
// Pre-processing
next.ServeHTTP(w, r)
// Post-processing
}
}
}
```
### Resource Management
Always implement proper resource cleanup using defer and context cancellation.
## Build and Test Commands
- Build: `go build ./...`
- Test: `go test ./...`
- Test with race detection: `go test -race ./...`
- Format: `gofmt -w .`
- Generate code: `goctl api go -api *.api -dir .`
Remember to run tests and ensure all checks pass before submitting changes. The project emphasizes high quality, performance, and reliability, so these should be primary considerations in all development work.

View File

@@ -15,7 +15,7 @@ jobs:
uses: actions/checkout@v5
- name: Set up Go 1.x
uses: actions/setup-go@v5
uses: actions/setup-go@v6
with:
go-version-file: go.mod
check-latest: true
@@ -55,7 +55,7 @@ jobs:
uses: actions/checkout@v5
- name: Set up Go 1.x
uses: actions/setup-go@v5
uses: actions/setup-go@v6
with:
# make sure Go version compatible with go-zero
go-version-file: go.mod

View File

@@ -7,7 +7,7 @@ jobs:
close-issues:
runs-on: ubuntu-latest
steps:
- uses: actions/stale@v9
- uses: actions/stale@v10
with:
days-before-issue-stale: 365
days-before-issue-close: 90

View File

@@ -13,7 +13,7 @@ jobs:
- uses: actions/checkout@v5
- name: Set up Go
uses: actions/setup-go@v5
uses: actions/setup-go@v6
with:
go-version: '1.21'

View File

@@ -3,6 +3,9 @@ package mr
import (
"context"
"errors"
"fmt"
"runtime/debug"
"strings"
"sync"
"sync/atomic"
@@ -183,12 +186,16 @@ func buildOptions(opts ...Option) *mapReduceOptions {
return options
}
func buildPanicInfo(r any, stack []byte) string {
return fmt.Sprintf("%+v\n\n%s", r, strings.TrimSpace(string(stack)))
}
func buildSource[T any](generate GenerateFunc[T], panicChan *onceChan) chan T {
source := make(chan T)
go func() {
defer func() {
if r := recover(); r != nil {
panicChan.write(r)
panicChan.write(buildPanicInfo(r, debug.Stack()))
}
close(source)
}()
@@ -235,7 +242,7 @@ func executeMappers[T, U any](mCtx mapperContext[T, U]) {
defer func() {
if r := recover(); r != nil {
atomic.AddInt32(&failed, 1)
mCtx.panicChan.write(r)
mCtx.panicChan.write(buildPanicInfo(r, debug.Stack()))
}
wg.Done()
<-pool
@@ -289,7 +296,7 @@ func mapReduceWithPanicChan[T, U, V any](source <-chan T, panicChan *onceChan, m
defer func() {
drain(collector)
if r := recover(); r != nil {
panicChan.write(r)
panicChan.write(buildPanicInfo(r, debug.Stack()))
}
finish()
}()

View File

@@ -3,6 +3,7 @@ package mr
import (
"context"
"errors"
"fmt"
"io"
"log"
"runtime"
@@ -148,11 +149,28 @@ func TestForEach(t *testing.T) {
assert.Equal(t, tasks/2, int(count))
})
}
t.Run("all", func(t *testing.T) {
defer goleak.VerifyNone(t)
func TestPanics(t *testing.T) {
defer goleak.VerifyNone(t)
const tasks = 1000
verify := func(t *testing.T, r any) {
panicStr := fmt.Sprintf("%v", r)
assert.Contains(t, panicStr, "foo")
assert.Contains(t, panicStr, "goroutine")
assert.Contains(t, panicStr, "runtime/debug.Stack")
panic(r)
}
t.Run("ForEach run panics", func(t *testing.T) {
assert.Panics(t, func() {
defer func() {
if r := recover(); r != nil {
verify(t, r)
}
}()
assert.PanicsWithValue(t, "foo", func() {
ForEach(func(source chan<- int) {
for i := 0; i < tasks; i++ {
source <- i
@@ -162,28 +180,31 @@ func TestForEach(t *testing.T) {
})
})
})
}
func TestGeneratePanic(t *testing.T) {
defer goleak.VerifyNone(t)
t.Run("ForEach generate panics", func(t *testing.T) {
assert.Panics(t, func() {
defer func() {
if r := recover(); r != nil {
verify(t, r)
}
}()
t.Run("all", func(t *testing.T) {
assert.PanicsWithValue(t, "foo", func() {
ForEach(func(source chan<- int) {
panic("foo")
}, func(item int) {
})
})
})
}
func TestMapperPanic(t *testing.T) {
defer goleak.VerifyNone(t)
const tasks = 1000
var run int32
t.Run("all", func(t *testing.T) {
assert.PanicsWithValue(t, "foo", func() {
t.Run("Mapper panics", func(t *testing.T) {
assert.Panics(t, func() {
defer func() {
if r := recover(); r != nil {
verify(t, r)
}
}()
_, _ = MapReduce(func(source chan<- int) {
for i := 0; i < tasks; i++ {
source <- i

View File

@@ -5,6 +5,8 @@ import (
"io"
"os"
"runtime"
"runtime/debug"
"runtime/metrics"
"time"
)
@@ -28,10 +30,29 @@ func displayStatsWithWriter(writer io.Writer, interval ...time.Duration) {
ticker := time.NewTicker(duration)
defer ticker.Stop()
for range ticker.C {
var m runtime.MemStats
runtime.ReadMemStats(&m)
var (
alloc, totalAlloc, sys uint64
samples = []metrics.Sample{
{Name: "/memory/classes/heap/objects:bytes"},
{Name: "/gc/heap/allocs:bytes"},
{Name: "/memory/classes/total:bytes"},
}
)
metrics.Read(samples)
if samples[0].Value.Kind() == metrics.KindUint64 {
alloc = samples[0].Value.Uint64()
}
if samples[1].Value.Kind() == metrics.KindUint64 {
totalAlloc = samples[1].Value.Uint64()
}
if samples[2].Value.Kind() == metrics.KindUint64 {
sys = samples[2].Value.Uint64()
}
var stats debug.GCStats
debug.ReadGCStats(&stats)
fmt.Fprintf(writer, "Goroutines: %d, Alloc: %vm, TotalAlloc: %vm, Sys: %vm, NumGC: %v\n",
runtime.NumGoroutine(), m.Alloc/mega, m.TotalAlloc/mega, m.Sys/mega, m.NumGC)
runtime.NumGoroutine(), alloc/mega, totalAlloc/mega, sys/mega, stats.NumGC)
}
}()
}

View File

@@ -1,7 +1,8 @@
package stat
import (
"runtime"
"runtime/debug"
"runtime/metrics"
"sync/atomic"
"time"
@@ -56,8 +57,28 @@ func bToMb(b uint64) float32 {
}
func printUsage() {
var m runtime.MemStats
runtime.ReadMemStats(&m)
var (
alloc, totalAlloc, sys uint64
samples = []metrics.Sample{
{Name: "/memory/classes/heap/objects:bytes"},
{Name: "/gc/heap/allocs:bytes"},
{Name: "/memory/classes/total:bytes"},
}
stats debug.GCStats
)
metrics.Read(samples)
if samples[0].Value.Kind() == metrics.KindUint64 {
alloc = samples[0].Value.Uint64()
}
if samples[1].Value.Kind() == metrics.KindUint64 {
totalAlloc = samples[1].Value.Uint64()
}
if samples[2].Value.Kind() == metrics.KindUint64 {
sys = samples[2].Value.Uint64()
}
debug.ReadGCStats(&stats)
logx.Statf("CPU: %dm, MEMORY: Alloc=%.1fMi, TotalAlloc=%.1fMi, Sys=%.1fMi, NumGC=%d",
CpuUsage(), bToMb(m.Alloc), bToMb(m.TotalAlloc), bToMb(m.Sys), m.NumGC)
CpuUsage(), bToMb(alloc), bToMb(totalAlloc), bToMb(sys), stats.NumGC)
}

View File

@@ -100,6 +100,34 @@ func (p *Pool) Put(x any) {
p.cond.Signal()
}
// DestroyAll destroys all resources in the pool.
// It calls the destroy function on each resource and resets the pool state.
// This is useful when you need to forcefully clean up all resources, for example:
// - When removing an obsolete pool
// - When refreshing all resources after configuration changes
// - When avoiding resource leaks in dynamic pool scenarios
func (p *Pool) DestroyAll() {
p.lock.Lock()
defer p.lock.Unlock()
// Iterate through the linked list and destroy all resources
current := p.head
for current != nil {
next := current.next
if p.destroy != nil {
p.destroy(current.item)
}
current = next
}
// Reset pool state
p.head = nil
p.created = 0
// Wake up all waiting goroutines since the pool is now empty
p.cond.Broadcast()
}
// WithMaxAge returns a function to customize a Pool with given max age.
func WithMaxAge(duration time.Duration) PoolOption {
return func(pool *Pool) {

View File

@@ -107,6 +107,155 @@ func TestNewPoolPanics(t *testing.T) {
})
}
func TestPoolDestroyAll(t *testing.T) {
var destroyed []int
var destroyCount int32
destroyFunc := func(item any) {
destroyed = append(destroyed, item.(int))
atomic.AddInt32(&destroyCount, 1)
}
pool := NewPool(limit, create, destroyFunc)
// Put some resources into the pool
pool.Put(10)
pool.Put(20)
pool.Put(30)
// Destroy all resources
pool.DestroyAll()
// Verify all resources were destroyed
assert.Equal(t, int32(3), atomic.LoadInt32(&destroyCount))
assert.Contains(t, destroyed, 10)
assert.Contains(t, destroyed, 20)
assert.Contains(t, destroyed, 30)
// Verify pool is empty - next Get should create new resource
val := pool.Get()
assert.Equal(t, 1, val) // create() returns 1
}
func TestPoolDestroyAllEmpty(t *testing.T) {
var destroyCount int32
destroyFunc := func(_ any) {
atomic.AddInt32(&destroyCount, 1)
}
pool := NewPool(limit, create, destroyFunc)
// DestroyAll on empty pool should not panic
pool.DestroyAll()
// No resources should have been destroyed
assert.Equal(t, int32(0), atomic.LoadInt32(&destroyCount))
// Pool should still work normally
val := pool.Get()
assert.Equal(t, 1, val)
}
func TestPoolDestroyAllWithNilDestroy(t *testing.T) {
pool := NewPool(limit, create, nil)
// Put some resources into the pool
pool.Put(10)
pool.Put(20)
// DestroyAll with nil destroy function should not panic
pool.DestroyAll()
// Pool should be empty and work normally
val := pool.Get()
assert.Equal(t, 1, val)
}
func TestPoolDestroyAllConcurrency(t *testing.T) {
var destroyCount int32
var createCount int32
createFunc := func() any {
return atomic.AddInt32(&createCount, 1)
}
destroyFunc := func(_ any) {
atomic.AddInt32(&destroyCount, 1)
}
pool := NewPool(limit, createFunc, destroyFunc)
// Add some initial resources
for i := 0; i < 5; i++ {
pool.Put(i + 100)
}
var wg sync.WaitGroup
const goroutines = 10
// Concurrently perform various operations
for i := 0; i < goroutines; i++ {
wg.Add(1)
go func(id int) {
defer wg.Done()
switch id % 4 {
case 0:
// DestroyAll
pool.DestroyAll()
case 1:
// Get resources
val := pool.Get()
pool.Put(val)
case 2:
// Put resources
pool.Put(id + 1000)
case 3:
// Get and don't put back
pool.Get()
}
}(i)
}
wg.Wait()
// Final DestroyAll to clean up
pool.DestroyAll()
// Pool should work after concurrent operations
val := pool.Get()
assert.NotNil(t, val)
}
func TestPoolDestroyAllWakesWaitingGoroutines(t *testing.T) {
pool := NewPool(1, create, destroy) // Small pool size
// Fill the pool
resource := pool.Get()
assert.Equal(t, 1, resource)
var wg sync.WaitGroup
var gotResource bool
// Start a goroutine that will wait for a resource
wg.Add(1)
go func() {
defer wg.Done()
val := pool.Get() // This will block since pool is full
gotResource = true
assert.Equal(t, 1, val) // Should get a newly created resource after DestroyAll
}()
// Give the goroutine time to start waiting
time.Sleep(10 * time.Millisecond)
// DestroyAll should wake up the waiting goroutine
pool.DestroyAll()
wg.Wait()
assert.True(t, gotResource)
}
func create() any {
return 1
}

2
go.mod
View File

@@ -16,7 +16,7 @@ require (
github.com/jhump/protoreflect v1.17.0
github.com/pelletier/go-toml/v2 v2.2.2
github.com/prometheus/client_golang v1.21.1
github.com/redis/go-redis/v9 v9.12.1
github.com/redis/go-redis/v9 v9.14.0
github.com/spaolacci/murmur3 v1.1.0
github.com/stretchr/testify v1.11.1
go.etcd.io/etcd/api/v3 v3.5.15

4
go.sum
View File

@@ -154,8 +154,8 @@ github.com/prometheus/common v0.62.0 h1:xasJaQlnWAeyHdUBeGjXmutelfJHWMRr+Fg4QszZ
github.com/prometheus/common v0.62.0/go.mod h1:vyBcEuLSvWos9B1+CyL7JZ2up+uFzXhkqml0W5zIY1I=
github.com/prometheus/procfs v0.15.1 h1:YagwOFzUgYfKKHX6Dr+sHT7km/hxC76UB0learggepc=
github.com/prometheus/procfs v0.15.1/go.mod h1:fB45yRUv8NstnjriLhBQLuOUt+WW4BsoGhij/e3PBqk=
github.com/redis/go-redis/v9 v9.12.1 h1:k5iquqv27aBtnTm2tIkROUDp8JBXhXZIVu1InSgvovg=
github.com/redis/go-redis/v9 v9.12.1/go.mod h1:huWgSWd8mW6+m0VPhJjSSQ+d6Nh1VICQ6Q5lHuCH/Iw=
github.com/redis/go-redis/v9 v9.14.0 h1:u4tNCjXOyzfgeLN+vAZaW1xUooqWDqVEsZN0U01jfAE=
github.com/redis/go-redis/v9 v9.14.0/go.mod h1:huWgSWd8mW6+m0VPhJjSSQ+d6Nh1VICQ6Q5lHuCH/Iw=
github.com/rivo/uniseg v0.2.0 h1:S1pD9weZBuJdFmowNwbpi7BJ8TNftyUImj/0WQi72jY=
github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
github.com/rogpeppe/go-internal v1.10.0 h1:TMyTOH3F/DB16zRVcYyreMH6GnZZrwQVAoYjRBZyWFQ=

View File

@@ -304,6 +304,7 @@ go-zero 已被许多公司用于生产部署,接入场景如在线教育、电
>106. 无锡盛算信息技术有限公司
>107. 深圳市聚货通信息科技有限公司
>108. 浙江银盾云科技有限公司
>109. 南京造世网络科技有限公司
如果贵公司也已使用 go-zero欢迎在 [登记地址](https://github.com/zeromicro/go-zero/issues/602) 登记,仅仅为了推广,不做其它用途。

View File

@@ -1,3 +1,6 @@
// Code scaffolded by goctl. Safe to edit.
// goctl {{.version}}
package config
import {{.authImport}}

View File

@@ -0,0 +1,181 @@
package gogen
import (
"os"
"path/filepath"
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/zeromicro/go-zero/tools/goctl/internal/version"
)
// TestGenerationComments verifies that all generated files have appropriate generation comments
func TestGenerationComments(t *testing.T) {
// Create a temporary directory for our test
tempDir, err := os.MkdirTemp("", "goctl_test_")
require.NoError(t, err)
defer os.RemoveAll(tempDir)
// Create a simple API spec for testing
apiContent := `
syntax = "v1"
type HelloRequest {
Name string ` + "`json:\"name\"`" + `
}
type HelloResponse {
Message string ` + "`json:\"message\"`" + `
}
service hello-api {
@handler helloHandler
post /hello (HelloRequest) returns (HelloResponse)
}`
// Write the API spec to a temporary file
apiFile := filepath.Join(tempDir, "test.api")
err = os.WriteFile(apiFile, []byte(apiContent), 0644)
require.NoError(t, err)
// Parse and generate the API files using the correct function signature
err = DoGenProject(apiFile, tempDir, "gozero", false)
require.NoError(t, err)
// Define expected files and their comment types
expectedFiles := map[string]string{
// Files that should have "DO NOT EDIT" comments (regenerated files)
"internal/types/types.go": "DO NOT EDIT",
// Files that should have "Safe to edit" comments (scaffolded files)
"internal/handler/hellohandler.go": "Safe to edit",
"internal/config/config.go": "Safe to edit",
"hello.go": "Safe to edit", // main file
"internal/svc/servicecontext.go": "Safe to edit",
"internal/logic/hellologic.go": "Safe to edit",
}
// Check each file for the correct generation comment
for filePath, expectedCommentType := range expectedFiles {
fullPath := filepath.Join(tempDir, filePath)
// Skip if file doesn't exist (some files might not be generated in all cases)
if _, err := os.Stat(fullPath); os.IsNotExist(err) {
t.Logf("File %s does not exist, skipping", filePath)
continue
}
content, err := os.ReadFile(fullPath)
require.NoError(t, err, "Failed to read file: %s", filePath)
contentStr := string(content)
lines := strings.Split(contentStr, "\n")
// Check that the file starts with proper generation comments
require.GreaterOrEqual(t, len(lines), 2, "File %s should have at least 2 lines", filePath)
if expectedCommentType == "DO NOT EDIT" {
assert.Contains(t, lines[0], "// Code generated by goctl. DO NOT EDIT.",
"File %s should have 'DO NOT EDIT' comment as first line", filePath)
} else if expectedCommentType == "Safe to edit" {
assert.Contains(t, lines[0], "// Code scaffolded by goctl. Safe to edit.",
"File %s should have 'Safe to edit' comment as first line", filePath)
}
// Check that the second line contains the version
assert.Contains(t, lines[1], "// goctl",
"File %s should have version comment as second line", filePath)
assert.Contains(t, lines[1], version.BuildVersion,
"File %s should contain version %s in second line", filePath, version.BuildVersion)
}
}
// TestRoutesGenerationComment verifies routes files have "DO NOT EDIT" comment
func TestRoutesGenerationComment(t *testing.T) {
// Create a temporary directory for our test
tempDir, err := os.MkdirTemp("", "goctl_routes_test_")
require.NoError(t, err)
defer os.RemoveAll(tempDir)
// Create an API spec with multiple handlers to ensure routes file is generated
apiContent := `
syntax = "v1"
type HelloRequest {
Name string ` + "`json:\"name\"`" + `
}
type HelloResponse {
Message string ` + "`json:\"message\"`" + `
}
service hello-api {
@handler helloHandler
post /hello (HelloRequest) returns (HelloResponse)
@handler worldHandler
get /world returns (HelloResponse)
}`
// Write the API spec to a temporary file
apiFile := filepath.Join(tempDir, "test.api")
err = os.WriteFile(apiFile, []byte(apiContent), 0644)
require.NoError(t, err)
// Generate the API files using the correct function signature
err = DoGenProject(apiFile, tempDir, "gozero", false)
require.NoError(t, err)
// Check the routes file specifically
routesFile := filepath.Join(tempDir, "internal/handler/routes.go")
if _, err := os.Stat(routesFile); os.IsNotExist(err) {
t.Skip("Routes file not generated, skipping test")
return
}
content, err := os.ReadFile(routesFile)
require.NoError(t, err, "Failed to read routes.go")
contentStr := string(content)
lines := strings.Split(contentStr, "\n")
// Check that routes.go has "DO NOT EDIT" comment
require.GreaterOrEqual(t, len(lines), 2, "Routes file should have at least 2 lines")
assert.Contains(t, lines[0], "// Code generated by goctl. DO NOT EDIT.",
"Routes file should have 'DO NOT EDIT' comment")
assert.Contains(t, lines[1], "// goctl",
"Routes file should have version comment")
assert.Contains(t, lines[1], version.BuildVersion,
"Routes file should contain version %s", version.BuildVersion)
}
// TestVersionInTemplateData verifies that version is correctly passed to templates
func TestVersionInTemplateData(t *testing.T) {
// Test that BuildVersion is available
assert.NotEmpty(t, version.BuildVersion, "BuildVersion should not be empty")
}
// TestCommentsFollowGoStandards verifies our comments follow Go community standards
func TestCommentsFollowGoStandards(t *testing.T) {
// Test the format of our generation comments
doNotEditComment := "// Code generated by goctl. DO NOT EDIT."
safeToEditComment := "// Code scaffolded by goctl. Safe to edit."
// Both should be valid Go comments
assert.True(t, strings.HasPrefix(doNotEditComment, "//"),
"DO NOT EDIT comment should start with //")
assert.True(t, strings.HasPrefix(safeToEditComment, "//"),
"Safe to edit comment should start with //")
// Should contain key information
assert.Contains(t, doNotEditComment, "goctl",
"DO NOT EDIT comment should mention goctl")
assert.Contains(t, safeToEditComment, "goctl",
"Safe to edit comment should mention goctl")
assert.Contains(t, doNotEditComment, "DO NOT EDIT",
"Should clearly state DO NOT EDIT")
assert.Contains(t, safeToEditComment, "Safe to edit",
"Should clearly state Safe to edit")
}

View File

@@ -7,6 +7,7 @@ import (
"github.com/zeromicro/go-zero/tools/goctl/api/spec"
"github.com/zeromicro/go-zero/tools/goctl/config"
"github.com/zeromicro/go-zero/tools/goctl/internal/version"
"github.com/zeromicro/go-zero/tools/goctl/util/format"
"github.com/zeromicro/go-zero/tools/goctl/vars"
)
@@ -61,6 +62,7 @@ func genConfig(dir, projectPkg string, cfg *config.Config, api *spec.ApiSpec) er
"auth": strings.Join(auths, "\n"),
"jwtTrans": strings.Join(jwtTransList, "\n"),
"projectPkg": projectPkg,
"version": version.BuildVersion,
},
})
}

View File

@@ -8,6 +8,7 @@ import (
"github.com/zeromicro/go-zero/tools/goctl/api/spec"
"github.com/zeromicro/go-zero/tools/goctl/config"
"github.com/zeromicro/go-zero/tools/goctl/internal/version"
"github.com/zeromicro/go-zero/tools/goctl/util"
"github.com/zeromicro/go-zero/tools/goctl/util/format"
"github.com/zeromicro/go-zero/tools/goctl/util/pathx"
@@ -64,6 +65,7 @@ func genHandler(dir, rootPkg, projectPkg string, cfg *config.Config, group spec.
"HasDoc": len(route.JoinedDoc()) > 0,
"Doc": getDoc(route.JoinedDoc()),
"projectPkg": projectPkg,
"version": version.BuildVersion,
},
})
}

View File

@@ -7,6 +7,7 @@ import (
"github.com/zeromicro/go-zero/tools/goctl/api/spec"
"github.com/zeromicro/go-zero/tools/goctl/config"
"github.com/zeromicro/go-zero/tools/goctl/internal/version"
"github.com/zeromicro/go-zero/tools/goctl/util"
"github.com/zeromicro/go-zero/tools/goctl/util/format"
"github.com/zeromicro/go-zero/tools/goctl/util/pathx"
@@ -51,6 +52,7 @@ func genHandlerTest(dir, rootPkg, projectPkg string, cfg *config.Config, group s
"HasDoc": len(route.JoinedDoc()) > 0,
"Doc": getDoc(route.JoinedDoc()),
"projectPkg": projectPkg,
"version": version.BuildVersion,
},
})
}

View File

@@ -10,6 +10,7 @@ import (
"github.com/zeromicro/go-zero/tools/goctl/api/parser/g4/gen/api"
"github.com/zeromicro/go-zero/tools/goctl/api/spec"
"github.com/zeromicro/go-zero/tools/goctl/config"
"github.com/zeromicro/go-zero/tools/goctl/internal/version"
"github.com/zeromicro/go-zero/tools/goctl/util/format"
"github.com/zeromicro/go-zero/tools/goctl/util/pathx"
"github.com/zeromicro/go-zero/tools/goctl/vars"
@@ -92,6 +93,7 @@ func genLogicByRoute(dir, rootPkg, projectPkg string, cfg *config.Config, group
"hasDoc": len(route.JoinedDoc()) > 0,
"doc": getDoc(route.JoinedDoc()),
"projectPkg": projectPkg,
"version": version.BuildVersion,
},
})
}

View File

@@ -7,6 +7,7 @@ import (
"github.com/zeromicro/go-zero/tools/goctl/api/spec"
"github.com/zeromicro/go-zero/tools/goctl/config"
"github.com/zeromicro/go-zero/tools/goctl/internal/version"
"github.com/zeromicro/go-zero/tools/goctl/util/format"
"github.com/zeromicro/go-zero/tools/goctl/util/pathx"
)
@@ -74,6 +75,7 @@ func genLogicTestByRoute(dir, rootPkg, projectPkg string, cfg *config.Config, gr
"hasDoc": len(route.JoinedDoc()) > 0,
"doc": getDoc(route.JoinedDoc()),
"projectPkg": projectPkg,
"version": version.BuildVersion,
},
})
}

View File

@@ -7,6 +7,7 @@ import (
"github.com/zeromicro/go-zero/tools/goctl/api/spec"
"github.com/zeromicro/go-zero/tools/goctl/config"
"github.com/zeromicro/go-zero/tools/goctl/internal/version"
"github.com/zeromicro/go-zero/tools/goctl/util/format"
"github.com/zeromicro/go-zero/tools/goctl/util/pathx"
"github.com/zeromicro/go-zero/tools/goctl/vars"
@@ -39,6 +40,7 @@ func genMain(dir, rootPkg, projectPkg string, cfg *config.Config, api *spec.ApiS
"importPackages": genMainImports(rootPkg),
"serviceName": configName,
"projectPkg": projectPkg,
"version": version.BuildVersion,
},
})
}

View File

@@ -6,6 +6,7 @@ import (
"github.com/zeromicro/go-zero/tools/goctl/api/spec"
"github.com/zeromicro/go-zero/tools/goctl/config"
"github.com/zeromicro/go-zero/tools/goctl/internal/version"
"github.com/zeromicro/go-zero/tools/goctl/util/format"
)
@@ -31,7 +32,8 @@ func genMiddleware(dir string, cfg *config.Config, api *spec.ApiSpec) error {
templateFile: middlewareImplementCodeFile,
builtinTemplate: middlewareImplementCode,
data: map[string]string{
"name": strings.Title(name),
"name": strings.Title(name),
"version": version.BuildVersion,
},
})
if err != nil {

View File

@@ -7,6 +7,7 @@ import (
"github.com/zeromicro/go-zero/tools/goctl/api/spec"
"github.com/zeromicro/go-zero/tools/goctl/config"
"github.com/zeromicro/go-zero/tools/goctl/internal/version"
"github.com/zeromicro/go-zero/tools/goctl/util/format"
"github.com/zeromicro/go-zero/tools/goctl/util/pathx"
"github.com/zeromicro/go-zero/tools/goctl/vars"
@@ -54,6 +55,7 @@ func genServiceContext(dir, rootPkg, projectPkg string, cfg *config.Config, api
"middleware": middlewareStr,
"middlewareAssignment": middlewareAssignment,
"projectPkg": projectPkg,
"version": version.BuildVersion,
},
})
}

View File

@@ -1,3 +1,6 @@
// Code scaffolded by goctl. Safe to edit.
// goctl {{.version}}
package {{.PkgName}}
import (

View File

@@ -1,3 +1,6 @@
// Code scaffolded by goctl. Safe to edit.
// goctl {{.version}}
package {{.PkgName}}
import (

View File

@@ -1,3 +1,6 @@
// Code scaffolded by goctl. Safe to edit.
// goctl {{.version}}
package {{.pkgName}}
import (

View File

@@ -1,3 +1,6 @@
// Code scaffolded by goctl. Safe to edit.
// goctl {{.version}}
package {{.pkgName}}
import (

View File

@@ -1,3 +1,6 @@
// Code scaffolded by goctl. Safe to edit.
// goctl {{.version}}
package main
import (

View File

@@ -1,3 +1,6 @@
// Code scaffolded by goctl. Safe to edit.
// goctl {{.version}}
package middleware
import "net/http"

View File

@@ -1,3 +1,6 @@
// Code scaffolded by goctl. Safe to edit.
// goctl {{.version}}
package {{.PkgName}}
import (
@@ -27,11 +30,10 @@ func {{.HandlerName}}(svcCtx *svc.ServiceContext) http.HandlerFunc {
// w.Header().Set("Cache-Control", "no-cache")
// w.Header().Set("Connection", "keep-alive")
client := make(chan {{.ResponseType}}, 16)
defer func() {
close(client)
}()
l := {{.LogicName}}.New{{.LogicType}}(r.Context(), svcCtx)
threading.GoSafeCtx(r.Context(), func() {
defer close(client)
err := l.{{.Call}}({{if .HasRequest}}&req, {{end}}client)
if err != nil {
logc.Errorw(r.Context(), "{{.HandlerName}}", logc.Field("error", err))
@@ -41,7 +43,10 @@ func {{.HandlerName}}(svcCtx *svc.ServiceContext) http.HandlerFunc {
for {
select {
case data := <-client:
case data, ok := <-client:
if !ok {
return
}
output, err := json.Marshal(data)
if err != nil {
logc.Errorw(r.Context(), "{{.HandlerName}}", logc.Field("error", err))

View File

@@ -1,3 +1,6 @@
// Code scaffolded by goctl. Safe to edit.
// goctl {{.version}}
package {{.pkgName}}
import (

View File

@@ -1,3 +1,6 @@
// Code scaffolded by goctl. Safe to edit.
// goctl {{.version}}
package svc
import (

View File

@@ -16,8 +16,10 @@ func getBoolFromKVOrDefault(properties map[string]string, key string, def bool)
if len(val) == 0 {
return def
}
str := util.Unquote(val[0])
if len(str) == 0 {
//I think this function and those below should handle error, but they didn't.
//Since a default value (def) is provided, any parsing errors will result in the default being returned.
str, err := strconv.Unquote(val[0])
if err != nil || len(str) == 0 {
return def
}
res, _ := strconv.ParseBool(str)
@@ -33,8 +35,8 @@ func getStringFromKVOrDefault(properties map[string]string, key string, def stri
if len(val) == 0 {
return def
}
str := util.Unquote(val[0])
if len(str) == 0 {
str, err := strconv.Unquote(val[0])
if err != nil || len(str) == 0 {
return def
}
return str
@@ -50,8 +52,8 @@ func getListFromInfoOrDefault(properties map[string]string, key string, def []st
return def
}
str := util.Unquote(val[0])
if len(str) == 0 {
str, err := strconv.Unquote(val[0])
if err != nil || len(str) == 0 {
return def
}
resp := util.FieldsAndTrimSpace(str, commaRune)
@@ -66,8 +68,8 @@ func getFirstUsableString(def ...string) string {
return ""
}
for _, val := range def {
str := util.Unquote(val)
if len(str) != 0 {
str, err := strconv.Unquote(val)
if err == nil && len(str) != 0 {
return str
}
}

View File

@@ -10,8 +10,8 @@ require (
github.com/go-sql-driver/mysql v1.9.0
github.com/gookit/color v1.6.0
github.com/iancoleman/strcase v0.3.0
github.com/spf13/cobra v1.9.1
github.com/spf13/pflag v1.0.7
github.com/spf13/cobra v1.10.1
github.com/spf13/pflag v1.0.10
github.com/stretchr/testify v1.11.1
github.com/withfig/autocomplete-tools/integrations/cobra v1.2.1
github.com/zeromicro/antlr v0.0.1

View File

@@ -155,11 +155,11 @@ github.com/rogpeppe/go-internal v1.11.0/go.mod h1:ddIwULY96R17DhadqLgMfk9H9tvdUz
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/spaolacci/murmur3 v1.1.0 h1:7c1g84S4BPRrfL5Xrdp6fOJ206sU9y293DDHaoy0bLI=
github.com/spaolacci/murmur3 v1.1.0/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA=
github.com/spf13/cobra v1.9.1 h1:CXSaggrXdbHK9CF+8ywj8Amf7PBRmPCOJugH954Nnlo=
github.com/spf13/cobra v1.9.1/go.mod h1:nDyEzZ8ogv936Cinf6g1RU9MRY64Ir93oCnqb9wxYW0=
github.com/spf13/pflag v1.0.6/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/spf13/pflag v1.0.7 h1:vN6T9TfwStFPFM5XzjsvmzZkLuaLX+HS+0SeFLRgU6M=
github.com/spf13/pflag v1.0.7/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/spf13/cobra v1.10.1 h1:lJeBwCfmrnXthfAupyUTzJ/J4Nc1RsHC/mSRU2dll/s=
github.com/spf13/cobra v1.10.1/go.mod h1:7SmJGaTHFVBY0jW4NXGluQoLvhqFQM+6XSKD+P4XaB0=
github.com/spf13/pflag v1.0.9/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/spf13/pflag v1.0.10 h1:4EBh2KAYBwaONj6b2Ye1GiHfwjqyROoF4RwYO+vPwFk=
github.com/spf13/pflag v1.0.10/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=

View File

@@ -6,7 +6,7 @@ import (
)
// BuildVersion is the version of goctl.
const BuildVersion = "1.9.0"
const BuildVersion = "1.9.1-alpha"
var tag = map[string]int{"pre-alpha": 0, "alpha": 1, "pre-beta": 2, "beta": 3, "released": 4, "": 5}

View File

@@ -425,9 +425,12 @@ func (a *Analyzer) getType(expr *ast.BodyStmt, req bool) (spec.Type, error) {
}
if body.LBrack != nil {
if body.Star != nil {
return spec.PointerType{
return spec.ArrayType{
RawName: rawText,
Type: tp,
Value: spec.PointerType{
RawName: rawText,
Type: tp,
},
}, nil
}
return spec.ArrayType{

View File

@@ -1,6 +1,8 @@
package util
import (
"slices"
"strconv"
"strings"
"github.com/zeromicro/go-zero/tools/goctl/util/console"
@@ -54,14 +56,9 @@ func Untitle(s string) string {
}
// Index returns the index where the item equal,it will return -1 if mismatched
// Deprecated: use slices.Index instead
func Index(slice []string, item string) int {
for i := range slice {
if slice[i] == item {
return i
}
}
return -1
return slices.Index(slice, item)
}
// SafeString converts the input string into a safe naming style in golang
@@ -134,21 +131,13 @@ func FieldsAndTrimSpace(s string, f func(r rune) bool) []string {
return resp
}
//Deprecated: This function implementation is incomplete and does not properly handle exceptional input cases.
//We strongly recommend using the standard library's strconv.Unquote function instead,
//which provides robust error handling and comprehensive support for various input formats.
func Unquote(s string) string {
if len(s) == 0 {
return s
ns, err := strconv.Unquote(s)
if err != nil {
return ""
}
left := s[0]
if left == '`' || left == '"' {
s = s[1:len(s)]
}
if len(s) == 0 {
return s
}
right := s[len(s)-1]
if right == '`' || right == '"' {
s = s[0 : len(s)-1]
}
return s
return ns
}