Compare commits

...

617 Commits

Author SHA1 Message Date
Kevin Wan e31128650e Revert "🐞 fix(gen): pg gen of insert (#1591)" (#1598)
This reverts commit cc4c4928e0.
2022-03-01 20:27:59 +08:00
Kevin Wan 168740b64d chore: upgrade etcd (#1597) 2022-03-01 20:16:44 +08:00
toutou_o cc4c4928e0 🐞 fix(gen): pg gen of insert (#1591)
Co-authored-by: kurimi1 <d0n41df@gmail.com>
2022-03-01 19:53:23 +08:00
Fyn fba6543b23 fix: goctl api dart support form tag (#1596) 2022-03-01 16:17:37 +08:00
Kevin Wan 877eb6ac56 Update readme.md
add producthunt.
2022-03-01 16:11:17 +08:00
Kevin Wan 259a5a13e7 chore: fix data race (#1593) 2022-02-28 23:17:51 +08:00
Fyn cf7c7cb392 build: update goctl dependency ddl-parser to v1.0.3 (#1586)
* build: update goctl dependency ddl-parser to v1.0.3

* fix: race condition when testing logx

Resolves: #1587
2022-02-28 17:31:59 +08:00
ccx 86d01e2e99 test: add testcase for FIFO Queue in collection module (#1589)
cover the case of non-zero value for q.Header when q.Elements expands
2022-02-28 17:15:11 +08:00
Kevin Wan 7a28e19a27 Update readme-cn.md 2022-02-27 23:30:14 +08:00
Kevin Wan 900ea63d68 Update readme-cn.md
add migration notice.
2022-02-27 23:29:45 +08:00
Kevin Wan 87ab86cdd0 Update readme.md 2022-02-27 23:26:03 +08:00
Kevin Wan 0697494ffd Update readme.md
Add migrate steps.
2022-02-27 23:25:22 +08:00
anqiansong ffd69a2f5e Fix bug int overflow while build goctl on arch 386 (#1582)
Co-authored-by: anqiansong <anqiansong@bytedance.com>
2022-02-27 10:51:57 +08:00
Kevin Wan 66f10bb5e6 chore: add goctl command help (#1578) 2022-02-26 17:02:04 +08:00
Kevin Wan 8131a0e777 Update readme.md
update chat link.
2022-02-25 23:02:09 +08:00
Kevin Wan 32a557dff6 Update readme.md
add discord.
2022-02-25 23:01:15 +08:00
Fyn db949e40f1 feat: supports importValue for more path formats (#1569)
`importValueRegex` now can match more path formats

Resolves: #1568
2022-02-25 11:16:57 +08:00
Kevin Wan e0454138e0 update goctl to go 1.16 for io/fs usage (#1571)
* update goctl to go 1.16 for io/fs usage

* feat: support pg serial type for auto_increment (#1563)

* add correct example for pg's url

* 🐞 fix: merge

* 🐞 fix: pg default port

*  feat: support serial type

Co-authored-by: kurimi1 <d0n41df@gmail.com>

* chore: format code

Co-authored-by: toutou_o <33993460+kurimi1@users.noreply.github.com>
Co-authored-by: kurimi1 <d0n41df@gmail.com>
2022-02-24 13:58:53 +08:00
toutou_o 3b07ed1b97 feat: support pg serial type for auto_increment (#1563)
* add correct example for pg's url

* 🐞 fix: merge

* 🐞 fix: pg default port

*  feat: support serial type

Co-authored-by: kurimi1 <d0n41df@gmail.com>
2022-02-24 13:39:31 +08:00
anqiansong daa98f5a27 Feature: Add goctl env (#1557) 2022-02-21 10:19:33 +08:00
Kevin Wan 842656aa90 feat: log 404 requests with traceid (#1554) 2022-02-19 20:50:33 +08:00
Kevin Wan aa29036cb3 feat: support ctx in sql model generation (#1551) 2022-02-17 10:28:55 +08:00
Kevin Wan 607bae27fa feat: support ctx in sqlx/sqlc, listed in ROADMAP (#1535)
* feat: support ctx in sqlx/sqlc

* chore: update roadmap

* fix: context.Canceled should be acceptable

* use %w to wrap errors

* chore: remove unused vars
2022-02-16 19:31:43 +08:00
Kevin Wan 7c63676be4 docs: add go-zero users (#1546) 2022-02-16 12:02:52 +08:00
Kevin Wan 9e113909b3 ignore context.Canceled for redis breaker (#1545) 2022-02-15 21:31:30 +08:00
Kevin Wan bd105474ca chore: update help message (#1544) 2022-02-15 21:19:40 +08:00
Mikael a078f5d764 add the serviceAccount of deployment (#1543)
Co-authored-by: 977231903@qq.com <>
2022-02-15 20:57:14 +08:00
Kevin Wan b215fa3ee6 fix #1541 (#1542) 2022-02-15 18:40:26 +08:00
mlr3000 50b1928502 chore:use struct pointer (#1538) 2022-02-15 11:34:48 +08:00
Kevin Wan 493e3bcf4b docs: update roadmap (#1537) 2022-02-15 08:37:03 +08:00
Kevin Wan 6deb80625d fix issue of default migrate version (#1536)
* fix issue of default migrate version

* chore: update console colors
2022-02-14 23:09:32 +08:00
Kevin Wan 6ab051568c Update readme-cn.md
add go-zero users
2022-02-14 16:57:48 +08:00
Kevin Wan 2732d3cdae chore: refactor cache (#1532) 2022-02-13 18:04:31 +08:00
chenquan e8c307e4dc feat: support ctx in Cache (#1518)
* feature: support ctx in `Cache`

Signed-off-by: chenquan <chenquan.dev@foxmail.com>

* fix: `errors.Is` instead of `=`

Signed-off-by: chenquan <chenquan.dev@foxmail.com>
2022-02-13 17:28:14 +08:00
Kevin Wan 84ddc660c4 chore: goctl format issue (#1531) 2022-02-13 13:17:19 +08:00
Kevin Wan e60e707955 upgrade grpc version (#1530) 2022-02-12 23:58:41 +08:00
Kevin Wan cf4321b2d0 fix #1525 (#1527) 2022-02-11 23:04:57 +08:00
chenquan 1993faf2f8 fix: fix a typo (#1522)
Signed-off-by: chenquan <chenquan.dev@foxmail.com>
2022-02-11 21:15:45 +08:00
Kevin Wan 0ce85376bf chore: update goctl version to 1.3.2 (#1524) 2022-02-11 21:02:50 +08:00
Kevin Wan a40254156f refactor: refactor yaml unmarshaler (#1517) 2022-02-09 17:22:52 +08:00
chenquan 05cc62f5ff chore: optimize yaml unmarshaler (#1513) 2022-02-09 16:57:00 +08:00
chenquan 9c2c90e533 chore: make error clearer (#1514) 2022-02-09 14:40:05 +08:00
Kevin Wan 822ee2e1c5 feat: update go-redis to v8, support ctx in redis methods (#1507)
* feat: update go-redis to v8, support ctx in redis methods

* fix compile errors

* chore: remove unused const

* chore: add tracing log on redis
2022-02-09 11:06:06 +08:00
anqiansong 77482c8946 fixes typo (#1511)
Co-authored-by: anqiansong <anqiansong@bytedance.com>
2022-02-08 22:16:38 +08:00
Kevin Wan 7ef0ab3119 Update readme-cn.md 2022-02-08 11:47:33 +08:00
anqiansong 8bd89a297a feature: Add goctl completion (#1505)
* feature: Add `goctl completion`

* Update const

Co-authored-by: anqiansong <anqiansong@bytedance.com>
2022-02-08 10:50:21 +08:00
Kevin Wan bb75cc796e test: change fuzz tests (#1504) 2022-02-05 09:44:01 +08:00
Kevin Wan 0fdd8f54eb ci: add test for win (#1503)
* ci: add test for win

* ci: update check names

* ci: use go build instead of go test to verify win test

* fix: windows test failure

* chore: disable logs in tests
2022-02-05 00:06:23 +08:00
anqiansong b1ffc464cd fix typo: goctl protoc usage (#1502)
Co-authored-by: anqiansong <anqiansong@bytedance.com>
2022-02-03 22:13:02 +08:00
Kevin Wan 50174960e4 chore: update command comment (#1501) 2022-02-02 22:02:08 +08:00
Kevin Wan 8f46eab977 fix: goctl not compile on windows (#1500) 2022-02-01 13:58:08 +08:00
Kevin Wan ec299085f5 docs: update tal-tech to zeromico in docs (#1498) 2022-02-01 13:03:30 +08:00
Kevin Wan 7727d70634 chore: update goctl version (#1497) 2022-02-01 09:50:26 +08:00
Kevin Wan 5f9d101bc6 feat: add runtime stats monitor (#1496) 2022-02-01 01:34:25 +08:00
Kevin Wan 6c2abe7474 fix: goroutine stuck on edge case (#1495)
* fix: goroutine stuck on edge case

* refactor: simplify mapreduce implementation
2022-01-30 13:09:21 +08:00
Kevin Wan 14a902c1a7 feat: handling panic in mapreduce, panic in calling goroutine, not inside goroutines (#1490)
* feat: handle panic

* chore: update fuzz test

* chore: optimize square sum algorithm
2022-01-28 10:59:41 +08:00
Kevin Wan 5ad6a6d229 Update readme-cn.md
add slogan
2022-01-27 17:16:30 +08:00
Kevin Wan 6f4b97864a chore: improve migrate confirmation (#1488) 2022-01-27 11:30:35 +08:00
Kevin Wan 0e0abc3a95 chore: update warning message (#1487) 2022-01-26 23:47:57 +08:00
anqiansong 696fda1db4 patch: goctl migrate (#1485)
* * Add signal check
* Add deprecated pkg check

* fix typo `replacementBuilderx`

* output to console if verbose

Co-authored-by: anqiansong <anqiansong@bytedance.com>
2022-01-26 23:24:25 +08:00
Kevin Wan c1d2634427 chore: update go version for goctl (#1484) 2022-01-26 14:27:43 +08:00
Kevin Wan 4b7a680ac5 refactor: rename from tal-tech to zeromicro for goctl (#1481) 2022-01-25 23:15:07 +08:00
Kevin Wan b3e7d2901f Feature/trie ac automation (#1479)
* fix: trie ac automation issues

* fix: trie ac automation issues

* fix: trie ac automation issues

* fix: trie ac automation issues
2022-01-25 11:14:56 +08:00
anqiansong cdf7ec213c fix #1468 (#1478)
Co-authored-by: anqiansong <anqiansong@bytedance.com>
2022-01-24 22:23:20 +08:00
Kevin Wan f1102fb262 chore: optimize string search with Aho–Corasick algorithm (#1476)
* chore: optimize string search with Aho–Corasick algorithm

* chore: optimize keywords replacer

* fix: replacer bugs

* chore: reorder members
2022-01-23 23:37:02 +08:00
Keqi Huang 09d1fad6e0 Polish the words in readme.md (#1475) 2022-01-22 12:20:11 +08:00
Kevin Wan 379c65a3ef docs: add go-zero users (#1473) 2022-01-20 22:36:17 +08:00
Kevin Wan fdc7f64d6f chore: update unauthorized callback calling order (#1469)
* chore: update unauthorized callback calling order

* chore: add comments
2022-01-20 21:09:45 +08:00
anqiansong df0f8ed59e Fix/issue#1289 (#1460)
* fix #1289

* Add unit test case

* fix `jwtTransKey`

* fix `jwtTransKey`

Co-authored-by: anqiansong <anqiansong@bytedance.com>
2022-01-18 11:52:30 +08:00
anqiansong c903966fc7 patch: save missing templates to disk (#1463)
Co-authored-by: anqiansong <anqiansong@bytedance.com>
2022-01-18 10:45:05 +08:00
anqiansong e57fa8ff53 Fix/issue#1447 (#1458)
* Add data for template to render

* fix #1447

Co-authored-by: anqiansong <anqiansong@bytedance.com>
2022-01-18 10:36:38 +08:00
Kevin Wan bf2feee5b7 feat: implement console plain output for debug logs (#1456)
* feat: implement console plain output for debug logs

* chore: rename console encoding to plain

* chore: refactor names
2022-01-17 12:43:15 +08:00
Letian Jiang ce05c429fc chore: check interface satisfaction w/o allocating new variable (#1454) 2022-01-16 23:34:42 +08:00
Kevin Wan 272a3f347d chore: remove jwt deprecated (#1452) 2022-01-16 10:34:44 +08:00
shenbaise9527 13db7a1931 feat: 支持redis的LTrim方法 (#1443) 2022-01-16 10:27:34 +08:00
Kevin Wan 468c237189 chore: upgrade dependencies (#1444)
* chore: upgrade dependencies

* ci: upgrade go to 1.15
2022-01-14 11:01:02 +08:00
Kevin Wan b9b80c068b ci: add translator action (#1441) 2022-01-12 17:57:39 +08:00
anqiansong 9b592b3dee Feature rpc protoc (#1251)
* code generation by protoc

* generate pb by protoc direct

* support: grpc code generation by protoc directly

* format code

* check --go_out & --go-grpc_out

* fix typo

* Update version

* fix typo

* optimize: remove deprecated unit test

* format code

Co-authored-by: anqiansong <anqiansong@bytedance.com>
2022-01-11 20:34:25 +08:00
Kevin Wan 2203809e5e chore: fix typo (#1437) 2022-01-11 20:23:59 +08:00
Kevin Wan 8d6d37f71e remove unnecessary drain, fix data race (#1435)
* remove unnecessary drain, fix data race

* chore: fix parameter order

* refactor: rename MapVoid to ForEach in mr
2022-01-11 16:17:51 +08:00
Kevin Wan ea4f2af67f fix: mr goroutine leak on context deadline (#1433)
* fix: mr goroutine leak on context deadline

* test: update fx test check
2022-01-10 22:06:10 +08:00
Kevin Wan 53af194ef9 chore: refactor periodlimit (#1428)
* chore: refactor periodlimit

* chore: add comments
2022-01-09 16:22:34 +08:00
Kevin Wan 5e0e2d2b14 docs: add go-zero users (#1425) 2022-01-08 21:41:27 +08:00
Kevin Wan 74c99184c5 docs: add go-zero users (#1424) 2022-01-08 17:08:44 +08:00
Kevin Wan eb4b86137a fix: golint issue (#1423) 2022-01-08 16:06:56 +08:00
Kevin Wan 9c4f4f3b4e update docs (#1421) 2022-01-07 12:08:45 +08:00
spectatorMrZ 240132e7c7 Fix pg model generation without tag (#1407)
1. fix pg model struct haven't tag
2. add pg command test from datasource
2022-01-07 10:45:26 +08:00
anqiansong 9d67fc4cfb feat: Add migrate (#1419)
* Add migrate

* Remove unused module

* refactor filename

* rename refactor to migrate

Co-authored-by: anqiansong <anqiansong@bytedance.com>
2022-01-06 18:48:34 +08:00
Kevin Wan 892f93a716 docs: update install readme (#1417) 2022-01-05 12:31:49 +08:00
Kevin Wan ba6a7c9dc8 chore: refactor rest/timeouthandler (#1415) 2022-01-05 11:17:10 +08:00
Kevin Wan a91c3907a8 feat: rename module from tal-tech to zeromicro (#1413) 2022-01-04 15:51:32 +08:00
Kevin Wan e267d94ee1 chore: update go-zero to v1.2.5 (#1410) 2022-01-03 21:54:53 +08:00
anqiansong 89ce5e492b refactor file|path (#1409)
Co-authored-by: anqiansong <anqiansong@bytedance.com>
2022-01-03 21:32:40 +08:00
Kevin Wan 290de6aa96 docs: update roadmap (#1405) 2022-01-02 21:30:02 +08:00
Kevin Wan a7aeb8ac0e feat: support tls for etcd client (#1390)
* feat: support tls for etcd client

* chore: fix typo

* refactor: rename TrustedCAFile to CACertFile

* docs: add comments

* fix: missing tls registration

* feat: add InsecureSkipVerify config for testing
2022-01-02 20:23:50 +08:00
Kevin Wan a8e7fafebf refactor: optimize fx (#1404)
* refactor: optimize fx

* chore: add more comments

* ci: make test robust
2022-01-02 14:56:30 +08:00
Kevin Wan 7cc64070b1 docs: update goctl installation command (#1403) 2022-01-02 14:31:31 +08:00
Kevin Wan c19d2637ea feat: implement fx.NoneMatch, fx.First, fx.Last (#1402)
* chore: use workers from options in fx.unlimitedWalk

* feat: add fx.NoneMatch

* feat: add fx.First, fx.Last

* chore: add more comments

* docs: add mr readme
2022-01-02 13:33:15 +08:00
Kevin Wan fe1da14332 chore: simplify mapreduce (#1401) 2022-01-01 19:24:35 +08:00
anqiansong 8e9110cedf fix #1330 (#1382)
Co-authored-by: anqiansong <anqiansong@bytedance.com>
2021-12-30 20:44:04 +08:00
Kevin Wan d6ff30a570 chore: fix golint issues (#1396) 2021-12-30 17:44:15 +08:00
Kevin Wan b98d46bfd6 chore: update goctl version (#1394) 2021-12-30 15:30:16 +08:00
Kevin Wan 768936b256 ci: remove 386 binaries (#1393) 2021-12-30 15:18:24 +08:00
Kevin Wan c6eb1a9670 ci: remove windows 386 binary (#1392)
* ci: remove windows 386 binary

* chore: update go-zero

* chore: update go-zero
2021-12-30 14:47:53 +08:00
Kevin Wan e4ab518576 test: add more tests (#1391) 2021-12-30 14:21:55 +08:00
moyrne dfc67b5fac fix readme-cn (#1388) 2021-12-30 10:42:23 +08:00
Kevin Wan 62266d8f91 fix #1070 (#1389)
* fix #1070

* test: add more tests
2021-12-29 21:34:28 +08:00
anqiansong b8ea16a88e feat: Add --remote (#1387)
Co-authored-by: anqiansong <anqiansong@bytedance.com>
2021-12-29 18:16:42 +08:00
Kevin Wan 23deaf50e6 feat: support array in default and options tags (#1386)
* feat: support array in default and options tags

* feat: ignore spaces in tags

* test: add more tests
2021-12-29 17:37:36 +08:00
Kevin Wan 38a36ed8d3 docs: add go-zero users (#1381) 2021-12-28 17:12:51 +08:00
anqiansong 49bab23c54 fix #1376 (#1380)
* fix #1376

* fix #1376

Co-authored-by: anqiansong <anqiansong@bytedance.com>
2021-12-28 16:40:26 +08:00
Leizhengzi 78ba00d3a7 fix: command system info missing go version (#1377) 2021-12-27 22:05:27 +08:00
Kevin Wan 787b046a70 docs: update slack invitation link (#1378) 2021-12-27 16:52:08 +08:00
Kevin Wan f827a7b985 chore: update goctl version to 1.2.4 for release tools/goctl/v1.2.4 (#1372) 2021-12-27 10:57:55 +08:00
行者 f5f2097d14 Updated MySQL生成表结构体遇到关键字db部分保持原字段名定义 (#1369) 2021-12-26 21:56:04 +08:00
Kevin Wan cfcfb87fd4 ci: add release action to auto build binaries (#1371) 2021-12-26 21:44:33 +08:00
Kevin Wan 1d223fc114 docs: update goctl markdown (#1370) 2021-12-26 20:32:31 +08:00
Kevin Wan c0647f0719 feat: support context in MapReduce (#1368) 2021-12-25 20:42:52 +08:00
Kevin Wan 8745ed9c61 chore: add 1s for tolerance in redislock (#1367) 2021-12-25 19:44:27 +08:00
种豆得豆 836726e710 fix redis try-lock bug (#1366)
#issue_id: 1338

Co-authored-by: zhangwei <>
2021-12-25 19:20:53 +08:00
JiangYiJun a67c118dcf go-zero tools ,fix a func,api new can not choose style (#1356) 2021-12-23 10:28:46 +08:00
Kevin Wan cd289465fd chore: coding style and comments (#1361)
* chore: coding style and comments

* chore: optimize `ParseJsonBody` (#1353)

* chore: optimize `ParseJsonBody`

* chore: optimize `ParseJsonBody`

* fix: fix a test

* chore: optimize `ParseJsonBody`

* fix a test

* chore: add comment

* chore: refactor

Co-authored-by: chenquan <chenquan.dev@foxmail.com>
2021-12-22 21:43:37 +08:00
chenquan 263e426ae1 chore: optimize ParseJsonBody (#1353)
* chore: optimize `ParseJsonBody`

* chore: optimize `ParseJsonBody`

* fix: fix a test

* chore: optimize `ParseJsonBody`

* fix a test

* chore: add comment
2021-12-22 20:24:55 +08:00
charliecen d5e493383a chose: cancel the assignment and judge later (#1359)
Co-authored-by: charliecen <chq@abierr.com>
2021-12-22 20:05:35 +08:00
Kevin Wan 6f1d27354a chore: put error message in error.log for verbose mode (#1355) 2021-12-21 11:36:01 +08:00
Kevin Wan 26101732d2 test: add more tests (#1352) 2021-12-20 22:42:36 +08:00
Kevin Wan 71d40e0c08 Revert "排除客户端中断导致的503错误 (#1343)" (#1351)
This reverts commit 2cdf5e7395.
2021-12-20 20:34:43 +08:00
Kevin Wan 4ba2ff7cdd feat: treat client closed requests as code 499 (#1350)
* feat: treat client closed requests as code 499

* chore: add comments
2021-12-20 19:43:38 +08:00
vic 2cdf5e7395 排除客户端中断导致的503错误 (#1343) 2021-12-20 19:43:13 +08:00
Kevin Wan 8315a55b3f Update FUNDING.yml
enable sponsorship.
2021-12-20 15:27:05 +08:00
Kevin Wan d1c2a31af7 chore: add tests & refactor (#1346)
* chore: add tests & refactor

* chore: refactor
2021-12-18 23:11:38 +08:00
MarkJoyMa 3e6c217408 Feature: support adding custom cache to mongoc and sqlc (#1313)
* merge

* Feature: support adding custom cache to mongoc and sqlc
2021-12-18 22:45:07 +08:00
Kevin Wan b299f350be chore: add comments (#1345) 2021-12-18 22:39:14 +08:00
Kevin Wan 8fd16c17dc chore: update goctl version to 1.2.5 (#1337) 2021-12-16 00:21:54 +08:00
anqiansong 5979b2aa0f Update template (#1335)
Co-authored-by: anqiansong <anqiansong@bytedance.com>
2021-12-15 23:24:32 +08:00
anqiansong 0b17e0e5d9 Feat goctl bug (#1332)
* Support goctl bug

* fix typo

* format code

Co-authored-by: anqiansong <anqiansong@bytedance.com>
2021-12-15 22:43:58 +08:00
Kevin Wan 3d8ad5e4f6 feat: tidy mod, update go-zero to latest (#1334) 2021-12-15 22:34:58 +08:00
Kevin Wan ff1752dd39 feat: tidy mod, update go-zero to latest (#1333) 2021-12-15 22:23:06 +08:00
Kevin Wan 1becaeb7be chore: refactor (#1331) 2021-12-15 20:44:23 +08:00
yangkequn 171afaadb9 Update types.go (#1314) 2021-12-15 20:16:17 +08:00
Kevin Wan 776e6e647d feat: tidy mod, add go.mod for goctl (#1328) 2021-12-15 19:44:49 +08:00
Kevin Wan 4ccdf4ec72 chore: format code (#1327) 2021-12-15 13:43:05 +08:00
CrazyZard a7bd993c0c commit missing method for redis (#1325)
* commit `decr ` `decrby` `lindex` missing method for redis

* fix(store_test):TestRedis_DecrBy

* add unit tests for redis commands. And put the functions in alphabetical order

* put the functions in alphabetical order

* add `lindex` unit test

* sort func
2021-12-15 13:15:39 +08:00
Kevin Wan a290ff4486 docs: add go-zero users (#1323) 2021-12-14 13:37:49 +08:00
Kevin Wan 490ef13822 style: format code (#1322) 2021-12-14 11:29:44 +08:00
anqiansong 1b14de2ff9 fix: #1318 (#1321)
* fix #1318

* fix #1318

* remove never used code

* fix unit tes

Co-authored-by: anqiansong <anqiansong@bytedance.com>
2021-12-13 22:55:11 +08:00
Kevin Wan 914692cc82 fix #1309 (#1317) 2021-12-13 11:58:58 +08:00
anqiansong 07191dc430 fix #1305 (#1307)
Co-authored-by: anqiansong <anqiansong@bytedance.com>
2021-12-07 22:24:18 +08:00
BYT0723 af3fb2b04d fix: go issue 16206 (#1298) 2021-12-07 15:52:37 +08:00
Kevin Wan 0240fa131a chore: rename service context from ctx to svcCtx (#1299) 2021-12-05 22:10:47 +08:00
Kevin Wan e96577dd38 docs: add go-zero users (#1294) 2021-12-03 22:32:35 +08:00
Kevin Wan 403dd7367a fix #1288 (#1292)
* fix #1288

* chore: make wrapup & shutdown callbacks run simulatenously
2021-12-02 22:41:57 +08:00
Kevin Wan 8086ad120b Revert "feat: reduce dependencies of framework by add go.mod in goctl (#1290)" (#1291)
This reverts commit 87a445689c.
2021-12-02 19:40:23 +08:00
Kevin Wan 87a445689c feat: reduce dependencies of framework by add go.mod in goctl (#1290) 2021-12-02 16:57:07 +08:00
Kevin Wan b6bda54870 chore: update cli version (#1287) 2021-12-01 23:33:23 +08:00
Kevin Wan 9d528dddd6 feat: support third party orm to interact with go-zero (#1286)
* fixes #987

* chore: fix test failure

* chore: add comments

* feat: support third party orm to interact with go-zero

* chore: refactor
2021-12-01 20:22:15 +08:00
Kevin Wan 543d590710 fixes #987 (#1283)
* fixes #987

* chore: fix test failure

* chore: add comments
2021-12-01 17:45:48 +08:00
anqiansong f1d70eb6b2 Feature api root path (#1261) 2021-12-01 10:09:07 +08:00
Kevin Wan d828c3f37e feat: add etcd resolver scheme, fix discov minor issue (#1281) 2021-11-28 20:08:18 +08:00
Kevin Wan 038491b7bc chore: cleanup zRPC retry code (#1280) 2021-11-27 18:39:52 +08:00
chenquan cf683411ee feature(retry): Delete retry mechanism (#1279) 2021-11-27 11:32:33 +08:00
Kevin Wan de5ed6a677 feat: support %w in logx.Errorf (#1278) 2021-11-26 15:57:23 +08:00
Kevin Wan 3dda557410 chore: only allow cors middleware to change headers (#1276) 2021-11-26 14:14:06 +08:00
Kevin Wan c800f6f723 chore: avoid superfluous WriteHeader call errors (#1275) 2021-11-26 11:09:57 +08:00
Kevin Wan 0395ba1816 feat: add rest.WithCustomCors to let caller customize the response (#1274) 2021-11-25 23:03:37 +08:00
Kevin Wan 86f9f63b46 Cli (#1272)
* Fix issue 1260 (#1262)

* Fix #1238 (#1266)

* docs: update readme to use goctl@cli (#1255)

* chore: update goctl version

* style: coding style

* docs: update readme to use goctl@cli

* fix #1238

* format code

* format code

Co-authored-by: Kevin Wan <wanjunfeng@gmail.com>
Co-authored-by: anqiansong <anqiansong@bytedance.com>

Co-authored-by: anqiansong <anqiansong@gmail.com>
Co-authored-by: anqiansong <anqiansong@bytedance.com>
2021-11-25 11:08:49 +08:00
Kevin Wan a7a6753118 fixes #1257 (#1271)
* fixes #1257

* chore: format code

* test: add more tests
2021-11-25 10:26:16 +08:00
Kevin Wan 2e80d12d6a docs: update readme to use goctl@cli (#1255)
* chore: update goctl version

* style: coding style

* docs: update readme to use goctl@cli
2021-11-17 21:10:45 +08:00
Kevin Wan 417a96cbf2 chore: update goctl version (#1250)
* chore: update goctl version

* style: coding style
2021-11-16 21:57:55 +08:00
Kevin Wan 2d4c29ea7c Revert "Revert "feat: enable retry for zrpc (#1237)"" (#1246) 2021-11-16 10:29:31 +08:00
Kevin Wan 67db40ed4f Revert "feat: enable retry for zrpc (#1237)" (#1245)
This reverts commit 09eb53f308.
2021-11-15 23:30:31 +08:00
FabioCircle 11c485a5ed Duplicate temporary variable (#1244)
Co-authored-by: fabiowzhang <fabiowzhang@wesure.cn>
2021-11-15 23:14:54 +08:00
anqiansong b0573af9a9 Update template (#1243) 2021-11-15 21:02:11 +08:00
Kevin Wan 09eb53f308 feat: enable retry for zrpc (#1237) 2021-11-14 22:33:01 +08:00
Kevin Wan 11f85d1b80 chore: remove conf.CheckedDuration (#1235) 2021-11-13 23:34:30 +08:00
anqiansong 0cb86c6990 reset link goctl (#1232) 2021-11-13 18:39:07 +08:00
Kevin Wan 57d2f22c24 feat: disable grpc retry, enable it in v1.2.4 (#1233) 2021-11-13 15:38:43 +08:00
Kevin Wan fa0c364982 fixes #1169 (#1229) 2021-11-12 14:05:28 +08:00
Kevin Wan a6c8113419 chore: refactor, better goctl message (#1228) 2021-11-11 22:58:33 +08:00
Kevin Wan 4f5c30e083 chore: remove unused const (#1224) 2021-11-10 21:45:42 +08:00
Kevin Wan 9d0b51fa26 fixes #1222 (#1223) 2021-11-10 21:25:51 +08:00
Kevin Wan ba5f8045a2 Update FUNDING.yml
disable sponsor button.
2021-11-10 21:22:34 +08:00
an 3a510a9138 chore: redislock use stringx.randn replace randomStr func (#1220) 2021-11-10 21:14:21 +08:00
Kevin Wan d3bfa16813 feat: exit with non-zero code on errors (#1218)
* feat: exit with non-zero code on errors

* chore: use const for code
2021-11-09 22:42:44 +08:00
Kevin Wan 28409791fa feat: support CORS, better implementation (#1217)
* feat: support CORS, better implementation

* chore: refine code
2021-11-09 20:35:57 +08:00
Kevin Wan c1abe87953 Create FUNDING.yml
add sponsor button
2021-11-09 14:27:36 +08:00
Kevin Wan f8367856e8 chore: refine code (#1215) 2021-11-08 23:12:13 +08:00
Kevin Wan a72b0a689b docs: add go-zero users (#1214) 2021-11-08 16:13:24 +08:00
anqiansong 69a4d213a3 Fix issue 1205 (#1211)
* fix #1205

* move builder into stores

* remove xrom

* Remove unused code

* Remove unused code

* refactor builderx to builder

Co-authored-by: anqiansong <anqiansong@bytedance.com>
2021-11-07 22:44:37 +08:00
Kevin Wan c28e01fed3 feat: support CORS by using rest.WithCors(...) (#1212)
* feat: support CORS by using rest.WithCors(...)

* chore: add comments

* refactor: lowercase unexported methods

* ci: fix lint errors
2021-11-07 22:42:40 +08:00
Kevin Wan e8efcef108 update dependencies. (#1210)
* chore: update dependencies

* chore: update dependencies

* chore: update dependencies

* chore: update dependencies

* chore: fix test failure
2021-11-07 16:38:20 +08:00
Kevin Wan d011316997 test: add more tests (#1209) 2021-11-07 11:41:24 +08:00
Kevin Wan 4d22b0c497 feat: ignore rest.WithPrefix on empty prefix (#1208) 2021-11-06 21:31:35 +08:00
晨曦中 539215d7df goctl docker command add -version (#1206)
* feature(优化): 优化goctl

goctl docker 命令新增version参数,指定builder golang 版本

* feature(优化): 优化goctl

goctl docker 命令新增version参数,指定builder golang 版本
2021-11-06 21:28:32 +08:00
Kevin Wan 3ede597a15 feat: support customizing timeout for specific route (#1203)
* feat: support customizing timeout for specific route

* test: add more tests
2021-11-03 22:20:32 +08:00
anqiansong 01786c5e63 Generate route with prefix (#1200)
* Generate route with prefix

* Update api convert

* Remove TrimSpace

* Update path join

* Format code

* Format code

Co-authored-by: anqiansong <anqiansong@bytedance.com>
2021-11-03 20:57:03 +08:00
yedf2 6aba5f74fc feat: add NewSessionFromTx to interact with other orm (#1202)
Co-authored-by: yedongfu <dongfuye@163.com>
2021-11-03 20:56:02 +08:00
Kevin Wan 3c894a3fb7 feat: simplify the grpc tls authentication (#1199) 2021-11-02 20:42:22 +08:00
Kevin Wan 1ece3a498f feat: use WithBlock() by default, NonBlock can be set in config or WithNonBlock() (#1198) 2021-11-02 19:02:02 +08:00
Kevin Wan b76c7ae55d chore: remove semicolon for routes of services in api files (#1195) 2021-11-01 20:37:05 +08:00
Kevin Wan 91b10bd3b9 feat: add rest.WithPrefix to support route prefix (#1194) 2021-11-01 20:15:10 +08:00
Kevin Wan 7e3fe77e7b chore: update goctl version to 1.2.3, prepare for release (#1193)
* feat: slow threshold customizable in rest

* chore: update goctl version to 1.2.3, prepare for release
2021-11-01 18:26:08 +08:00
Kevin Wan ba43214dae feat: slow threshold customizable in zrpc (#1191)
* feat: slow threshold customizable in rest

* feat: slow threshold customizable in rest

* feat: slow threshold customizable in rest

* feat: slow threshold customizable in zrpc
2021-11-01 15:04:38 +08:00
Kevin Wan ebc90720ea feat: slow threshold customizable in rest (#1189)
* feat: slow threshold customizable in rest

* feat: slow threshold customizable in rest
2021-11-01 14:48:26 +08:00
Kevin Wan 785d100be9 feat: slow threshold customizable in sqlx (#1188) 2021-11-01 08:37:44 +08:00
Kevin Wan f13e6f1149 feat: slow threshold customizable in redis (#1187) 2021-11-01 08:20:35 +08:00
Kevin Wan 8be0f77d96 feat: slow threshold customizable in mongo (#1186) 2021-11-01 07:12:53 +08:00
Kevin Wan 429f85a9de feat: slow threshold customizable in redis (#1185)
* feat: slow threshold customizable in redis

* chore: improve config robustness
2021-10-31 22:14:20 +08:00
Kevin Wan b4d1c6da2c docs: update roadmap (#1184) 2021-10-31 21:00:34 +08:00
Kevin Wan 3c1cfd4c1e feat: support multiple trace agents (#1183)
* feat: support multiple trace agents

* feat: support multiple trace agents, let later calls run if error happens

* test: add more tests
2021-10-31 19:58:01 +08:00
Kevin Wan a71a210704 feat: let different services start prometheus on demand (#1182) 2021-10-31 18:54:13 +08:00
Kevin Wan 769d06c8ab refactor: simplify tls config in rest (#1181) 2021-10-31 14:10:47 +08:00
Howie cd1f8da13f [update] add plugin config (#1180)
Signed-off-by: lihaowei <haoweili35@gmail.com>
2021-10-31 12:56:25 +08:00
Kevin Wan 8230474667 test: add more tests (#1179) 2021-10-31 11:33:13 +08:00
Kevin Wan 27f553bf84 docs: update roadmap (#1178) 2021-10-31 11:13:45 +08:00
Kevin Wan d48bff8c8b docs: add go-zero users (#1176) 2021-10-31 10:02:46 +08:00
Kevin Wan 59b9687f31 feat: support auth account for etcd (#1174) 2021-10-31 09:05:38 +08:00
Kevin Wan c1a8ccda11 feat: support ssl on zrpc, simplify the config (#1175) 2021-10-30 23:15:39 +08:00
workman-Lu 9df6786b09 support RpcClient Vertify With Unilateralism and Mutual (#647)
Co-authored-by: Kevin Wan <wanjunfeng@gmail.com>
2021-10-30 22:07:15 +08:00
anqiansong bef5bd4e4f fix the package name of grpc client (#1170)
* fix the package name of grpc client

* Remove k8s/utils

Co-authored-by: anqiansong <anqiansong@bytedance.com>
2021-10-30 21:35:05 +08:00
Kevin Wan 68acfb1891 docs: add go-zero users (#1172) 2021-10-29 21:39:28 +08:00
zeromake 9fd3f752d1 fix(goctl): repeat creation protoc-gen-goctl symlink (#1162) 2021-10-29 09:56:51 +08:00
anqiansong 9c48e9ceab Feature add template version (#1152) 2021-10-29 09:55:41 +08:00
Kevin Wan bd26783b33 test: add more tests (#1166)
* chore: reverse the order of stopping services

* chore: reverse the order of stopping services

* test: add more tests
2021-10-28 10:04:59 +08:00
Kevin Wan eda8230521 chore: reorg imports, format code, make MaxRetires default to 0 (#1165)
* chore: reverse the order of stopping services

* chore: reverse the order of stopping services

* chore: reorg imports, format code

* chore: format code, and refactor

* feat: change MaxRetries default to 0, disable retry
2021-10-27 20:57:18 +08:00
chenquan 462ddbb145 Add grpc retry (#1160)
* Add grpc retry

* Update grpc retry

* Add tests

* Fix a bug

* Add api && some tests

* Add comment

* Add double check

* Add server retry quota

* Update optimize code

* Fix bug

* Update optimize code

* Update optimize code

* Fix bug
2021-10-27 19:46:07 +08:00
Kevin Wan 496a2f341e test: add more tests (#1163)
* chore: reverse the order of stopping services

* chore: reverse the order of stopping services

* test: add more tests
2021-10-25 21:10:08 +08:00
Kevin Wan 7109d6d635 chore: reverse the order of stopping services (#1159)
* chore: reverse the order of stopping services

* chore: reverse the order of stopping services
2021-10-24 12:01:17 +08:00
Kevin Wan ca72241fa3 docs: update qr code (#1158) 2021-10-23 22:12:50 +08:00
Kevin Wan a6bdffd225 test: add more tests (#1154) 2021-10-21 21:16:18 +08:00
Kevin Wan 5636bf4955 test: add more tests (#1150) 2021-10-20 17:50:01 +08:00
anqiansong a944a7fd7e Mark deprecated syntax (#1148) 2021-10-20 10:58:25 +08:00
Kevin Wan a40fa405e4 test: add more tests (#1149) 2021-10-19 23:48:25 +08:00
Kevin Wan eab77e21dd test: add more tests (#1147)
* test: add more tests

* test: add more tests
2021-10-19 22:37:56 +08:00
Kevin Wan d41163f5c1 docs: add go-zero users (#1141) 2021-10-18 18:38:01 +08:00
Kevin Wan 265b1f2459 test: add more tests (#1138) 2021-10-15 16:27:30 +08:00
Kevin Wan c92ea59228 test: add more tests (#1137) 2021-10-15 16:07:38 +08:00
Kevin Wan afddfea093 docs: add go-zero users (#1135) 2021-10-14 12:50:25 +08:00
Kevin Wan fa4dc151ca test: add more tests (#1134) 2021-10-13 22:42:54 +08:00
anqiansong 44202acb18 Fix issue #1127 (#1131)
* fix #1127

* fix #1127

* fixed unit test

* add go keyword converter

Co-authored-by: anqiansong <anqiansong@bytedance.com>
2021-10-13 20:48:47 +08:00
Kevin Wan cf00786209 docs: add go-zero users (#1130) 2021-10-12 22:34:13 +08:00
Kevin Wan 6a8638fc85 chore: refine rpc template in goctl (#1129) 2021-10-12 22:13:14 +08:00
Mikael 837a9ffa03 go-zero/core/hash/hash_test.go 增加测试 TestMd5Hex (#1128) 2021-10-12 21:57:07 +08:00
chenquan d28cfe5f20 Add opts ...grpc.CallOption in grpc client (#1122)
* Add `opts ...grpc.CallOption` in grpc client

* Update format

* Update format

* Add import package

* Update format
2021-10-12 21:52:50 +08:00
chenquan 022c100dc9 Add request method in http log (#1120)
* Add request method in http log

* Update log format
2021-10-12 21:50:30 +08:00
Kevin Wan 426b09c356 update goctl version to 1.2.2 (#1125) 2021-10-12 14:00:35 +08:00
Kevin Wan 40dc21e4cf add cncf landscape (#1123) 2021-10-11 15:31:29 +08:00
Kevin Wan 9b114e3251 test: add more tests (#1119) 2021-10-08 17:31:37 +08:00
Kevin Wan 4c6234f108 add more tests (#1115)
* add more tests

* fix lint errors
2021-10-04 21:08:44 +08:00
Kevin Wan 3cdfcb05f1 add more tests (#1114) 2021-10-04 20:02:25 +08:00
Kevin Wan 9f5bfa0088 add more tests (#1113) 2021-10-04 14:14:25 +08:00
Kevin Wan 2d42c8fa00 test: add more tests (#1112) 2021-10-03 21:30:48 +08:00
Kevin Wan 10e7922597 feat: opentelemetry integration, removed self designed tracing (#1111)
* feat: opentelemetry integration, removed self designed tracing

* feat: support zipkin on opentelemetry integration

* feat: support zipkin on opentelemetry integration, enable it in conf

* style: format code

* fix: support logx without exporter configured

* fix: check return values

* refactor: simplify code

* refactor: simplify opentelemetry integration

* ci: fix staticcheck errors
2021-10-03 20:53:50 +08:00
Kevin Wan 6e34b55ba7 docs: update roadmap (#1110) 2021-10-02 18:16:59 +08:00
Kevin Wan ed15ca04f4 fix: opentelemetry traceid not correct (#1108) 2021-10-01 22:44:37 +08:00
小小小下 295ec27e1b feat: reflection grpc service (#1107)
* feat: reflection grpc service

* feat: reflection grpc service
2021-10-01 22:02:04 +08:00
Kevin Wan d1e702e8a3 test: add more tests (#1106)
* style: format code

* test: add more tests

* fix: staticcheck errors
2021-10-01 10:03:56 +08:00
chenquan d1bfb5ef61 Fix the resources variable not reset after the resource manager is closed (#1105)
* Fix the resource variable not reset after the resource manager is closed

* Format code
2021-09-30 16:55:36 +08:00
Kevin Wan e43357164c chore: replace redis.NewRedis with redis.New (#1103) 2021-09-29 23:01:10 +08:00
Kevin Wan cd21c9fa74 chore: mark redis.NewRedis as Deprecated, use redis.New instead. (#1100)
* chore: mark redis.NewRedis as Deprecated

* chore: mark redis.NewRedis as Deprecated
2021-09-29 22:07:05 +08:00
Kevin Wan cdd2fcbbc9 update grpc package (#1099) 2021-09-29 19:30:06 +08:00
理工男 8d2db09d45 Update Makefile (#1098) 2021-09-29 18:04:08 +08:00
Kevin Wan 65905b914d ci: add reviewdog (#1096) 2021-09-29 13:09:20 +08:00
z-micro 80e3407be1 fix bug: generating dart code error (#1090) 2021-09-28 09:01:27 +08:00
Kevin Wan 657d27213a docs: update roadmap (#1094) 2021-09-26 17:41:19 +08:00
Kevin Wan 8ac18a9422 docs: update roadmap (#1093) 2021-09-26 17:38:49 +08:00
Kevin Wan d3ae9cfd49 ci: accurate error reporting on lint check (#1089) 2021-09-25 23:25:40 +08:00
Kevin Wan d7f42161fd update zero-doc links in readme (#1088) 2021-09-24 11:31:00 +08:00
Kevin Wan e03229cabe docs: change organization from tal-tech to zeromicro in readme (#1087) 2021-09-23 20:24:46 +08:00
Kevin Wan 8403ed16ae ci: add Lint check on commits (#1086)
* ci: add Lint check on commits

* ci: fix Lint script error

* test: fix go vet errors

* test: fix go vet errors, remove gofumpt to check go vet

* test: fix go vet errors, try gofumpt

* test: fix go vet errors, try gofumpt, round 1

* test: fix go vet errors, try gofumpt, round 2

* ci: fix Lint errors
2021-09-23 19:57:05 +08:00
Kevin Wan d87d203c3b Revert "chore: run unit test with go 1.14 (#1084)" (#1085)
This reverts commit 3ae6a882a7.
2021-09-23 15:34:10 +08:00
Kevin Wan 3ae6a882a7 chore: run unit test with go 1.14 (#1084) 2021-09-23 15:30:40 +08:00
Amor 41c980f00c update goctl api (#1052)
* update goctl api

* add LoadTemplate

* update new api template

* update
2021-09-23 14:31:11 +08:00
shenbaise9527 f34d81ca2c chore: when run goctl-rpc, the order of proto message aliases should be (#1078)
fixed

Co-authored-by: steven <steven.zhou@1quant.com>
2021-09-23 14:24:21 +08:00
NevS 004ee488a6 fix AtomicError panic when Set nil (#1049) (#1050) 2021-09-23 14:23:02 +08:00
Kevin Wan 2e12cd2c99 coding style (#1083) 2021-09-23 11:19:57 +08:00
neosu 2695c30886 we can use otel.ErrorHandlerFunc instead of custom struct when we update OpenTelemetry to 1.0.0 (#1081) 2021-09-23 10:42:45 +08:00
Kevin Wan c74fb988e0 update go.mod (#1079) 2021-09-22 21:54:54 +08:00
anqiansong e8a340c1c0 Create a symbol link file named protoc-gen-goctl from goctl (#1076) 2021-09-21 23:13:31 +08:00
Kevin Wan 06e114e5a3 update OpenTelemetry to 1.0.0 (#1075) 2021-09-21 23:11:36 +08:00
Kevin Wan 74ad681a66 update issue templates (#1074) 2021-09-21 14:13:28 +08:00
Kevin Wan e7bbc09093 Update issue templates 2021-09-21 13:58:21 +08:00
Kevin Wan 1eb1450c43 downgrade golang-jwt to support go 1.14 (#1073) 2021-09-21 13:42:45 +08:00
anqiansong 9a724fe907 Add MustTempDir (#1069) 2021-09-21 10:13:43 +08:00
Kevin Wan 30e49f2939 fix jwt security issue by using golang-jwt package (#1066) 2021-09-19 22:33:35 +08:00
Kevin Wan a5407479a6 upgrade grpc version & replace github.com/golang/protobuf/protoc-gen-go with google.golang.org/protobuf (#1065) 2021-09-19 21:37:55 +08:00
anqiansong 7fb5bab26b fix #1058 (#1064) 2021-09-19 08:53:41 +08:00
Kevin Wan 27249e021f add repo moving notice (#1062) 2021-09-18 22:24:45 +08:00
Kevin Wan d809795fec add go-zero users (#1061) 2021-09-18 22:01:15 +08:00
Kevin Wan c9db9588b7 chore: fix comment issues (#1056) 2021-09-17 17:11:03 +08:00
Kevin Wan 872c50b71a chore: make comment accurate (#1055) 2021-09-17 11:11:42 +08:00
Kevin Wan 7c83155e4f mention cncf landscape (#1054) 2021-09-17 10:44:42 +08:00
Kevin Wan 358d86b8ae add go-zero users (#1051) 2021-09-16 15:06:42 +08:00
Kevin Wan f4bb9f5635 fix test error on ubuntu (#1048) 2021-09-15 13:31:20 +08:00
Jerry Liang 5c6a3132eb fix typo parse.go error message (#1041) 2021-09-14 10:26:44 +08:00
Kevin Wan 2bd95aa007 update goctl version to 1.2.1 (#1042) 2021-09-14 08:32:56 +08:00
Kevin Wan e8376936d5 remove goctl config command (#1035) 2021-09-13 14:04:06 +08:00
Kevin Wan 71c0288023 fix symlink issue on windows for goctl (#1034)
* fix symlink issue on windows for goctl

* move readlink into separate file
2021-09-13 11:49:07 +08:00
Kevin Wan 9e2f07a842 update k8s.io/client-go etc to use go 1.15 (#1031) 2021-09-13 10:45:34 +08:00
Kevin Wan 24fd34413f fix golint issues (#1027) 2021-09-12 16:15:42 +08:00
Kevin Wan 3f47251892 fix proc.Done not found in windows (#1026) 2021-09-12 15:41:33 +08:00
Kevin Wan 0b6bc69afa reorg imports, format code (#1024) 2021-09-11 21:28:47 +08:00
anqiansong 5b9bdc8d02 Merge pull request #1023 from anqiansong/1014-rollback
revert changes
2021-09-11 20:58:42 +08:00
anqiansong ded22e296e revert changes 2021-09-11 20:57:58 +08:00
anqiansong f0ed2370a3 fix #1014 (#1018)
* fix #1014

* remove unused code

* * optimize generate pb.go on Windows
* format code
* optimize console.go

* version rollback

Co-authored-by: anqiansong <anqiansong@bytedance.com>
2021-09-11 12:48:32 +08:00
Kevin Wan 6bf6cfdd01 add go-zero users (#1022) 2021-09-10 22:03:24 +08:00
Kevin Wan 5cc9eb0de4 rename sharedcalls to singleflight (#1017) 2021-09-09 18:06:27 +08:00
Kevin Wan f070d447ef refactor for better error reporting on sql error (#1016)
* refactor for better error reporting on sql error

* fix errors

* add docs
2021-09-09 15:43:33 +08:00
Kevin Wan f6d9e19ecb expose sql.DB to let orm operate on it (#1015)
* expose sql.DB to let orm operate on it

* add missing RawDB methods

* add NewSqlConnFromDB for cooperate with dtm
2021-09-09 11:40:28 +08:00
Kevin Wan 56807aabf6 fix golint issues, update codecov settings. (#1011)
* update codecov settings

* fix golint issues
2021-09-07 22:33:02 +08:00
Kevin Wan 861dcf2f36 update codecov settings (#1010) 2021-09-07 18:09:53 +08:00
Kevin Wan c837dc21bb refactoring tracing interceptors. (#1009)
* refactor tracing interceptors

* add stream tracing interceptor
2021-09-07 17:58:22 +08:00
Kevin Wan 96a35ecf1a fix #1006 (#1008)
* use sdktrace instead of trace for opentelemetry to avoid conflicts

* disable opentelemetry for right now

* use os.ModeSymlink instead of fs.ModeSymlink for backward compatibility
2021-09-07 15:38:40 +08:00
Kevin Wan bdec5f2349 use sdktrace instead of trace for opentelemetry to avoid conflicts (#1005)
* use sdktrace instead of trace for opentelemetry to avoid conflicts

* disable opentelemetry for right now
2021-09-07 12:02:41 +08:00
shenbaise9527 bc92b57bdb api文件中使用group时生成的handler和logic的包名应该为group的名字 (#545)
* api文件中使用group时生成的handler和logic的包名应该为group的名字

* Update genhandlers.go

fix errors.

Co-authored-by: Kevin Wan <wanjunfeng@gmail.com>
2021-09-07 11:03:04 +08:00
neosu d8905b9e9e add api template file (#1003) 2021-09-07 10:16:10 +08:00
neosu dec6309c55 add opentelemetry test (#1002) 2021-09-07 09:26:45 +08:00
Kevin Wan 10805577f5 reorg imports, format code (#1000) 2021-09-06 14:56:46 +08:00
SunJun a4d8286e36 开启otel后,tracelog自动获取otle的traceId和spanId (#946)
* 开启otel后,tracelog自动获取otle的traceId和spanId

* 去除opentelemetry判断

* 通过 IsRecording 判断span是否活跃
2021-09-06 14:31:54 +08:00
anqiansong 84d2b64e7c optimize unit test (#999) 2021-09-06 14:20:21 +08:00
Kevin Wan 6476da4a18 rest log with context (#998) 2021-09-05 22:58:42 +08:00
Kevin Wan 79eab0ea2f refactor to shorter config name (#997) 2021-09-05 22:43:15 +08:00
Allen Liu 3b683fd498 feat: change logger to traceLogger for getting traceId when recovering (#374) 2021-09-05 22:40:10 +08:00
toven tang d179b342b2 修复使用 postgres 数据库时,位置参数重复,导致参数与值不对应的问题。 (#960)
* 修复使用 postgres 数据库时,位置参数重复,导致参数与值不对应的问题。

* 修复使用 postgres 数据库时,位置参数重复,导致参数与值不对应的问题。

Co-authored-by: toven <toven@advan.onaliyun.com>
2021-09-05 22:27:59 +08:00
Kevin Wan 58874779e7 move opentelemetry into trace package, and refactoring (#996)
* move opentelemetry into trace package, and refactoring

* rename rewritten package names
2021-09-05 22:18:35 +08:00
anqiansong 8829c31c0d Feature goctl error wrap (#995)
* Add `Wrap` in file errorx.go

* Wrap error with `GoctlError`

* format code

* Refactor package `env` to `version`

* Refactor package `IsVersionGatherThan`

* fix typo

Co-authored-by: anqiansong <anqiansong@bytedance.com>
2021-09-05 21:57:44 +08:00
Kevin Wan b42f3fa047 disable codecov github checks (#993) 2021-09-04 13:32:23 +08:00
Kevin Wan 9bdadf2381 fix golint issues (#992) 2021-09-04 12:16:30 +08:00
Kevin Wan 20f665ede8 implement k8s service discovery (#988)
* implement k8s service discovery

* simplify code

* use default namespace if not provided

* disable codecov bot comment

* ignore adhoc dir

* simplify building target in NewClient

* reformat code

* Fix filepath (#990)

* format code, and reorg imports (#991)

* add more unit test

Co-authored-by: anqiansong <anqiansong@gmail.com>
2021-09-04 10:27:08 +08:00
Kevin Wan 0325d8e92d format code, and reorg imports (#991) 2021-09-04 10:08:49 +08:00
anqiansong 2125977281 Fix filepath (#990) 2021-09-04 08:15:22 +08:00
Vee Zhang c26c187e11 remote handler blank line when .HasRequest is false (#986) 2021-09-03 07:25:04 +08:00
Kevin Wan 4ef1859f0b use codecov action v1 (#985) 2021-09-02 19:07:30 +08:00
Kevin Wan 407a6cbf9c format coding style (#983) 2021-09-01 19:52:56 +08:00
Vee Zhang 76fc1ef460 httpx.Error response without body (#982)
* httpx.Error support response without body

* fix doc
2021-09-01 19:33:33 +08:00
miaogaolin 423955c55f format code (#979) 2021-08-31 17:15:36 +08:00
Kevin Wan db95b3f0e3 configurable for load and stat statistics logs (#980) 2021-08-31 17:14:31 +08:00
Kevin Wan 4bee60eb7f add go-zero users (#978) 2021-08-31 16:08:33 +08:00
Kevin Wan 7618139dad refactor (#977) 2021-08-31 12:04:09 +08:00
Kevin Wan 6fd08027ff update go version to 1.14 for github workflow (#976) 2021-08-31 11:46:22 +08:00
chenquan b9e268aae8 Update Codecov action (#974) 2021-08-31 11:25:58 +08:00
Kevin Wan 4c1bb1148b fix #971 (#972) 2021-08-29 11:38:34 +08:00
Kevin Wan 50a6bbe6b9 format coding style (#970) 2021-08-28 12:11:11 +08:00
chenquan dfb3cb510a Fix context error in grpc (#962)
* Fix context error in rpc

* Add a test case

* Optimize judgment conditions

* Add customized breaker errors for the client and server

* Update method signature

* Delete customized breaker errors

* Delete the wrong test case
2021-08-28 12:07:14 +08:00
Kevin Wan 519db812b4 format coding style (#969) 2021-08-27 23:09:47 +08:00
anqiansong 3203f8e06b Fix issues (#965) 2021-08-27 15:05:54 +08:00
chenquan b71ac2042a Add a test case for database code generation tool (#961) 2021-08-27 06:50:09 +08:00
Kevin Wan d0f9e57022 fix #957 (#959) 2021-08-26 16:47:28 +08:00
SunJun aa68210cde 修复stream拦截器tracer名问题 (#944) 2021-08-24 15:17:53 +08:00
zhoushuguang 280e837c9e rest otel support (#943) 2021-08-24 10:04:12 +08:00
Kevin Wan f669e1226c fix #556 (#938) 2021-08-22 23:36:35 +08:00
Kevin Wan cd15c19250 fix lint errors (#937) 2021-08-22 10:24:32 +08:00
neosu 5b35fa17de add the opentelemetry tracing (#908)
* add the opentelemetry tracing

* fix the error sampler config

* 添加stream的链路跟踪

* fix the error field name
2021-08-22 10:03:56 +08:00
Kevin Wan 9672298fa8 make sure setting code happen before callback in rest (#936) 2021-08-22 09:27:20 +08:00
Kevin Wan bf3ce16823 fix #820 (#934) 2021-08-19 22:48:21 +08:00
anqiansong 189721da16 Fix issues (#931)
* fix #929

* fix #925

* add test case

* update model README

* fix #929

* fix #929

* fix #929

* refactor dir

* Adding todo comments

Co-authored-by: anqiansong <anqiansong@xiaoheiban.cn>
2021-08-19 22:47:45 +08:00
Kevin Wan a523ab1f93 update slack invite url (#933) 2021-08-19 22:25:41 +08:00
Kevin Wan 7ea8b636d9 add go-zero users (#928) 2021-08-19 10:39:47 +08:00
anqiansong b2fea65faa Optimize model naming (#910)
* add unit test

* fix #907

* format code

* format code

* format code

Co-authored-by: anqiansong <anqiansong@xiaoheiban.cn>
2021-08-18 17:09:34 +08:00
anqiansong a1fe8bf6cd fix missing updateMethodTemplateFile (#924) 2021-08-18 12:23:43 +08:00
Kevin Wan 67ee9e4391 add unit test (#921) 2021-08-17 17:09:26 +08:00
Kevin Wan 9c1ee50497 refactor (#920) 2021-08-17 10:24:12 +08:00
chenquan 7c842f22d0 Add traceId to the response headers (#919)
* Add traceId to the request headers

* Add test cases

* Update refactor code
2021-08-17 10:12:08 +08:00
anqiansong 14ec29991c fix #915 (#917) 2021-08-16 15:07:11 +08:00
Kevin Wan c7f5aad83a add stringx.FirstN with ellipsis (#916) 2021-08-16 12:08:33 +08:00
lucaq e77747cff8 redis.go,type StringCmd = red.StringCmd (#790)
* Add Sinter,Sinterstore; Modify TestRedis_Set

* type StringCmd

* redis.go,type StringCmd = red.StringCmd

Co-authored-by: lucq <lucq@toopsoon.com>
2021-08-16 09:52:16 +08:00
Kevin Wan f2612db4b1 add stringx.FirstN (#914) 2021-08-15 23:02:48 +08:00
Kevin Wan a21ff71373 fix #889 (#912) 2021-08-15 15:33:45 +08:00
Kevin Wan fc04ad7854 export pathvar for user-defined routers (#911)
* refactor

* export pathvar for user-defined routers
2021-08-14 22:57:28 +08:00
Kevin Wan fbf2eebc42 add Errorv/Infov/Slowv (#909) 2021-08-13 18:28:39 +08:00
anqiansong dc43430812 optimize grpc generation env check (#900)
* optimize grpc generation env check

* optimize grpc generation env check

* format code

* fix postgresql data type convert error

Co-authored-by: anqiansong <anqiansong@xiaoheiban.cn>
2021-08-13 11:47:42 +08:00
Kevin Wan c6642bc2e6 add workflow for closing inactive issues (#906)
* add workflow for closing inactive issues

* add workflow for closing inactive issues, add newline
2021-08-12 15:28:29 +08:00
Phibe bdca24dd3b Update readme for better description. (#904) 2021-08-12 14:59:17 +08:00
Kevin Wan 00c5734021 format coding style (#905) 2021-08-12 14:58:37 +08:00
lovelly 33f87cf1f0 带下划线的项目,配置文件名字错误。 (#733) 2021-08-12 14:55:39 +08:00
Kevin Wan 69935c1ba3 refactor goctl, fix golint issues (#903) 2021-08-11 18:08:01 +08:00
Kevin Wan 1fb356f328 refactor goctl (#902) 2021-08-11 18:04:42 +08:00
市民233 0b0406f41a fix: 解决golint 部分警告 (#897)
* feat:  解决goreportcard的警报
ps: warning: if block ends with a return statement, so drop this else and outdent its block (golint)

* feat: 优化golint警告,将processFieldPrimitiveWithJsonNumber 改成 processFieldPrimitiveWithJSONNumber
unmarshaler.go:248:23: method processFieldPrimitiveWithJsonNumber should be processFieldPrimitiveWithJSONNumber

* feat: 添加 WithCanonicalKeyFunc 注释

* feat: 添加DisableStat的注释

* feat: 添加 RegisterGoctlHome 注释

* feat: 添加 PostgreSqlJoin 注释

* feat: 解决goline警告should not use basic type string as key in context.WithValue问题

* feat: 解决golint警告信息: should not use basic type string as key in context.WithValue

* fix: 定义自定义字段类型,导致go test fail 问题

* update: 恢复原有测试用例

* fix golint warning
2021-08-11 17:57:20 +08:00
_ksco cc264dcf55 refactor (#878)
* refactor(tools ): refactor cod

Improve code readability and performance

* fix(tools ): fix len bug

Co-authored-by: ksco <hyang@33.cn>
2021-08-11 17:33:18 +08:00
Kevin Wan e024aebb66 fix golint issues (#899) 2021-08-11 11:38:55 +08:00
Phibe f204729482 remove unnecessary chars. (#898) 2021-08-11 10:51:28 +08:00
Phibe d20cf56a69 simplify type definition in readme (#896)
* better text rendering

* simplify type definition in readme
2021-08-10 18:02:11 +08:00
Kevin Wan 54d57c7d4b refactor rest code (#895) 2021-08-10 17:59:33 +08:00
voidint 28a7c9d38f fix http header binding failure bug #885 (#887) 2021-08-10 17:38:03 +08:00
Kevin Wan 872e75e10d add logx.DisableStat() to disable stat logs (#893)
* add logx.DisableStat() to disable stat logs

* refactor logx code
2021-08-10 16:55:38 +08:00
Phibe af1730079e better text rendering (#892) 2021-08-10 15:13:09 +08:00
Kevin Wan 04521e2d24 format code (#888) 2021-08-09 23:03:08 +08:00
Kevin Wan 02adcccbf4 format code (#884) 2021-08-09 18:11:08 +08:00
anqiansong a74aaf1823 optimize mongo generation without cache (fix #881) (#882)
* fix #881

* fix #881

Co-authored-by: anqiansong <anqiansong@xiaoheiban.cn>
2021-08-09 11:33:01 +08:00
Kevin Wan 1eb2089c69 add go-zero users (#883) 2021-08-09 08:37:43 +08:00
Kevin Wan f7f3730e1a update goctl version to 1.1.10 (#874) 2021-08-04 19:29:40 +08:00
Kevin Wan 0ee7654407 fix #792 (#873) 2021-08-04 18:45:05 +08:00
neosu 16cc990fdd fix context missing (#872)
Co-authored-by: suzhenpeng <suzhenpeng@ecoplants.tech>
2021-08-04 17:46:51 +08:00
neosu 00061c2e5b add goctl rpc template home flag (#871)
Co-authored-by: suzhenpeng <suzhenpeng@ecoplants.tech>
2021-08-04 15:54:43 +08:00
Kevin Wan 6793f7a1de fix bug that proc.SetTimeToForceQuit not working in windows (#869) 2021-08-04 11:31:33 +08:00
anqiansong c8428a7f65 fix issue #861 (#862)
* fix issue #861

* fix issue #861

Co-authored-by: anqiansong <anqiansong@xiaoheiban.cn>
2021-08-01 23:00:57 +08:00
toutou_o a5e1d0d0dc add correct example for pg's url (#857) 2021-07-30 13:58:44 +08:00
anqiansong 8270c7deed optimize typo (#855) 2021-07-29 21:53:16 +08:00
anqiansong 9f4a882a1b fix issue #831 (#850)
* fix issue #831

* fix typo

Co-authored-by: anqiansong <anqiansong@xiaoheiban.cn>
2021-07-28 16:32:15 +08:00
anqiansong cb7b7cb72e fix issue #836 (#849)
Co-authored-by: anqiansong <anqiansong@xiaoheiban.cn>
2021-07-27 22:13:17 +08:00
Kevin Wan 603c93aa4a upgrade grpc package (#845) 2021-07-24 22:29:02 +08:00
masonchen2014 cb8d9d413a simplify timeoutinterceptor (#840)
Co-authored-by: chenmusheng <chenmusheng@laoyuegou.com>
2021-07-24 21:51:46 +08:00
Kevin Wan ff7443c6a7 fix #796 (#844) 2021-07-24 12:58:14 +08:00
fangjianwei b812e74d6f Fixed http listener error. (#843) 2021-07-24 12:57:56 +08:00
anqiansong 089cdaa75f Feature model postgresql (#842)
* Support postgresql generate

* Update template Var

* Support to generate postgresql model

* Support to generate postgresql model

* Update template

Co-authored-by: anqiansong <anqiansong@xiaoheiban.cn>
2021-07-23 11:45:15 +08:00
fangjianwei 476026e393 Added database prefix of cache key. (#835) 2021-07-22 11:29:09 +08:00
Kevin Wan 75952308f9 remove faq for old versions (#828) 2021-07-19 22:54:21 +08:00
Kevin Wan df0550d6dc add go-zero users, update faq (#827) 2021-07-19 17:13:33 +08:00
neosu e481b63b21 Fix the error stream method name (#826) 2021-07-18 22:05:28 +08:00
Kevin Wan e47079f0f4 go format with extra rules (#821) 2021-07-17 20:51:23 +08:00
anqiansong 9b2a279948 Fix issues: #725, #740 (#813)
* Fix issues: #725, #740

* Update filed sort

Co-authored-by: anqiansong <anqiansong@xiaoheiban.cn>
2021-07-16 22:55:39 +08:00
anqiansong db87fd3239 To generate grpc stream, fix issue #616 (#815)
Co-authored-by: anqiansong <anqiansong@xiaoheiban.cn>
2021-07-16 22:54:07 +08:00
aaffo 598fda0c97 optimized (#819) 2021-07-15 23:50:44 +08:00
Chen Quan b0e335e7b0 Fix rpc generator bug (#799)
* Fix rpc自动生成generate bug

* Delete mock
2021-07-10 13:12:52 +08:00
anqiansong efdf475da4 Add --go_opt flag to adapt to the version after 1.4.0 of protoc-gen-go (#767)
Co-authored-by: anqiansong <anqiansong@xiaoheiban.cn>
2021-07-08 10:11:11 +08:00
Chen Quan 22a1315136 [WIP]Add parse headers info (#805)
* Add parse headers info

* Update parse headers info
2021-07-07 23:20:09 +08:00
Kevin Wan 5b22823018 fix bug that empty query in transaction (#801) 2021-06-29 23:18:32 +08:00
Kevin Wan 9ccb997ed8 refactor mapping (#782) 2021-06-23 14:57:37 +08:00
skykiss 01c92a6bc5 fix: Fix problems with non support for multidimensional arrays and basic type pointer arrays (#778)
Co-authored-by: shaoqian <shaoqian.zhang@appshahe.com>
2021-06-23 10:58:01 +08:00
lucaq c9a2a60e28 Add Sinter,Sinterstore & Modify TestRedis_Set (#779)
* Add Sinter,Sinterstore; Modify TestRedis_Set

* Update redis_test.go

fix test failure

Co-authored-by: lucq <lucq@toopsoon.com>
Co-authored-by: Kevin Wan <wanjunfeng@gmail.com>
2021-06-23 10:46:16 +08:00
Kevin Wan b0739d63c0 update readme images (#776) 2021-06-21 16:45:44 +08:00
Kevin Wan c22f84cb5f update image rendering in readme (#775) 2021-06-21 16:34:14 +08:00
Kevin Wan 60450bab02 disable load & stat logs for goctl (#773) 2021-06-21 14:25:33 +08:00
Kevin Wan 3e8cec5c78 upgrade grpc & etcd dependencies (#771) 2021-06-21 09:05:20 +08:00
Kevin Wan 74ee163761 fix bug that etcd stream cancelled without re-watch (#770) 2021-06-17 18:46:16 +08:00
anqiansong ea4f680052 Fix issue #747 (#765)
Co-authored-by: anqiansong <anqiansong@xiaoheiban.cn>
2021-06-15 18:54:41 +08:00
heyanfu 58cdba2c5d remove useless annotation (#761) 2021-06-14 16:00:47 +08:00
Kevin Wan a2fbc14c70 add roadmap (#764) 2021-06-13 11:59:30 +08:00
Kevin Wan 158df8c270 fix broken link (#763) 2021-06-13 11:35:19 +08:00
Kevin Wan 30ec236a87 add contributing guid (#762) 2021-06-13 11:33:29 +08:00
Kevin Wan ac3653b3f9 add code of conduct (#760) 2021-06-12 22:17:00 +08:00
Kevin Wan 8520db4fd9 refactor fx (#759)
* refactor fx

* refactor fx, format code
2021-06-10 19:57:36 +08:00
Chen Quan 14141fed62 Add some stream features (#712)
* Add some stream features

* Update empty

* Fix initialization loop

* Delete ForeachOrdered && Fix FindFirst

* Add test case && Delete redundant code

* Update test case

* Delete SplitSteam

* Delete redundant code
2021-06-10 18:20:40 +08:00
Kevin Wan 5d86cc2f20 add go-zero users (#756) 2021-06-07 14:08:54 +08:00
Kevin Wan 8a6e4b7580 add go-zero users (#751) 2021-06-03 15:07:21 +08:00
anqiansong 453f949638 replace cache key with colon (#746)
Co-authored-by: anqiansong <anqiansong@xiaoheiban.cn>
2021-06-02 10:37:49 +08:00
Kevin Wan 75a330184d add go-zero users (#739) 2021-05-29 23:58:43 +08:00
kingxt 546fcd8bab fix #736 (#738)
* optimize performance

* rename

* rename

* revert
2021-05-29 23:01:02 +08:00
Xavier Cheng 3022f93b6d Fix a typo (#729)
alread -> already
2021-05-28 23:39:07 +08:00
Kevin Wan 8ffc392c66 add go-zero users, update slack invite link (#728) 2021-05-28 14:32:47 +08:00
Kevin Wan ae7d85dadf add go-zero users (#726) 2021-05-28 10:50:33 +08:00
Kevin Wan e89268ac37 add go-zero users. (#723)
* add go-zero users

* add go-zero users
2021-05-27 22:53:55 +08:00
Kevin Wan aaa3623404 optimize nested conditional (#709) 2021-05-22 23:18:38 +08:00
heyanfu 8998f16054 optimize nested conditional (#708)
Co-authored-by: heyanfu <heyanfu@kingsoft.com>
2021-05-22 22:56:06 +08:00
anqiansong 94417be018 Add document & comment for spec (#703)
* Add document & comment for spec

* remove duplicate field

* use alias
2021-05-21 10:40:59 +08:00
Kevin Wan f300408fc0 fix golint issues, and optimize code (#705) 2021-05-21 10:38:38 +08:00
Kevin Wan aaa39e17a3 print entire sql statements in logx if necessary (#704) 2021-05-20 16:14:44 +08:00
Bo-Yi Wu 73906f996d chore(format): change by gofumpt tool (#697)
Signed-off-by: Bo-Yi Wu <appleboy.tw@gmail.com>
2021-05-18 14:43:09 +08:00
Kevin Wan 73417f54db update goctl version to 1.1.8 (#696) 2021-05-17 15:13:48 +08:00
Kevin Wan 491213afb8 fix #683 (#690)
* fix #683

* fix errors
2021-05-15 15:37:24 +08:00
Julian-Chu edf743cd72 fix invalid link (#689) 2021-05-15 15:37:10 +08:00
Kevin Wan 78a88be787 add go-zero users (#688) 2021-05-14 22:52:19 +08:00
anqiansong 9f6a574f97 resolve #610 (#684) 2021-05-13 18:42:05 +08:00
anqiansong ea01cc78f0 Optimize model nl (#686) 2021-05-12 12:28:23 +08:00
Kevin Wan a87978568a fix #676 (#682) 2021-05-10 23:10:57 +08:00
Kevin Wan 14cecb9b31 update readme for documents links (#681) 2021-05-10 22:22:32 +08:00
_ksco 0ce54100a4 fix zh_cn document url (#678) 2021-05-10 22:18:33 +08:00
anqiansong d28ac35ff7 fix issue: https://github.com/zeromicro/goctl-swagger/issues/6 (#680) 2021-05-10 19:57:12 +08:00
heyanfu a5962f677f fix some typo (#677) 2021-05-10 00:09:00 +08:00
Kevin Wan 8478474f7f update readme (#673) 2021-05-08 21:55:14 +08:00
anqiansong df5ae9507f replace antlr module (#672)
* replace antlr module

* refactor version of antlr
2021-05-08 21:35:27 +08:00
noel faf4d7e3bb modify the order of PrometheusHandler (#670)
* modify the order of PrometheusHandler

* modify the order of PrometheusHandler
2021-05-08 17:11:16 +08:00
anqiansong f64fe5eb5e fix antlr mod (#669) 2021-05-08 00:03:01 +08:00
heyanfu 97d889103a fix some typo (#667) 2021-05-04 21:33:08 +08:00
Kevin Wan 9a44310d00 update wechat qrcode (#665) 2021-05-02 15:06:16 +08:00
Kevin Wan 06eeef2cf3 disable prometheus if not configured (#663) 2021-04-30 15:09:49 +08:00
Kevin Wan 9adc7d4cb9 fix comment function names (#649) 2021-04-23 11:56:41 +08:00
Kevin Wan 006f78c3d5 add go-zero users (#643) 2021-04-21 10:24:15 +08:00
Kevin Wan 64a8e65f4a update readme (#640) 2021-04-20 23:57:57 +08:00
anqiansong 8fd1e76d29 update readme (#638) 2021-04-19 14:37:47 +08:00
heyanfu 0466af5e49 optimize code (#637) 2021-04-18 22:49:03 +08:00
heyanfu 7405d7f506 spelling mistakes (#634) 2021-04-17 20:15:19 +08:00
Bo-Yi Wu afd9ff889e chore: update code format. (#628) 2021-04-15 19:49:17 +08:00
另维64 7e087de6e6 doc: fix spell mistake (#627) 2021-04-14 17:58:27 +08:00
Kevin Wan 5aded99df5 update go-zero users (#623) 2021-04-13 14:38:40 +08:00
Kevin Wan 08fb980ad2 add syncx.Guard func (#620) 2021-04-13 00:04:19 +08:00
Kevin Wan b94d7aa532 update readme (#617) 2021-04-10 19:19:05 +08:00
Kevin Wan ee630b8b57 add code coverage (#615)
* add code coverage

* simplify redis code
2021-04-09 22:40:43 +08:00
Kevin Wan bd82b7d8de add FAQs in readme (#612) 2021-04-09 18:59:17 +08:00
Kevin Wan 3d729c77a6 update go-zero users (#611) 2021-04-09 14:16:31 +08:00
Kevin Wan e944b59bb3 update go-zero users (#609)
* add go-zero users registry notes

* update go-zero users

* fix typo
2021-04-09 10:43:47 +08:00
Kevin Wan 54b5e3f4b2 add go-zero users registry notes (#608) 2021-04-08 22:44:41 +08:00
Kevin Wan b913229028 add go-zero users (#607) 2021-04-08 22:30:45 +08:00
Kevin Wan 9963ffb1c1 simplify redis tls implementation (#606) 2021-04-08 18:19:36 +08:00
r00mz 8cb6490724 redis增加tls支持 (#595)
* redis连接增加支持tls选项

* 优化redis tls config 写法

* redis增加tls支持

* 增加redis tls测试用例,但redis tls local server不支持,测试用例全部NotNil

Co-authored-by: liuyi <liuyi@fangyb.com>
Co-authored-by: yi.liu <yi.liu@xshoppy.com>
2021-04-07 20:44:16 +08:00
Kevin Wan 05e37ee20f refactor - remove ShrinkDeadline, it's the same as context.WithTimeout (#599) 2021-04-05 22:59:24 +08:00
zjbztianya d88da4cc88 Replace contextx.ShrinkDeadline with context.WithTimeout (#598) 2021-04-05 21:20:35 +08:00
Oraoto 425430f67c Simplify contextx.ShrinkDeadline (#596) 2021-04-03 21:25:32 +08:00
Zcc、 4e0d91f6c0 fix (#592)
Co-authored-by: zhoudeyu <zhoudeyu@xiaoheiban.cn>
2021-04-01 18:42:50 +08:00
Kevin Wan 8584351b6d update regression test comment (#590) 2021-03-30 21:23:07 +08:00
Kevin Wan b19c5223a9 update regression test comment (#589) 2021-03-30 20:53:35 +08:00
bittoy 99a2d95433 remove rt mode log (#587) 2021-03-30 20:45:55 +08:00
Ted Chen 9db222bf5b fix a simple typo (#588) 2021-03-29 23:35:49 +08:00
Kevin Wan ac648d08cb fix typo (#586) 2021-03-28 22:10:07 +08:00
Kevin Wan 6df7fa619c fix typo (#585) 2021-03-28 21:20:04 +08:00
Kevin Wan bbb4ce586f fix golint issues (#584) 2021-03-28 20:42:11 +08:00
anqiansong 888551627c optimize code (#579)
* optimize code

* optimize returns & unit test
2021-03-27 17:33:17 +08:00
Kevin Wan bd623aaac3 support postgresql (#583)
support postgresql
2021-03-27 17:14:32 +08:00
Kevin Wan 9e6c2ba2c0 avoid goroutine leak after timeout (#575) 2021-03-21 16:54:34 +08:00
Kevin Wan c0db8d017d gofmt logs (#574) 2021-03-20 16:40:09 +08:00
TonyWang 52b4f8ca91 add timezone and timeformat (#572)
* add timezone and timeformat

* rm time zone and keep time format

Co-authored-by: Tony Wang <tonywang.data@gmail.com>
2021-03-20 16:36:19 +08:00
Kevin Wan 4884a7b3c6 zrpc timeout & unit tests (#573)
* zrpc timeout & unit tests
2021-03-19 18:41:26 +08:00
Kevin Wan 3c6951577d make hijack more stable (#565) 2021-03-15 20:11:09 +08:00
Kevin Wan fcd15c9b17 refactor, and add comments to describe graceful shutdown (#564) 2021-03-14 08:51:10 +08:00
Kevin Wan 155e6061cb fix golint issues (#561) 2021-03-12 23:08:04 +08:00
anqiansong dda7666097 Feature mongo gen (#546)
* add feature: mongo code generation

* upgrade version

* update doc

* format code

* update update.tpl of mysql
2021-03-12 17:49:28 +08:00
hanhotfox c954568b61 Hdel support for multiple key deletion (#542)
* Hdel support for multiple key deletion

* Hdel field -> fields

Co-authored-by: duanyan <duanyan@xiaoheiban.cn>
2021-03-12 17:47:21 +08:00
Kevin Wan c2acc43a52 add important notes in readme (#560) 2021-03-12 16:48:25 +08:00
Kevin Wan 1a1a6f5239 add http hijack methods (#555) 2021-03-09 21:30:45 +08:00
anqiansong 60c7edf8f8 fix spelling (#551) 2021-03-08 18:23:12 +08:00
Kevin Wan 7ad86a52f3 update doc link (#552) 2021-03-08 17:56:03 +08:00
kingxt 1e4e5a02b2 rename (#543) 2021-03-04 17:13:07 +08:00
Kevin Wan 39540e21d2 fix golint issues (#540) 2021-03-03 17:16:09 +08:00
hexiaoen b321622c95 暴露redis EvalSha 以及ScriptLoad接口 (#538)
Co-authored-by: shanehe <shanehe@zego.im>
2021-03-03 17:09:27 +08:00
kingxt a25cba5380 fix collection breaker (#537)
* fix collection breaker

* optimized

* optimized

* optimized
2021-03-03 10:44:29 +08:00
Kevin Wan f01472c9ea fix golint issues (#535) 2021-03-02 11:02:57 +08:00
Kevin Wan af531cf264 fix golint issues (#533) 2021-03-02 00:11:18 +08:00
Kevin Wan c4b2cddef7 fix golint issues (#532) 2021-03-02 00:04:12 +08:00
Kevin Wan 51de0d0620 fix golint issues in zrpc (#531) 2021-03-01 23:52:44 +08:00
anqiansong dd393351cc patch 1.1.5 (#530) 2021-03-01 21:14:07 +08:00
Kevin Wan 655ae8034c fix golint issues in rest (#529) 2021-03-01 19:15:35 +08:00
anqiansong d894b88c3e feature 1.1.5 (#411) 2021-03-01 17:29:07 +08:00
Kevin Wan 791e76bcf0 fix broken build (#528) 2021-02-28 23:53:58 +08:00
Kevin Wan c566b5ff82 fix golint issues in core/stores (#527) 2021-02-28 23:02:49 +08:00
Kevin Wan 490241d639 fix golint issues in core/syncx (#526) 2021-02-28 16:16:22 +08:00
Kevin Wan f02711a9cb golint core/discov (#525) 2021-02-27 23:56:18 +08:00
Kevin Wan ad32f9de23 fix golint issues in core/threading (#524) 2021-02-26 16:27:04 +08:00
Kevin Wan f309e9f80c fix golint issues in core/utils (#520)
* fix golint issues in core/utils

* fix golint issues in core/trace

* fix golint issues in core/trace
2021-02-26 16:20:47 +08:00
hao 2087ac1e89 修正http转发头字段值错误 (#521) 2021-02-26 16:17:30 +08:00
kingxt e6ef1fca12 Code optimized (#523)
* optimized markdown generator

* optimized markdown generator

* optimized markdown generator

* add more comment

* add comment

* add comment

* add comments for rpc tool

* add comments for model tool

* add comments for model tool

* add comments for model tool

* add comments for config tool

* add comments for config tool

* add comments

* add comments

* add comments

* add comments

* add comment

* remove rpc main head info

* add comment

* optimized

Co-authored-by: anqiansong <anqiansong@xiaoheiban.cn>
2021-02-26 16:11:47 +08:00
Kevin Wan ef146cf5ba fix golint issues in core/timex (#517) 2021-02-24 16:27:11 +08:00
Kevin Wan 04b0f26182 fix golint issues in core/stringx (#516) 2021-02-24 16:09:07 +08:00
Kevin Wan acdaee0fb6 fix golint issues in core/stat (#515)
* change to use ServiceGroup to make it more clear

* fix golint issues in core/stat
2021-02-24 15:13:56 +08:00
Kevin Wan 56ad4776d4 fix misspelling (#513) 2021-02-23 13:53:19 +08:00
Kevin Wan 904d168f18 fix golint issues in core/service (#512) 2021-02-22 22:43:24 +08:00
Kevin Wan 4bd4981bfb fix golint issues in core/search (#509) 2021-02-22 18:58:03 +08:00
Kevin Wan 90562df826 fix golint issues in core/rescue (#508) 2021-02-22 16:47:02 +08:00
Kevin Wan 497762ab47 fix golint issues in core/queue (#507) 2021-02-22 16:38:42 +08:00
Kevin Wan 6e4c98e52d fix golint issues in core/prometheus (#506) 2021-02-22 14:55:04 +08:00
Kevin Wan b4bb5c0323 fix broken links in readme (#505) 2021-02-22 14:13:33 +08:00
Kevin Wan a58fac9000 fix golint issues in core/prof (#503) 2021-02-22 10:20:54 +08:00
Kevin Wan d84e3d4b53 fix golint issues in core/proc (#502) 2021-02-22 10:07:39 +08:00
Kevin Wan 221f923fae fix golint issues in core/netx (#501) 2021-02-22 09:56:56 +08:00
Kevin Wan bbb9126302 fix golint issues in core/mr (#500) 2021-02-22 09:47:06 +08:00
Kevin Wan e7c9ef16fe fix golint issues in core/metric (#499) 2021-02-21 21:18:07 +08:00
Kevin Wan 8872d7cbd3 fix golint issues in core/mathx (#498) 2021-02-21 20:47:01 +08:00
Kevin Wan 334ee4213f fix golint issues in core/mapping (#497) 2021-02-20 23:18:22 +08:00
Kevin Wan 226513ed60 fix golint issues in core/logx (#496) 2021-02-20 22:45:58 +08:00
Kevin Wan dac00d10c1 fix golint issues in core/load (#495) 2021-02-20 22:02:09 +08:00
Kevin Wan 84d2b6f8f5 fix golint issues in core/limit (#494) 2021-02-20 21:55:54 +08:00
kingxt f98c9246b2 Code optimized (#493) 2021-02-20 19:50:03 +08:00
Kevin Wan 059027bc9d fix golint issues in core/lang (#492) 2021-02-20 18:21:23 +08:00
Kevin Wan af68caeaf6 fix golint issues in core/jsonx (#491) 2021-02-20 16:59:31 +08:00
Zcc、 fdeacfc89f add redis bitmap command (#490)
Co-authored-by: zhoudeyu <zhoudeyu@xiaoheiban.cn>
2021-02-20 16:26:49 +08:00
Kevin Wan 5b33dd59d9 fix golint issues in core/jsontype (#489) 2021-02-20 15:07:49 +08:00
Kevin Wan 1f92bfde6a fix golint issues in core/iox (#488) 2021-02-19 18:40:26 +08:00
Kevin Wan 0c094cb2d7 fix golint issues in core/hash (#487) 2021-02-19 18:14:34 +08:00
Kevin Wan f238290dd3 fix golint issues in core/fx (#486) 2021-02-19 17:49:39 +08:00
Kevin Wan c376ffc351 fix golint issues in core/filex (#485) 2021-02-19 14:30:38 +08:00
Kevin Wan 802549ac7c fix golint issues in core/executors (#484) 2021-02-19 12:03:05 +08:00
Zcc、 72580dee38 redis add bitcount (#483)
Co-authored-by: zhoudeyu <zhoudeyu@xiaoheiban.cn>
2021-02-19 11:41:01 +08:00
Kevin Wan 086113c843 prevent negative timeout settings (#482)
* prevent negative timeout settings

* fix misleading comment
2021-02-19 10:44:39 +08:00
HarryWang29 d239952d2d zrpc client support block (#412) 2021-02-19 10:24:03 +08:00
Kevin Wan 7472d1e70b fix golint issues in core/errorx (#480) 2021-02-19 10:08:38 +08:00
Kevin Wan 2446d8a668 fix golint issues in core/discov (#479) 2021-02-18 22:56:35 +08:00
Kevin Wan f6894448bd fix golint issues in core/contextx (#477) 2021-02-18 18:00:20 +08:00
Kevin Wan 425be6b4a1 fix golint issues in core/conf (#476) 2021-02-18 15:56:19 +08:00
Kevin Wan 457048bfac fix golint issues in core/collection, refine cache interface (#475) 2021-02-18 15:49:56 +08:00
kingxt f14ab70035 Code optimized (#474)
* optimized markdown generator

* optimized markdown generator

* optimized markdown generator

* optimized markdown generator
2021-02-18 15:08:20 +08:00
Kevin Wan 8f1c88e07d fix golint issues in core/codec (#473) 2021-02-18 14:11:09 +08:00
Kevin Wan 9602494454 fix issue #469 (#471) 2021-02-17 21:42:22 +08:00
Kevin Wan 38abfb80ed fix gocyclo warnings (#468) 2021-02-17 14:01:05 +08:00
Kevin Wan 87938bcc09 fix golint issues in core/cmdline (#467) 2021-02-17 11:08:30 +08:00
Kevin Wan 8ebf6750b9 fix golint issues in core/breaker (#466) 2021-02-17 10:45:55 +08:00
Kevin Wan 6f92daae12 fix golint issues in core/bloom (#465) 2021-02-17 09:58:35 +08:00
Kevin Wan 80e1c85b50 add more tests for service (#463) 2021-02-11 23:48:19 +08:00
Kevin Wan 395a1db22f add more tests for rest (#462) 2021-02-10 23:08:48 +08:00
bittoy 28009c4224 Update serviceconf.go (#460)
add regression environment config
2021-02-09 15:35:50 +08:00
Kevin Wan 211f3050e9 fix golint issues (#459) 2021-02-09 14:10:38 +08:00
Kevin Wan 03b5fd4a10 fix golint issues (#458) 2021-02-09 14:03:19 +08:00
Kevin Wan 5e969cbef0 fix golint issues, else blocks (#457) 2021-02-09 13:50:21 +08:00
Kevin Wan 42883d0899 fix golint issues, redis methods (#455) 2021-02-09 10:58:11 +08:00
Kevin Wan 06f6dc9937 fix golint issues, package comments (#454) 2021-02-08 22:31:52 +08:00
Kevin Wan 1789b12db2 move examples into zero-examples (#453)
* move examples to zero-examples

* tidy go.mod

* add examples refer in readme
2021-02-08 22:23:36 +08:00
Kevin Wan c7f3e6119d remove images, use zero-doc instead (#452) 2021-02-08 21:57:40 +08:00
Kevin Wan 54414db91d fix golint issues, exported doc (#451) 2021-02-08 21:31:56 +08:00
Kevin Wan 9b0625bb83 fix golint issues (#450) 2021-02-08 17:08:40 +08:00
Kevin Wan 0dda05fd57 add api doc (#449) 2021-02-08 11:10:55 +08:00
Kevin Wan 5b79ba2618 add discov tests (#448) 2021-02-07 20:24:47 +08:00
Kevin Wan 22a1fa649e remove etcd facade, added for testing purpose (#447) 2021-02-07 19:07:15 +08:00
Kevin Wan 745e76c335 add more tests for stores (#446) 2021-02-07 17:22:47 +08:00
Kevin Wan 852891dbd8 add more tests for stores (#445) 2021-02-07 15:27:01 +08:00
Kevin Wan 316195e912 add more tests for mongoc (#443) 2021-02-07 14:41:00 +08:00
Kevin Wan 8e889d694d add more tests for sqlx (#442)
* add more tests for sqlx

* add more tests for sqlx
2021-02-07 11:54:41 +08:00
Kevin Wan ec6132b754 add more tests for zrpc (#441) 2021-02-06 12:25:45 +08:00
Kevin Wan c282bb1d86 add more tests for sqlx (#440) 2021-02-05 22:53:21 +08:00
Kevin Wan d04b54243d add more tests for proc (#439) 2021-02-05 15:11:27 +08:00
Kevin Wan b88ba14597 fixes issue #425 (#438) 2021-02-05 13:32:56 +08:00
理工男 7b3c3de35e ring struct add lock (#434)
Co-authored-by: liuhuan210 <liuhuan210@jd.com>
2021-02-03 21:41:10 +08:00
Kevin Wan abab7c2852 Update readme.md 2021-02-03 15:43:35 +08:00
Kevin Wan 30f5ab0b99 update readme for broken links (#432) 2021-02-03 12:02:22 +08:00
foyon 8b273a075c Support redis command Rpop (#431)
* ss

* ss

* add go-zero:stores:redis-command:Rpop and redis_test

* Delete 1.go

* support redis command Rpop

Co-authored-by: fanhongyi <fanhongyi@tal.com>
2021-02-03 10:19:42 +08:00
Liang Zheng 76026fc211 fix readme.md error (#429)
Signed-off-by: Liang Zheng <microyahoo@163.com>
2021-02-03 10:18:28 +08:00
Hkesd 04284e31cd support hscan in redis (#428) 2021-02-02 17:02:18 +08:00
Kevin Wan c3b9c3c5ab use english readme as default, because of github ranking (#427) 2021-02-02 16:58:45 +08:00
FengZhang a8b550e7ef Modify the http content-length max range : 30MB --> 32MB (#424)
Because we are programmer :)
2021-01-30 18:49:33 +08:00
FengZhang cbfbebed00 modify the maximum content-length to 30MB (#413) 2021-01-29 22:14:48 +08:00
kingxt 2b07f22672 optimize code (#417)
* optimize code

* optimize code

* optimize code

* optimize code
2021-01-26 17:37:22 +08:00
Kevin Wan a784982030 support zunionstore in redis (#410) 2021-01-21 21:03:24 +08:00
Kevin Wan ebec5aafab use env if necessary in loading config (#409) 2021-01-21 19:33:34 +08:00
Kevin Wan 572b32729f update goctl version to 1.1.3 (#402) 2021-01-18 16:34:00 +08:00
kingxt 43e712d86a fix type convert error (#395) 2021-01-16 18:24:11 +08:00
kingxt 4db20677f7 optimized (#392) 2021-01-15 11:36:37 +08:00
Kevin Wan 6887fb22de add more tests for codec (#391) 2021-01-14 23:39:44 +08:00
Kevin Wan 50fbdbcfd7 update readme (#390) 2021-01-14 22:26:31 +08:00
ALMAS c77b8489d7 Update periodicalexecutor.go (#389) 2021-01-14 22:20:09 +08:00
Kevin Wan eca4ed2cc0 format code (#386) 2021-01-14 13:24:24 +08:00
954 changed files with 34595 additions and 23624 deletions
+2 -3
View File
@@ -1,4 +1,3 @@
comment: false
ignore:
- "doc"
- "example"
- "tools"
- "tools"
+12
View File
@@ -0,0 +1,12 @@
# These are supported funding model platforms
github: # Replace with up to 4 GitHub Sponsors-enabled usernames e.g., [user1, user2]
patreon: # Replace with a single Patreon username
open_collective: # Replace with a single Open Collective username
ko_fi: # Replace with a single Ko-fi username
tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel
community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry
liberapay: # Replace with a single Liberapay username
issuehunt: # Replace with a single IssueHunt username
otechie: # Replace with a single Otechie username
custom: https://gitee.com/kevwan/static/raw/master/images/sponsor.jpg
+40
View File
@@ -0,0 +1,40 @@
---
name: Bug report
about: Create a report to help us improve
title: ''
labels: ''
assignees: ''
---
**Describe the bug**
A clear and concise description of what the bug is.
**To Reproduce**
Steps to reproduce the behavior, if applicable:
1. The code is
```go
```
2. The error is
```
```
**Expected behavior**
A clear and concise description of what you expected to happen.
**Screenshots**
If applicable, add screenshots to help explain your problem.
**Environments (please complete the following information):**
- OS: [e.g. Linux]
- go-zero version [e.g. 1.2.1]
- goctl version [e.g. 1.2.1, optional]
**More description**
Add any other context about the problem here.
+20
View File
@@ -0,0 +1,20 @@
---
name: Feature request
about: Suggest an idea for this project
title: ''
labels: ''
assignees: ''
---
**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
**Describe the solution you'd like**
A clear and concise description of what you want to happen.
**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.
**Additional context**
Add any other context or screenshots about the feature request here.
+10
View File
@@ -0,0 +1,10 @@
---
name: Question
about: Ask a question on using go-zero or goctl
title: ''
labels: ''
assignees: ''
---
+40 -18
View File
@@ -7,28 +7,50 @@ on:
branches: [ master ]
jobs:
build:
name: Build
test-linux:
name: Linux
runs-on: ubuntu-latest
steps:
- name: Set up Go 1.x
uses: actions/setup-go@v2
with:
go-version: ^1.15
id: go
- name: Set up Go 1.x
uses: actions/setup-go@v2
with:
go-version: ^1.13
id: go
- name: Check out code into the Go module directory
uses: actions/checkout@v2
- name: Check out code into the Go module directory
uses: actions/checkout@v2
- name: Get dependencies
run: |
go get -v -t -d ./...
- name: Get dependencies
run: |
go get -v -t -d ./...
- name: Lint
run: |
go vet -stdmethods=false $(go list ./...)
go install mvdan.cc/gofumpt@latest
test -z "$(gofumpt -s -l -extra .)" || echo "Please run 'gofumpt -l -w -extra .'"
- name: Test
run: go test -race -coverprofile=coverage.txt -covermode=atomic ./...
- name: Test
run: go test -race -coverprofile=coverage.txt -covermode=atomic ./...
- name: Codecov
uses: codecov/codecov-action@v1.0.6
with:
token: ${{secrets.CODECOV_TOKEN}}
- name: Codecov
uses: codecov/codecov-action@v2
test-win:
name: Windows
runs-on: windows-latest
steps:
- name: Set up Go 1.x
uses: actions/setup-go@v2
with:
go-version: ^1.15
- name: Checkout codebase
uses: actions/checkout@v2
- name: Test
run: |
go mod verify
go mod download
go test -v -race ./...
cd tools/goctl && go build -v goctl.go
+18
View File
@@ -0,0 +1,18 @@
name: 'issue-translator'
on:
issue_comment:
types: [created]
issues:
types: [opened]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: tomsun28/issues-translate-action@v2.6
with:
IS_MODIFY_TITLE: true
# not require, default false, . Decide whether to modify the issue title
# if true, the robot account @Issues-translate-bot must have modification permissions, invite @Issues-translate-bot to your project or use your custom bot.
CUSTOM_BOT_NOTE: Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿
# not require. Customize the translation robot prefix message.
+19
View File
@@ -0,0 +1,19 @@
name: Close inactive issues
on:
schedule:
- cron: "30 1 * * *"
jobs:
close-issues:
runs-on: ubuntu-latest
steps:
- uses: actions/stale@v3
with:
days-before-issue-stale: 30
days-before-issue-close: 14
stale-issue-label: "stale"
stale-issue-message: "This issue is stale because it has been open for 30 days with no activity."
close-issue-message: "This issue was closed because it has been inactive for 14 days since being marked as stale."
days-before-pr-stale: -1
days-before-pr-close: -1
repo-token: ${{ secrets.GITHUB_TOKEN }}
+19
View File
@@ -0,0 +1,19 @@
name: reviewdog
on: [pull_request]
jobs:
staticcheck:
name: runner / staticcheck
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: reviewdog/action-staticcheck@v1
with:
github_token: ${{ secrets.github_token }}
# Change reviewdog reporter if you need [github-pr-check,github-check,github-pr-review].
reporter: github-pr-review
# Report all results.
filter_mode: nofilter
# Exit with 1 when it find at least one finding.
fail_on_error: true
# Set staticcheck flags
staticcheck_flags: -checks=inherit,-SA1019,-SA1029,-SA5008
+5
View File
@@ -10,10 +10,15 @@
!*/
!api
# ignore
.idea
**/.DS_Store
**/logs
# for test purpose
**/adhoc
**/testdata
# gitlab ci
.cache
+102
View File
@@ -0,0 +1,102 @@
# Contributing
Welcome to go-zero!
- [Before you get started](#before-you-get-started)
- [Code of Conduct](#code-of-conduct)
- [Community Expectations](#community-expectations)
- [Getting started](#getting-started)
- [Your First Contribution](#your-first-contribution)
- [Find something to work on](#find-something-to-work-on)
- [Find a good first topic](#find-a-good-first-topic)
- [Work on an Issue](#work-on-an-issue)
- [File an Issue](#file-an-issue)
- [Contributor Workflow](#contributor-workflow)
- [Creating Pull Requests](#creating-pull-requests)
- [Code Review](#code-review)
- [Testing](#testing)
# Before you get started
## Code of Conduct
Please make sure to read and observe our [Code of Conduct](/code-of-conduct.md).
## Community Expectations
go-zero is a community project driven by its community which strives to promote a healthy, friendly and productive environment.
go-zero is a web and rpc framework written in Go. It's born to ensure the stability of the busy sites with resilient design. Builtin goctl greatly improves the development productivity.
# Getting started
- Fork the repository on GitHub.
- Make your changes on your fork repository.
- Submit a PR.
# Your First Contribution
We will help you to contribute in different areas like filing issues, developing features, fixing critical bugs and
getting your work reviewed and merged.
If you have questions about the development process,
feel free to [file an issue](https://github.com/zeromicro/go-zero/issues/new/choose).
## Find something to work on
We are always in need of help, be it fixing documentation, reporting bugs or writing some code.
Look at places where you feel best coding practices aren't followed, code refactoring is needed or tests are missing.
Here is how you get started.
### Find a good first topic
[go-zero](https://github.com/zeromicro/go-zero) has beginner-friendly issues that provide a good first issue.
For example, [go-zero](https://github.com/zeromicro/go-zero) has
[help wanted](https://github.com/zeromicro/go-zero/issues?q=is%3Aopen+is%3Aissue+label%3A%22help+wanted%22) and
[good first issue](https://github.com/zeromicro/go-zero/issues?q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22)
labels for issues that should not need deep knowledge of the system.
We can help new contributors who wish to work on such issues.
Another good way to contribute is to find a documentation improvement, such as a missing/broken link.
Please see [Contributing](#contributing) below for the workflow.
#### Work on an issue
When you are willing to take on an issue, just reply on the issue. The maintainer will assign it to you.
### File an Issue
While we encourage everyone to contribute code, it is also appreciated when someone reports an issue.
Please follow the prompted submission guidelines while opening an issue.
# Contributor Workflow
Please do not ever hesitate to ask a question or send a pull request.
This is a rough outline of what a contributor's workflow looks like:
- Create a topic branch from where to base the contribution. This is usually master.
- Make commits of logical units.
- Push changes in a topic branch to a personal fork of the repository.
- Submit a pull request to [go-zero](https://github.com/zeromicro/go-zero).
## Creating Pull Requests
Pull requests are often called simply "PR".
go-zero generally follows the standard [github pull request](https://help.github.com/articles/about-pull-requests/) process.
To submit a proposed change, please develop the code/fix and add new test cases.
After that, run these local verifications before submitting pull request to predict the pass or
fail of continuous integration.
* Format the code with `gofmt`
* Run the test with data race enabled `go test -race ./...`
## Code Review
To make it easier for your PR to receive reviews, consider the reviewers will need you to:
* follow [good coding guidelines](https://github.com/golang/go/wiki/CodeReviewComments).
* write [good commit messages](https://chris.beams.io/posts/git-commit/).
* break large changes into a logical series of smaller patches which individually make easily understandable changes, and in aggregate solve a broader issue.
+28
View File
@@ -0,0 +1,28 @@
# go-zero Roadmap
This document defines a high level roadmap for go-zero development and upcoming releases.
Community and contributor involvement is vital for successfully implementing all desired items for each release.
We hope that the items listed below will inspire further engagement from the community to keep go-zero progressing and shipping exciting and valuable features.
## 2021 Q2
- [x] Support service discovery through K8S client api
- [x] Log full sql statements for easier sql problem solving
## 2021 Q3
- [x] Support `goctl model pg` to support PostgreSQL code generation
- [x] Adapt builtin tracing mechanism to opentracing solutions
## 2021 Q4
- [x] Support `username/password` authentication in ETCD
- [x] Support `SSL/TLS` in ETCD
- [x] Support `SSL/TLS` in `zRPC`
- [x] Support `TLS` in redis connections
- [x] Support `goctl bug` to report bugs conveniently
## 2022
- [x] Support `context` in redis related methods for timeout and tracing
- [x] Support `context` in sql related methods for timeout and tracing
- [ ] Support `context` in mongodb related methods for timeout and tracing
- [ ] Add `httpx.Client` with governance, like circuit breaker etc.
- [ ] Support `goctl doctor` command to report potential issues for given service
- [ ] Support `goctl mock` command to start a mocking server with given `.api` file
+76
View File
@@ -0,0 +1,76 @@
# Contributor Covenant Code of Conduct
## Our Pledge
In the interest of fostering an open and welcoming environment, we as
contributors and maintainers pledge to make participation in our project and
our community a harassment-free experience for everyone, regardless of age, body
size, disability, ethnicity, sex characteristics, gender identity and expression,
level of experience, education, socio-economic status, nationality, personal
appearance, race, religion, or sexual identity and orientation.
## Our Standards
Examples of behavior that contributes to creating a positive environment
include:
* Using welcoming and inclusive language
* Being respectful of differing viewpoints and experiences
* Gracefully accepting constructive criticism
* Focusing on what is best for the community
* Showing empathy towards other community members
Examples of unacceptable behavior by participants include:
* The use of sexualized language or imagery and unwelcome sexual attention or
advances
* Trolling, insulting/derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or electronic
address, without explicit permission
* Other conduct which could reasonably be considered inappropriate in a
professional setting
## Our Responsibilities
Project maintainers are responsible for clarifying the standards of acceptable
behavior and are expected to take appropriate and fair corrective action in
response to any instances of unacceptable behavior.
Project maintainers have the right and responsibility to remove, edit, or
reject comments, commits, code, wiki edits, issues, and other contributions
that are not aligned to this Code of Conduct, or to ban temporarily or
permanently any contributor for other behaviors that they deem inappropriate,
threatening, offensive, or harmful.
## Scope
This Code of Conduct applies within all project spaces, and it also applies when
an individual is representing the project or its community in public spaces.
Examples of representing a project or community include using an official
project e-mail address, posting via an official social media account, or acting
as an appointed representative at an online or offline event. Representation of
a project may be further defined and clarified by project maintainers.
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported by contacting the project team at [INSERT EMAIL ADDRESS]. All
complaints will be reviewed and investigated and will result in a response that
is deemed necessary and appropriate to the circumstances. The project team is
obligated to maintain confidentiality with regard to the reporter of an incident.
Further details of specific enforcement policies may be posted separately.
Project maintainers who do not follow or enforce the Code of Conduct in good
faith may face temporary or permanent repercussions as determined by other
members of the project's leadership.
## Attribution
This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html
[homepage]: https://www.contributor-covenant.org
For answers to common questions about this code of conduct, see
https://www.contributor-covenant.org/faq
+25 -24
View File
@@ -4,8 +4,8 @@ import (
"errors"
"strconv"
"github.com/tal-tech/go-zero/core/hash"
"github.com/tal-tech/go-zero/core/stores/redis"
"github.com/zeromicro/go-zero/core/hash"
"github.com/zeromicro/go-zero/core/stores/redis"
)
const (
@@ -27,43 +27,43 @@ return true
`
)
// ErrTooLargeOffset indicates the offset is too large in bitset.
var ErrTooLargeOffset = errors.New("too large offset")
type (
BitSetProvider interface {
// A Filter is a bloom filter.
Filter struct {
bits uint
bitSet bitSetProvider
}
bitSetProvider interface {
check([]uint) (bool, error)
set([]uint) error
}
BloomFilter struct {
bits uint
bitSet BitSetProvider
}
)
// New create a BloomFilter, store is the backed redis, key is the key for the bloom filter,
// New create a Filter, store is the backed redis, key is the key for the bloom filter,
// bits is how many bits will be used, maps is how many hashes for each addition.
// best practices:
// elements - means how many actual elements
// when maps = 14, formula: 0.7*(bits/maps), bits = 20*elements, the error rate is 0.000067 < 1e-4
// for detailed error rate table, see http://pages.cs.wisc.edu/~cao/papers/summary-cache/node8.html
func New(store *redis.Redis, key string, bits uint) *BloomFilter {
return &BloomFilter{
func New(store *redis.Redis, key string, bits uint) *Filter {
return &Filter{
bits: bits,
bitSet: newRedisBitSet(store, key, bits),
}
}
func (f *BloomFilter) Add(data []byte) error {
// Add adds data into f.
func (f *Filter) Add(data []byte) error {
locations := f.getLocations(data)
err := f.bitSet.set(locations)
if err != nil {
return err
}
return nil
return f.bitSet.set(locations)
}
func (f *BloomFilter) Exists(data []byte) (bool, error) {
// Exists checks if data is in f.
func (f *Filter) Exists(data []byte) (bool, error) {
locations := f.getLocations(data)
isSet, err := f.bitSet.check(locations)
if err != nil {
@@ -76,7 +76,7 @@ func (f *BloomFilter) Exists(data []byte) (bool, error) {
return true, nil
}
func (f *BloomFilter) getLocations(data []byte) []uint {
func (f *Filter) getLocations(data []byte) []uint {
locations := make([]uint, maps)
for i := uint(0); i < maps; i++ {
hashValue := hash.Hash(append(data, byte(i)))
@@ -127,11 +127,12 @@ func (r *redisBitSet) check(offsets []uint) (bool, error) {
return false, err
}
if exists, ok := resp.(int64); !ok {
exists, ok := resp.(int64)
if !ok {
return false, nil
} else {
return exists == 1, nil
}
return exists == 1, nil
}
func (r *redisBitSet) del() error {
@@ -152,7 +153,7 @@ func (r *redisBitSet) set(offsets []uint) error {
_, err = r.store.Eval(setScript, []string{r.key}, args)
if err == redis.Nil {
return nil
} else {
return err
}
return err
}
+1 -1
View File
@@ -4,7 +4,7 @@ import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/tal-tech/go-zero/core/stores/redis/redistest"
"github.com/zeromicro/go-zero/core/stores/redis/redistest"
)
func TestRedisBitSet_New_Set_Test(t *testing.T) {
+15 -6
View File
@@ -6,11 +6,11 @@ import (
"strings"
"sync"
"github.com/tal-tech/go-zero/core/mathx"
"github.com/tal-tech/go-zero/core/proc"
"github.com/tal-tech/go-zero/core/stat"
"github.com/tal-tech/go-zero/core/stringx"
"github.com/tal-tech/go-zero/core/timex"
"github.com/zeromicro/go-zero/core/mathx"
"github.com/zeromicro/go-zero/core/proc"
"github.com/zeromicro/go-zero/core/stat"
"github.com/zeromicro/go-zero/core/stringx"
"github.com/zeromicro/go-zero/core/timex"
)
const (
@@ -18,12 +18,14 @@ const (
timeFormat = "15:04:05"
)
// ErrServiceUnavailable is returned when the CB state is open
// ErrServiceUnavailable is returned when the Breaker state is open.
var ErrServiceUnavailable = errors.New("circuit breaker is open")
type (
// Acceptable is the func to check if the error can be accepted.
Acceptable func(err error) bool
// A Breaker represents a circuit breaker.
Breaker interface {
// Name returns the name of the Breaker.
Name() string
@@ -61,10 +63,14 @@ type (
DoWithFallbackAcceptable(req func() error, fallback func(err error) error, acceptable Acceptable) error
}
// Option defines the method to customize a Breaker.
Option func(breaker *circuitBreaker)
// Promise interface defines the callbacks that returned by Breaker.Allow.
Promise interface {
// Accept tells the Breaker that the call is successful.
Accept()
// Reject tells the Breaker that the call is failed.
Reject(reason string)
}
@@ -89,6 +95,8 @@ type (
}
)
// NewBreaker returns a Breaker object.
// opts can be used to customize the Breaker.
func NewBreaker(opts ...Option) Breaker {
var b circuitBreaker
for _, opt := range opts {
@@ -127,6 +135,7 @@ func (cb *circuitBreaker) Name() string {
return cb.name
}
// WithName returns a function to set the name of a Breaker.
func WithName(name string) Option {
return func(b *circuitBreaker) {
b.name = name
+2 -3
View File
@@ -8,7 +8,7 @@ import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/tal-tech/go-zero/core/stat"
"github.com/zeromicro/go-zero/core/stat"
)
func init() {
@@ -122,8 +122,7 @@ func BenchmarkGoogleBreaker(b *testing.B) {
}
}
type mockedPromise struct {
}
type mockedPromise struct{}
func (m *mockedPromise) Accept() {
}
+7 -1
View File
@@ -7,24 +7,28 @@ var (
breakers = make(map[string]Breaker)
)
// Do calls Breaker.Do on the Breaker with given name.
func Do(name string, req func() error) error {
return do(name, func(b Breaker) error {
return b.Do(req)
})
}
// DoWithAcceptable calls Breaker.DoWithAcceptable on the Breaker with given name.
func DoWithAcceptable(name string, req func() error, acceptable Acceptable) error {
return do(name, func(b Breaker) error {
return b.DoWithAcceptable(req, acceptable)
})
}
// DoWithFallback calls Breaker.DoWithFallback on the Breaker with given name.
func DoWithFallback(name string, req func() error, fallback func(err error) error) error {
return do(name, func(b Breaker) error {
return b.DoWithFallback(req, fallback)
})
}
// DoWithFallbackAcceptable calls Breaker.DoWithFallbackAcceptable on the Breaker with given name.
func DoWithFallbackAcceptable(name string, req func() error, fallback func(err error) error,
acceptable Acceptable) error {
return do(name, func(b Breaker) error {
@@ -32,6 +36,7 @@ func DoWithFallbackAcceptable(name string, req func() error, fallback func(err e
})
}
// GetBreaker returns the Breaker with the given name.
func GetBreaker(name string) Breaker {
lock.RLock()
b, ok := breakers[name]
@@ -51,7 +56,8 @@ func GetBreaker(name string) Breaker {
return b
}
func NoBreakFor(name string) {
// NoBreakerFor disables the circuit breaker for the given name.
func NoBreakerFor(name string) {
lock.Lock()
breakers[name] = newNoOpBreaker()
lock.Unlock()
+2 -2
View File
@@ -6,7 +6,7 @@ import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/tal-tech/go-zero/core/stat"
"github.com/zeromicro/go-zero/core/stat"
)
func init() {
@@ -55,7 +55,7 @@ func TestBreakersDoWithAcceptable(t *testing.T) {
}
func TestBreakersNoBreakerFor(t *testing.T) {
NoBreakFor("any")
NoBreakerFor("any")
errDummy := errors.New("any")
for i := 0; i < 10000; i++ {
assert.Equal(t, errDummy, GetBreaker("any").Do(func() error {
+5 -5
View File
@@ -4,8 +4,8 @@ import (
"math"
"time"
"github.com/tal-tech/go-zero/core/collection"
"github.com/tal-tech/go-zero/core/mathx"
"github.com/zeromicro/go-zero/core/collection"
"github.com/zeromicro/go-zero/core/mathx"
)
const (
@@ -64,9 +64,9 @@ func (b *googleBreaker) doReq(req func() error, fallback func(err error) error,
if err := b.accept(); err != nil {
if fallback != nil {
return fallback(err)
} else {
return err
}
return err
}
defer func() {
@@ -94,7 +94,7 @@ func (b *googleBreaker) markFailure() {
b.stat.Add(0)
}
func (b *googleBreaker) history() (accepts int64, total int64) {
func (b *googleBreaker) history() (accepts, total int64) {
b.stat.Reduce(func(b *collection.Bucket) {
accepts += int64(b.Sum)
total += b.Count
+3 -3
View File
@@ -7,9 +7,9 @@ import (
"time"
"github.com/stretchr/testify/assert"
"github.com/tal-tech/go-zero/core/collection"
"github.com/tal-tech/go-zero/core/mathx"
"github.com/tal-tech/go-zero/core/stat"
"github.com/zeromicro/go-zero/core/collection"
"github.com/zeromicro/go-zero/core/mathx"
"github.com/zeromicro/go-zero/core/stat"
)
const (
+2
View File
@@ -7,11 +7,13 @@ import (
"strings"
)
// EnterToContinue let stdin waiting for an enter key to continue.
func EnterToContinue() {
fmt.Print("Press 'Enter' to continue...")
bufio.NewReader(os.Stdin).ReadBytes('\n')
}
// ReadLine shows prompt to stdout and read a line from stdin.
func ReadLine(prompt string) string {
fmt.Print(prompt)
input, _ := bufio.NewReader(os.Stdin).ReadString('\n')
+2 -2
View File
@@ -8,8 +8,8 @@ import (
"time"
"github.com/stretchr/testify/assert"
"github.com/tal-tech/go-zero/core/iox"
"github.com/tal-tech/go-zero/core/lang"
"github.com/zeromicro/go-zero/core/iox"
"github.com/zeromicro/go-zero/core/lang"
)
func TestEnterToContinue(t *testing.T) {
+18 -8
View File
@@ -7,9 +7,10 @@ import (
"encoding/base64"
"errors"
"github.com/tal-tech/go-zero/core/logx"
"github.com/zeromicro/go-zero/core/logx"
)
// ErrPaddingSize indicates bad padding size.
var ErrPaddingSize = errors.New("padding size error")
type ecb struct {
@@ -26,6 +27,7 @@ func newECB(b cipher.Block) *ecb {
type ecbEncrypter ecb
// NewECBEncrypter returns an ECB encrypter.
func NewECBEncrypter(b cipher.Block) cipher.BlockMode {
return (*ecbEncrypter)(newECB(b))
}
@@ -52,6 +54,7 @@ func (x *ecbEncrypter) CryptBlocks(dst, src []byte) {
type ecbDecrypter ecb
// NewECBDecrypter returns an ECB decrypter.
func NewECBDecrypter(b cipher.Block) cipher.BlockMode {
return (*ecbDecrypter)(newECB(b))
}
@@ -78,6 +81,7 @@ func (x *ecbDecrypter) CryptBlocks(dst, src []byte) {
}
}
// EcbDecrypt decrypts src with the given key.
func EcbDecrypt(key, src []byte) ([]byte, error) {
block, err := aes.NewCipher(key)
if err != nil {
@@ -92,6 +96,8 @@ func EcbDecrypt(key, src []byte) ([]byte, error) {
return pkcs5Unpadding(decrypted, decrypter.BlockSize())
}
// EcbDecryptBase64 decrypts base64 encoded src with the given base64 encoded key.
// The returned string is also base64 encoded.
func EcbDecryptBase64(key, src string) (string, error) {
keyBytes, err := getKeyBytes(key)
if err != nil {
@@ -111,6 +117,7 @@ func EcbDecryptBase64(key, src string) (string, error) {
return base64.StdEncoding.EncodeToString(decryptedBytes), nil
}
// EcbEncrypt encrypts src with the given key.
func EcbEncrypt(key, src []byte) ([]byte, error) {
block, err := aes.NewCipher(key)
if err != nil {
@@ -126,6 +133,8 @@ func EcbEncrypt(key, src []byte) ([]byte, error) {
return crypted, nil
}
// EcbEncryptBase64 encrypts base64 encoded src with the given base64 encoded key.
// The returned string is also base64 encoded.
func EcbEncryptBase64(key, src string) (string, error) {
keyBytes, err := getKeyBytes(key)
if err != nil {
@@ -146,15 +155,16 @@ func EcbEncryptBase64(key, src string) (string, error) {
}
func getKeyBytes(key string) ([]byte, error) {
if len(key) > 32 {
if keyBytes, err := base64.StdEncoding.DecodeString(key); err != nil {
return nil, err
} else {
return keyBytes, nil
}
if len(key) <= 32 {
return []byte(key), nil
}
return []byte(key), nil
keyBytes, err := base64.StdEncoding.DecodeString(key)
if err != nil {
return nil, err
}
return keyBytes, nil
}
func pkcs5Padding(ciphertext []byte, blockSize int) []byte {
+65
View File
@@ -0,0 +1,65 @@
package codec
import (
"encoding/base64"
"testing"
"github.com/stretchr/testify/assert"
)
func TestAesEcb(t *testing.T) {
var (
key = []byte("q4t7w!z%C*F-JaNdRgUjXn2r5u8x/A?D")
val = []byte("hello")
badKey1 = []byte("aaaaaaaaa")
// more than 32 chars
badKey2 = []byte("aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa")
)
_, err := EcbEncrypt(badKey1, val)
assert.NotNil(t, err)
_, err = EcbEncrypt(badKey2, val)
assert.NotNil(t, err)
dst, err := EcbEncrypt(key, val)
assert.Nil(t, err)
_, err = EcbDecrypt(badKey1, dst)
assert.NotNil(t, err)
_, err = EcbDecrypt(badKey2, dst)
assert.NotNil(t, err)
_, err = EcbDecrypt(key, val)
// not enough block, just nil
assert.Nil(t, err)
src, err := EcbDecrypt(key, dst)
assert.Nil(t, err)
assert.Equal(t, val, src)
}
func TestAesEcbBase64(t *testing.T) {
const (
val = "hello"
badKey1 = "aaaaaaaaa"
// more than 32 chars
badKey2 = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
)
key := []byte("q4t7w!z%C*F-JaNdRgUjXn2r5u8x/A?D")
b64Key := base64.StdEncoding.EncodeToString(key)
b64Val := base64.StdEncoding.EncodeToString([]byte(val))
_, err := EcbEncryptBase64(badKey1, val)
assert.NotNil(t, err)
_, err = EcbEncryptBase64(badKey2, val)
assert.NotNil(t, err)
_, err = EcbEncryptBase64(b64Key, val)
assert.NotNil(t, err)
dst, err := EcbEncryptBase64(b64Key, b64Val)
assert.Nil(t, err)
_, err = EcbDecryptBase64(badKey1, dst)
assert.NotNil(t, err)
_, err = EcbDecryptBase64(badKey2, dst)
assert.NotNil(t, err)
_, err = EcbDecryptBase64(b64Key, val)
assert.NotNil(t, err)
src, err := EcbDecryptBase64(b64Key, dst)
assert.Nil(t, err)
b, err := base64.StdEncoding.DecodeString(src)
assert.Nil(t, err)
assert.Equal(t, val, string(b))
}
+10 -2
View File
@@ -11,8 +11,11 @@ import (
// 2048-bit MODP Group
var (
ErrInvalidPriKey = errors.New("invalid private key")
ErrInvalidPubKey = errors.New("invalid public key")
// ErrInvalidPriKey indicates the invalid private key.
ErrInvalidPriKey = errors.New("invalid private key")
// ErrInvalidPubKey indicates the invalid public key.
ErrInvalidPubKey = errors.New("invalid public key")
// ErrPubKeyOutOfBound indicates the public key is out of bound.
ErrPubKeyOutOfBound = errors.New("public key out of bound")
p, _ = new(big.Int).SetString("FFFFFFFFFFFFFFFFC90FDAA22168C234C4C6628B80DC1CD129024E088A67CC74020BBEA63B139B22514A08798E3404DDEF9519B3CD3A431B302B0A6DF25F14374FE1356D6D51C245E485B576625E7EC6F44C42E9A637ED6B0BFF5CB6F406B7EDEE386BFB5A899FA5AE9F24117C4B1FE649286651ECE45B3DC2007CB8A163BF0598DA48361C55D39A69163FA8FD24CF5F83655D23DCA3AD961C62F356208552BB9ED529077096966D670C354E4ABC9804F1746C08CA18217C32905E462E36CE3BE39E772C180E86039B2783A2EC07A28FB5C55DF06F4C52C9DE2BCBF6955817183995497CEA956AE515D2261898FA051015728E5A8AACAA68FFFFFFFFFFFFFFFF", 16)
@@ -20,11 +23,13 @@ var (
zero = big.NewInt(0)
)
// DhKey defines the Diffie Hellman key.
type DhKey struct {
PriKey *big.Int
PubKey *big.Int
}
// ComputeKey returns a key from public key and private key.
func ComputeKey(pubKey, priKey *big.Int) (*big.Int, error) {
if pubKey == nil {
return nil, ErrInvalidPubKey
@@ -41,6 +46,7 @@ func ComputeKey(pubKey, priKey *big.Int) (*big.Int, error) {
return new(big.Int).Exp(pubKey, priKey, p), nil
}
// GenerateKey returns a Diffie Hellman key.
func GenerateKey() (*DhKey, error) {
var err error
var x *big.Int
@@ -63,10 +69,12 @@ func GenerateKey() (*DhKey, error) {
return key, nil
}
// NewPublicKey returns a public key from the given bytes.
func NewPublicKey(bs []byte) *big.Int {
return new(big.Int).SetBytes(bs)
}
// Bytes returns public key bytes.
func (k *DhKey) Bytes() []byte {
if k.PubKey == nil {
return nil
+2
View File
@@ -8,6 +8,7 @@ import (
const unzipLimit = 100 * 1024 * 1024 // 100MB
// Gzip compresses bs.
func Gzip(bs []byte) []byte {
var b bytes.Buffer
@@ -18,6 +19,7 @@ func Gzip(bs []byte) []byte {
return b.Bytes()
}
// Gunzip uncompresses bs.
func Gunzip(bs []byte) ([]byte, error) {
r, err := gzip.NewReader(bytes.NewBuffer(bs))
if err != nil {
+2
View File
@@ -7,12 +7,14 @@ import (
"io"
)
// Hmac returns HMAC bytes for body with the given key.
func Hmac(key []byte, body string) []byte {
h := hmac.New(sha256.New, key)
io.WriteString(h, body)
return h.Sum(nil)
}
// HmacBase64 returns the base64 encoded string of HMAC for body with the given key.
func HmacBase64(key []byte, body string) string {
return base64.StdEncoding.EncodeToString(Hmac(key, body))
}
+9 -2
View File
@@ -11,17 +11,22 @@ import (
)
var (
// ErrPrivateKey indicates the invalid private key.
ErrPrivateKey = errors.New("private key error")
ErrPublicKey = errors.New("failed to parse PEM block containing the public key")
ErrNotRsaKey = errors.New("key type is not RSA")
// ErrPublicKey indicates the invalid public key.
ErrPublicKey = errors.New("failed to parse PEM block containing the public key")
// ErrNotRsaKey indicates the invalid RSA key.
ErrNotRsaKey = errors.New("key type is not RSA")
)
type (
// RsaDecrypter represents a RSA decrypter.
RsaDecrypter interface {
Decrypt(input []byte) ([]byte, error)
DecryptBase64(input string) ([]byte, error)
}
// RsaEncrypter represents a RSA encrypter.
RsaEncrypter interface {
Encrypt(input []byte) ([]byte, error)
}
@@ -41,6 +46,7 @@ type (
}
)
// NewRsaDecrypter returns a RsaDecrypter with the given file.
func NewRsaDecrypter(file string) (RsaDecrypter, error) {
content, err := ioutil.ReadFile(file)
if err != nil {
@@ -84,6 +90,7 @@ func (r *rsaDecrypter) DecryptBase64(input string) ([]byte, error) {
return r.Decrypt(base64Decoded)
}
// NewRsaEncrypter returns a RsaEncrypter with the given key.
func NewRsaEncrypter(key []byte) (RsaEncrypter, error) {
block, _ := pem.Decode(key)
if block == nil {
+1 -1
View File
@@ -5,7 +5,7 @@ import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/tal-tech/go-zero/core/fs"
"github.com/zeromicro/go-zero/core/fs"
)
const (
+18 -8
View File
@@ -6,9 +6,9 @@ import (
"sync/atomic"
"time"
"github.com/tal-tech/go-zero/core/logx"
"github.com/tal-tech/go-zero/core/mathx"
"github.com/tal-tech/go-zero/core/syncx"
"github.com/zeromicro/go-zero/core/logx"
"github.com/zeromicro/go-zero/core/mathx"
"github.com/zeromicro/go-zero/core/syncx"
)
const (
@@ -23,8 +23,10 @@ const (
var emptyLruCache = emptyLru{}
type (
// CacheOption defines the method to customize a Cache.
CacheOption func(cache *Cache)
// A Cache object is a in-memory cache.
Cache struct {
name string
lock sync.Mutex
@@ -32,18 +34,19 @@ type (
expire time.Duration
timingWheel *TimingWheel
lruCache lru
barrier syncx.SharedCalls
barrier syncx.SingleFlight
unstableExpiry mathx.Unstable
stats *cacheStat
}
)
// NewCache returns a Cache with given expire.
func NewCache(expire time.Duration, opts ...CacheOption) (*Cache, error) {
cache := &Cache{
data: make(map[string]interface{}),
expire: expire,
lruCache: emptyLruCache,
barrier: syncx.NewSharedCalls(),
barrier: syncx.NewSingleFlight(),
unstableExpiry: mathx.NewUnstable(expiryDeviation),
}
@@ -72,6 +75,7 @@ func NewCache(expire time.Duration, opts ...CacheOption) (*Cache, error) {
return cache, nil
}
// Del deletes the item with the given key from c.
func (c *Cache) Del(key string) {
c.lock.Lock()
delete(c.data, key)
@@ -80,6 +84,7 @@ func (c *Cache) Del(key string) {
c.timingWheel.RemoveTimer(key)
}
// Get returns the item with the given key from c.
func (c *Cache) Get(key string) (interface{}, bool) {
value, ok := c.doGet(key)
if ok {
@@ -91,6 +96,7 @@ func (c *Cache) Get(key string) (interface{}, bool) {
return value, ok
}
// Set sets value into c with key.
func (c *Cache) Set(key string, value interface{}) {
c.lock.Lock()
_, ok := c.data[key]
@@ -106,6 +112,9 @@ func (c *Cache) Set(key string, value interface{}) {
}
}
// Take returns the item with the given key.
// If the item is in c, return it directly.
// If not, use fetch method to get the item, set into c and return it.
func (c *Cache) Take(key string, fetch func() (interface{}, error)) (interface{}, error) {
if val, ok := c.doGet(key); ok {
c.stats.IncrementHit()
@@ -136,11 +145,10 @@ func (c *Cache) Take(key string, fetch func() (interface{}, error)) (interface{}
if fresh {
c.stats.IncrementMiss()
return val, nil
} else {
// got the result from previous ongoing query
c.stats.IncrementHit()
}
// got the result from previous ongoing query
c.stats.IncrementHit()
return val, nil
}
@@ -168,6 +176,7 @@ func (c *Cache) size() int {
return len(c.data)
}
// WithLimit customizes a Cache with items up to limit.
func WithLimit(limit int) CacheOption {
return func(cache *Cache) {
if limit > 0 {
@@ -176,6 +185,7 @@ func WithLimit(limit int) CacheOption {
}
}
// WithName customizes a Cache with the given name.
func WithName(name string) CacheOption {
return func(cache *Cache) {
cache.name = name
+5
View File
@@ -2,6 +2,7 @@ package collection
import "sync"
// A Queue is a FIFO queue.
type Queue struct {
lock sync.Mutex
elements []interface{}
@@ -11,6 +12,7 @@ type Queue struct {
count int
}
// NewQueue returns a Queue object.
func NewQueue(size int) *Queue {
return &Queue{
elements: make([]interface{}, size),
@@ -18,6 +20,7 @@ func NewQueue(size int) *Queue {
}
}
// Empty checks if q is empty.
func (q *Queue) Empty() bool {
q.lock.Lock()
empty := q.count == 0
@@ -26,6 +29,7 @@ func (q *Queue) Empty() bool {
return empty
}
// Put puts element into q at the last position.
func (q *Queue) Put(element interface{}) {
q.lock.Lock()
defer q.lock.Unlock()
@@ -44,6 +48,7 @@ func (q *Queue) Put(element interface{}) {
q.count++
}
// Take takes the first element out of q if not empty.
func (q *Queue) Take() (interface{}, bool) {
q.lock.Lock()
defer q.lock.Unlock()
+38
View File
@@ -61,3 +61,41 @@ func TestPutMore(t *testing.T) {
assert.Equal(t, string(element), string(body.([]byte)))
}
}
func TestPutMoreWithHeaderNotZero(t *testing.T) {
elements := [][]byte{
[]byte("hello"),
[]byte("world"),
[]byte("again"),
}
queue := NewQueue(4)
for i := range elements {
queue.Put(elements[i])
}
// take 1
body, ok := queue.Take()
assert.True(t, ok)
element, ok := body.([]byte)
assert.True(t, ok)
assert.Equal(t, element, []byte("hello"))
// put more
queue.Put([]byte("b4"))
queue.Put([]byte("b5")) // will store in elements[0]
queue.Put([]byte("b6")) // cause expansion
results := [][]byte{
[]byte("world"),
[]byte("again"),
[]byte("b4"),
[]byte("b5"),
[]byte("b6"),
}
for _, element := range results {
body, ok := queue.Take()
assert.True(t, ok)
assert.Equal(t, string(element), string(body.([]byte)))
}
}
+13
View File
@@ -1,10 +1,15 @@
package collection
import "sync"
// A Ring can be used as fixed size ring.
type Ring struct {
elements []interface{}
index int
lock sync.Mutex
}
// NewRing returns a Ring object with the given size n.
func NewRing(n int) *Ring {
if n < 1 {
panic("n should be greater than 0")
@@ -15,12 +20,20 @@ func NewRing(n int) *Ring {
}
}
// Add adds v into r.
func (r *Ring) Add(v interface{}) {
r.lock.Lock()
defer r.lock.Unlock()
r.elements[r.index%len(r.elements)] = v
r.index++
}
// Take takes all items from r.
func (r *Ring) Take() []interface{} {
r.lock.Lock()
defer r.lock.Unlock()
var size int
var start int
if r.index > len(r.elements) {
+28
View File
@@ -1,6 +1,7 @@
package collection
import (
"sync"
"testing"
"github.com/stretchr/testify/assert"
@@ -29,3 +30,30 @@ func TestRingMore(t *testing.T) {
elements := ring.Take()
assert.ElementsMatch(t, []interface{}{6, 7, 8, 9, 10}, elements)
}
func TestRingAdd(t *testing.T) {
ring := NewRing(5051)
wg := sync.WaitGroup{}
for i := 1; i <= 100; i++ {
wg.Add(1)
go func(i int) {
defer wg.Done()
for j := 1; j <= i; j++ {
ring.Add(i)
}
}(i)
}
wg.Wait()
assert.Equal(t, 5050, len(ring.Take()))
}
func BenchmarkRingAdd(b *testing.B) {
ring := NewRing(500)
b.RunParallel(func(pb *testing.PB) {
for pb.Next() {
for i := 0; i < b.N; i++ {
ring.Add(i)
}
}
})
}
+3 -3
View File
@@ -4,7 +4,7 @@ import (
"sync"
"time"
"github.com/tal-tech/go-zero/core/timex"
"github.com/zeromicro/go-zero/core/timex"
)
type (
@@ -73,9 +73,9 @@ func (rw *RollingWindow) span() int {
offset := int(timex.Since(rw.lastTime) / rw.interval)
if 0 <= offset && offset < rw.size {
return offset
} else {
return rw.size
}
return rw.size
}
func (rw *RollingWindow) updateOffset() {
+2 -2
View File
@@ -6,7 +6,7 @@ import (
"time"
"github.com/stretchr/testify/assert"
"github.com/tal-tech/go-zero/core/stringx"
"github.com/zeromicro/go-zero/core/stringx"
)
const duration = time.Millisecond * 50
@@ -139,7 +139,7 @@ func TestRollingWindowBucketTimeBoundary(t *testing.T) {
func TestRollingWindowDataRace(t *testing.T) {
const size = 3
r := NewRollingWindow(size, duration)
var stop = make(chan bool)
stop := make(chan bool)
go func() {
for {
select {
+8 -3
View File
@@ -18,6 +18,7 @@ type SafeMap struct {
dirtyNew map[interface{}]interface{}
}
// NewSafeMap returns a SafeMap.
func NewSafeMap() *SafeMap {
return &SafeMap{
dirtyOld: make(map[interface{}]interface{}),
@@ -25,6 +26,7 @@ func NewSafeMap() *SafeMap {
}
}
// Del deletes the value with the given key from m.
func (m *SafeMap) Del(key interface{}) {
m.lock.Lock()
if _, ok := m.dirtyOld[key]; ok {
@@ -53,18 +55,20 @@ func (m *SafeMap) Del(key interface{}) {
m.lock.Unlock()
}
// Get gets the value with the given key from m.
func (m *SafeMap) Get(key interface{}) (interface{}, bool) {
m.lock.RLock()
defer m.lock.RUnlock()
if val, ok := m.dirtyOld[key]; ok {
return val, true
} else {
val, ok := m.dirtyNew[key]
return val, ok
}
val, ok := m.dirtyNew[key]
return val, ok
}
// Set sets the value into m with the given key.
func (m *SafeMap) Set(key, value interface{}) {
m.lock.Lock()
if m.deletionOld <= maxDeletion {
@@ -83,6 +87,7 @@ func (m *SafeMap) Set(key, value interface{}) {
m.lock.Unlock()
}
// Size returns the size of m.
func (m *SafeMap) Size() int {
m.lock.RLock()
size := len(m.dirtyOld) + len(m.dirtyNew)
+1 -1
View File
@@ -4,7 +4,7 @@ import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/tal-tech/go-zero/core/stringx"
"github.com/zeromicro/go-zero/core/stringx"
)
func TestSafeMap(t *testing.T) {
+24 -17
View File
@@ -1,8 +1,8 @@
package collection
import (
"github.com/tal-tech/go-zero/core/lang"
"github.com/tal-tech/go-zero/core/logx"
"github.com/zeromicro/go-zero/core/lang"
"github.com/zeromicro/go-zero/core/logx"
)
const (
@@ -21,6 +21,7 @@ type Set struct {
tp int
}
// NewSet returns a managed Set, can only put the values with the same type.
func NewSet() *Set {
return &Set{
data: make(map[interface{}]lang.PlaceholderType),
@@ -28,6 +29,7 @@ func NewSet() *Set {
}
}
// NewUnmanagedSet returns a unmanaged Set, which can put values with different types.
func NewUnmanagedSet() *Set {
return &Set{
data: make(map[interface{}]lang.PlaceholderType),
@@ -35,42 +37,49 @@ func NewUnmanagedSet() *Set {
}
}
// Add adds i into s.
func (s *Set) Add(i ...interface{}) {
for _, each := range i {
s.add(each)
}
}
// AddInt adds int values ii into s.
func (s *Set) AddInt(ii ...int) {
for _, each := range ii {
s.add(each)
}
}
// AddInt64 adds int64 values ii into s.
func (s *Set) AddInt64(ii ...int64) {
for _, each := range ii {
s.add(each)
}
}
// AddUint adds uint values ii into s.
func (s *Set) AddUint(ii ...uint) {
for _, each := range ii {
s.add(each)
}
}
// AddUint64 adds uint64 values ii into s.
func (s *Set) AddUint64(ii ...uint64) {
for _, each := range ii {
s.add(each)
}
}
// AddStr adds string values ss into s.
func (s *Set) AddStr(ss ...string) {
for _, each := range ss {
s.add(each)
}
}
// Contains checks if i is in s.
func (s *Set) Contains(i interface{}) bool {
if len(s.data) == 0 {
return false
@@ -81,6 +90,7 @@ func (s *Set) Contains(i interface{}) bool {
return ok
}
// Keys returns the keys in s.
func (s *Set) Keys() []interface{} {
var keys []interface{}
@@ -91,13 +101,12 @@ func (s *Set) Keys() []interface{} {
return keys
}
// KeysInt returns the int keys in s.
func (s *Set) KeysInt() []int {
var keys []int
for key := range s.data {
if intKey, ok := key.(int); !ok {
continue
} else {
if intKey, ok := key.(int); ok {
keys = append(keys, intKey)
}
}
@@ -105,13 +114,12 @@ func (s *Set) KeysInt() []int {
return keys
}
// KeysInt64 returns int64 keys in s.
func (s *Set) KeysInt64() []int64 {
var keys []int64
for key := range s.data {
if intKey, ok := key.(int64); !ok {
continue
} else {
if intKey, ok := key.(int64); ok {
keys = append(keys, intKey)
}
}
@@ -119,13 +127,12 @@ func (s *Set) KeysInt64() []int64 {
return keys
}
// KeysUint returns uint keys in s.
func (s *Set) KeysUint() []uint {
var keys []uint
for key := range s.data {
if intKey, ok := key.(uint); !ok {
continue
} else {
if intKey, ok := key.(uint); ok {
keys = append(keys, intKey)
}
}
@@ -133,13 +140,12 @@ func (s *Set) KeysUint() []uint {
return keys
}
// KeysUint64 returns uint64 keys in s.
func (s *Set) KeysUint64() []uint64 {
var keys []uint64
for key := range s.data {
if intKey, ok := key.(uint64); !ok {
continue
} else {
if intKey, ok := key.(uint64); ok {
keys = append(keys, intKey)
}
}
@@ -147,13 +153,12 @@ func (s *Set) KeysUint64() []uint64 {
return keys
}
// KeysStr returns string keys in s.
func (s *Set) KeysStr() []string {
var keys []string
for key := range s.data {
if strKey, ok := key.(string); !ok {
continue
} else {
if strKey, ok := key.(string); ok {
keys = append(keys, strKey)
}
}
@@ -161,11 +166,13 @@ func (s *Set) KeysStr() []string {
return keys
}
// Remove removes i from s.
func (s *Set) Remove(i interface{}) {
s.validate(i)
delete(s.data, i)
}
// Count returns the number of items in s.
func (s *Set) Count() int {
return len(s.data)
}
+1 -1
View File
@@ -5,7 +5,7 @@ import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/tal-tech/go-zero/core/logx"
"github.com/zeromicro/go-zero/core/logx"
)
func init() {
+12 -4
View File
@@ -5,16 +5,18 @@ import (
"fmt"
"time"
"github.com/tal-tech/go-zero/core/lang"
"github.com/tal-tech/go-zero/core/threading"
"github.com/tal-tech/go-zero/core/timex"
"github.com/zeromicro/go-zero/core/lang"
"github.com/zeromicro/go-zero/core/threading"
"github.com/zeromicro/go-zero/core/timex"
)
const drainWorkers = 8
type (
// Execute defines the method to execute the task.
Execute func(key, value interface{})
// A TimingWheel is a timing wheel object to schedule tasks.
TimingWheel struct {
interval time.Duration
ticker timex.Ticker
@@ -54,6 +56,7 @@ type (
}
)
// NewTimingWheel returns a TimingWheel.
func NewTimingWheel(interval time.Duration, numSlots int, execute Execute) (*TimingWheel, error) {
if interval <= 0 || numSlots <= 0 || execute == nil {
return nil, fmt.Errorf("interval: %v, slots: %d, execute: %p", interval, numSlots, execute)
@@ -85,10 +88,12 @@ func newTimingWheelWithClock(interval time.Duration, numSlots int, execute Execu
return tw, nil
}
// Drain drains all items and executes them.
func (tw *TimingWheel) Drain(fn func(key, value interface{})) {
tw.drainChannel <- fn
}
// MoveTimer moves the task with the given key to the given delay.
func (tw *TimingWheel) MoveTimer(key interface{}, delay time.Duration) {
if delay <= 0 || key == nil {
return
@@ -100,6 +105,7 @@ func (tw *TimingWheel) MoveTimer(key interface{}, delay time.Duration) {
}
}
// RemoveTimer removes the task with the given key.
func (tw *TimingWheel) RemoveTimer(key interface{}) {
if key == nil {
return
@@ -108,6 +114,7 @@ func (tw *TimingWheel) RemoveTimer(key interface{}) {
tw.removeChannel <- key
}
// SetTimer sets the task value with the given key to the delay.
func (tw *TimingWheel) SetTimer(key, value interface{}, delay time.Duration) {
if delay <= 0 || key == nil {
return
@@ -122,6 +129,7 @@ func (tw *TimingWheel) SetTimer(key, value interface{}, delay time.Duration) {
}
}
// Stop stops tw.
func (tw *TimingWheel) Stop() {
close(tw.stopChannel)
}
@@ -143,7 +151,7 @@ func (tw *TimingWheel) drainAll(fn func(key, value interface{})) {
}
}
func (tw *TimingWheel) getPositionAndCircle(d time.Duration) (pos int, circle int) {
func (tw *TimingWheel) getPositionAndCircle(d time.Duration) (pos, circle int) {
steps := int(d / tw.interval)
pos = (tw.tickedPos + steps) % tw.numSlots
circle = (steps - 1) / tw.numSlots
+4 -4
View File
@@ -8,10 +8,10 @@ import (
"time"
"github.com/stretchr/testify/assert"
"github.com/tal-tech/go-zero/core/lang"
"github.com/tal-tech/go-zero/core/stringx"
"github.com/tal-tech/go-zero/core/syncx"
"github.com/tal-tech/go-zero/core/timex"
"github.com/zeromicro/go-zero/core/lang"
"github.com/zeromicro/go-zero/core/stringx"
"github.com/zeromicro/go-zero/core/syncx"
"github.com/zeromicro/go-zero/core/timex"
)
const (
+26 -9
View File
@@ -7,7 +7,7 @@ import (
"os"
"path"
"github.com/tal-tech/go-zero/core/mapping"
"github.com/zeromicro/go-zero/core/mapping"
)
var loaders = map[string]func([]byte, interface{}) error{
@@ -16,26 +16,43 @@ var loaders = map[string]func([]byte, interface{}) error{
".yml": LoadConfigFromYamlBytes,
}
func LoadConfig(file string, v interface{}) error {
if content, err := ioutil.ReadFile(file); err != nil {
// LoadConfig loads config into v from file, .json, .yaml and .yml are acceptable.
func LoadConfig(file string, v interface{}, opts ...Option) error {
content, err := ioutil.ReadFile(file)
if err != nil {
return err
} else if loader, ok := loaders[path.Ext(file)]; ok {
return loader([]byte(os.ExpandEnv(string(content))), v)
} else {
return fmt.Errorf("unrecoginized file type: %s", file)
}
loader, ok := loaders[path.Ext(file)]
if !ok {
return fmt.Errorf("unrecognized file type: %s", file)
}
var opt options
for _, o := range opts {
o(&opt)
}
if opt.env {
return loader([]byte(os.ExpandEnv(string(content))), v)
}
return loader(content, v)
}
// LoadConfigFromJsonBytes loads config into v from content json bytes.
func LoadConfigFromJsonBytes(content []byte, v interface{}) error {
return mapping.UnmarshalJsonBytes(content, v)
}
// LoadConfigFromYamlBytes loads config into v from content yaml bytes.
func LoadConfigFromYamlBytes(content []byte, v interface{}) error {
return mapping.UnmarshalYamlBytes(content, v)
}
func MustLoad(path string, v interface{}) {
if err := LoadConfig(path, v); err != nil {
// MustLoad loads config into v from path, exits on error.
func MustLoad(path string, v interface{}, opts ...Option) {
if err := LoadConfig(path, v, opts...); err != nil {
log.Fatalf("error: config file %s, %s", path, err.Error())
}
}
+42 -3
View File
@@ -6,8 +6,8 @@ import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/tal-tech/go-zero/core/fs"
"github.com/tal-tech/go-zero/core/hash"
"github.com/zeromicro/go-zero/core/fs"
"github.com/zeromicro/go-zero/core/hash"
)
func TestLoadConfig_notExists(t *testing.T) {
@@ -30,7 +30,8 @@ func TestConfigJson(t *testing.T) {
text := `{
"a": "foo",
"b": 1,
"c": "${FOO}"
"c": "${FOO}",
"d": "abcd!@#$112"
}`
for _, test := range tests {
test := test
@@ -45,11 +46,49 @@ func TestConfigJson(t *testing.T) {
A string `json:"a"`
B int `json:"b"`
C string `json:"c"`
D string `json:"d"`
}
MustLoad(tmpfile, &val)
assert.Equal(t, "foo", val.A)
assert.Equal(t, 1, val.B)
assert.Equal(t, "${FOO}", val.C)
assert.Equal(t, "abcd!@#$112", val.D)
})
}
}
func TestConfigJsonEnv(t *testing.T) {
tests := []string{
".json",
".yaml",
".yml",
}
text := `{
"a": "foo",
"b": 1,
"c": "${FOO}",
"d": "abcd!@#$a12 3"
}`
for _, test := range tests {
test := test
t.Run(test, func(t *testing.T) {
os.Setenv("FOO", "2")
defer os.Unsetenv("FOO")
tmpfile, err := createTempFile(test, text)
assert.Nil(t, err)
defer os.Remove(tmpfile)
var val struct {
A string `json:"a"`
B int `json:"b"`
C string `json:"c"`
D string `json:"d"`
}
MustLoad(tmpfile, &val, UseEnv())
assert.Equal(t, "foo", val.A)
assert.Equal(t, 1, val.B)
assert.Equal(t, "2", val.C)
assert.Equal(t, "abcd!@# 3", val.D)
})
}
}
+17
View File
@@ -0,0 +1,17 @@
package conf
type (
// Option defines the method to customize the config options.
Option func(opt *options)
options struct {
env bool
}
)
// UseEnv customizes the config to use environment variables.
func UseEnv() Option {
return func(opt *options) {
opt.env = true
}
}
+17 -7
View File
@@ -2,11 +2,12 @@ package conf
import (
"fmt"
"os"
"strconv"
"strings"
"sync"
"github.com/tal-tech/go-zero/core/iox"
"github.com/zeromicro/go-zero/core/iox"
)
// PropertyError represents a configuration error message.
@@ -30,14 +31,19 @@ type mapBasedProperties struct {
lock sync.RWMutex
}
// Loads the properties into a properties configuration instance.
// LoadProperties loads the properties into a properties configuration instance.
// Returns an error that indicates if there was a problem loading the configuration.
func LoadProperties(filename string) (Properties, error) {
func LoadProperties(filename string, opts ...Option) (Properties, error) {
lines, err := iox.ReadTextLines(filename, iox.WithoutBlank(), iox.OmitWithPrefix("#"))
if err != nil {
return nil, err
}
var opt options
for _, o := range opts {
o(&opt)
}
raw := make(map[string]string)
for i := range lines {
pair := strings.Split(lines[i], "=")
@@ -50,7 +56,11 @@ func LoadProperties(filename string) (Properties, error) {
key := strings.TrimSpace(pair[0])
value := strings.TrimSpace(pair[1])
raw[key] = value
if opt.env {
raw[key] = os.ExpandEnv(value)
} else {
raw[key] = value
}
}
return &mapBasedProperties{
@@ -87,7 +97,7 @@ func (config *mapBasedProperties) SetInt(key string, value int) {
config.lock.Unlock()
}
// Dumps the configuration internal map into a string.
// ToString dumps the configuration internal map into a string.
func (config *mapBasedProperties) ToString() string {
config.lock.RLock()
ret := fmt.Sprintf("%s", config.properties)
@@ -96,12 +106,12 @@ func (config *mapBasedProperties) ToString() string {
return ret
}
// Returns the error message.
// Error returns the error message.
func (configError *PropertyError) Error() string {
return configError.message
}
// Builds a new properties configuration structure
// NewProperties builds a new properties configuration structure.
func NewProperties() Properties {
return &mapBasedProperties{
properties: make(map[string]string),
+34 -1
View File
@@ -5,7 +5,7 @@ import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/tal-tech/go-zero/core/fs"
"github.com/zeromicro/go-zero/core/fs"
)
func TestProperties(t *testing.T) {
@@ -31,6 +31,39 @@ func TestProperties(t *testing.T) {
assert.Contains(t, val, "app.threads")
}
func TestPropertiesEnv(t *testing.T) {
text := `app.name = test
app.program=app
app.env1 = ${FOO}
app.env2 = $none
# this is comment
app.threads = 5`
tmpfile, err := fs.TempFilenameWithText(text)
assert.Nil(t, err)
defer os.Remove(tmpfile)
os.Setenv("FOO", "2")
defer os.Unsetenv("FOO")
props, err := LoadProperties(tmpfile, UseEnv())
assert.Nil(t, err)
assert.Equal(t, "test", props.GetString("app.name"))
assert.Equal(t, "app", props.GetString("app.program"))
assert.Equal(t, 5, props.GetInt("app.threads"))
assert.Equal(t, "2", props.GetString("app.env1"))
assert.Equal(t, "", props.GetString("app.env2"))
val := props.ToString()
assert.Contains(t, val, "app.name")
assert.Contains(t, val, "app.program")
assert.Contains(t, val, "app.threads")
assert.Contains(t, val, "app.env1")
assert.Contains(t, val, "app.env2")
}
func TestLoadProperties_badContent(t *testing.T) {
filename, err := fs.TempFilenameWithText("hello")
assert.Nil(t, err)
-17
View File
@@ -1,17 +0,0 @@
package contextx
import (
"context"
"time"
)
func ShrinkDeadline(ctx context.Context, timeout time.Duration) (context.Context, func()) {
if deadline, ok := ctx.Deadline(); ok {
leftTime := time.Until(deadline)
if leftTime < timeout {
timeout = leftTime
}
}
return context.WithDeadline(ctx, time.Now().Add(timeout))
}
-31
View File
@@ -1,31 +0,0 @@
package contextx
import (
"context"
"testing"
"time"
"github.com/stretchr/testify/assert"
)
func TestShrinkDeadlineLess(t *testing.T) {
deadline := time.Now().Add(time.Second)
ctx, cancel := context.WithDeadline(context.Background(), deadline)
defer cancel()
ctx, cancel = ShrinkDeadline(ctx, time.Minute)
defer cancel()
dl, ok := ctx.Deadline()
assert.True(t, ok)
assert.Equal(t, deadline, dl)
}
func TestShrinkDeadlineMore(t *testing.T) {
deadline := time.Now().Add(time.Minute)
ctx, cancel := context.WithDeadline(context.Background(), deadline)
defer cancel()
ctx, cancel = ShrinkDeadline(ctx, time.Second)
defer cancel()
dl, ok := ctx.Deadline()
assert.True(t, ok)
assert.True(t, dl.Before(deadline))
}
+2 -1
View File
@@ -3,7 +3,7 @@ package contextx
import (
"context"
"github.com/tal-tech/go-zero/core/mapping"
"github.com/zeromicro/go-zero/core/mapping"
)
const contextTagKey = "ctx"
@@ -19,6 +19,7 @@ func (cv contextValuer) Value(key string) (interface{}, bool) {
return v, v != nil
}
// For unmarshals ctx into v.
func For(ctx context.Context, v interface{}) error {
return unmarshaler.UnmarshalValuer(contextValuer{
Context: ctx,
+3 -1
View File
@@ -47,9 +47,11 @@ func TestUnmarshalContextWithMissing(t *testing.T) {
Name string `ctx:"name"`
Age int `ctx:"age"`
}
type name string
const PersonNameKey name = "name"
ctx := context.Background()
ctx = context.WithValue(ctx, "name", "kevin")
ctx = context.WithValue(ctx, PersonNameKey, "kevin")
var person Person
err := For(ctx, &person)
+1
View File
@@ -21,6 +21,7 @@ func (valueOnlyContext) Err() error {
return nil
}
// ValueOnlyFrom takes all values from the given ctx, without deadline and error control.
func ValueOnlyFrom(ctx context.Context) context.Context {
return valueOnlyContext{
Context: ctx,
+3 -1
View File
@@ -9,7 +9,9 @@ import (
)
func TestContextCancel(t *testing.T) {
c := context.WithValue(context.Background(), "key", "value")
type key string
var nameKey key = "name"
c := context.WithValue(context.Background(), nameKey, "value")
c1, cancel := context.WithCancel(c)
o := ValueOnlyFrom(c1)
c2, cancel2 := context.WithCancel(o)
+14
View File
@@ -0,0 +1,14 @@
package discov
import "github.com/zeromicro/go-zero/core/discov/internal"
// RegisterAccount registers the username/password to the given etcd cluster.
func RegisterAccount(endpoints []string, user, pass string) {
internal.AddAccount(endpoints, user, pass)
}
// RegisterTLS registers the CertFile/CertKeyFile/CACertFile to the given etcd.
func RegisterTLS(endpoints []string, certFile, certKeyFile, caFile string,
insecureSkipVerify bool) error {
return internal.AddTLS(endpoints, certFile, certKeyFile, caFile, insecureSkipVerify)
}
+21
View File
@@ -0,0 +1,21 @@
package discov
import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/zeromicro/go-zero/core/discov/internal"
"github.com/zeromicro/go-zero/core/stringx"
)
func TestRegisterAccount(t *testing.T) {
endpoints := []string{
"localhost:2379",
}
user := "foo" + stringx.Rand()
RegisterAccount(endpoints, user, "bar")
account, ok := internal.GetAccount(endpoints)
assert.True(t, ok)
assert.Equal(t, user, account.User)
assert.Equal(t, "bar", account.Pass)
}
+2 -1
View File
@@ -4,7 +4,7 @@ import (
"fmt"
"strings"
"github.com/tal-tech/go-zero/core/discov/internal"
"github.com/zeromicro/go-zero/core/discov/internal"
)
const (
@@ -14,6 +14,7 @@ const (
const timeToLive int64 = 10
// TimeToLive is seconds to live in etcd.
var TimeToLive = timeToLive
func extract(etcdKey string, index int) (string, bool) {
+4 -1
View File
@@ -5,7 +5,7 @@ import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/tal-tech/go-zero/core/discov/internal"
"github.com/zeromicro/go-zero/core/discov/internal"
)
var mockLock sync.Mutex
@@ -28,6 +28,9 @@ func TestExtract(t *testing.T) {
_, ok = extract("any", -1)
assert.False(t, ok)
_, ok = extract("any", 10)
assert.False(t, ok)
}
func TestMakeKey(t *testing.T) {
+29 -4
View File
@@ -2,16 +2,41 @@ package discov
import "errors"
var (
// errEmptyEtcdHosts indicates that etcd hosts are empty.
errEmptyEtcdHosts = errors.New("empty etcd hosts")
// errEmptyEtcdKey indicates that etcd key is empty.
errEmptyEtcdKey = errors.New("empty etcd key")
)
// EtcdConf is the config item with the given key on etcd.
type EtcdConf struct {
Hosts []string
Key string
Hosts []string
Key string
User string `json:",optional"`
Pass string `json:",optional"`
CertFile string `json:",optional"`
CertKeyFile string `json:",optional=CertFile"`
CACertFile string `json:",optional=CertFile"`
InsecureSkipVerify bool `json:",optional"`
}
// HasAccount returns if account provided.
func (c EtcdConf) HasAccount() bool {
return len(c.User) > 0 && len(c.Pass) > 0
}
// HasTLS returns if TLS CertFile/CertKeyFile/CACertFile are provided.
func (c EtcdConf) HasTLS() bool {
return len(c.CertFile) > 0 && len(c.CertKeyFile) > 0 && len(c.CACertFile) > 0
}
// Validate validates c.
func (c EtcdConf) Validate() error {
if len(c.Hosts) == 0 {
return errors.New("empty etcd hosts")
return errEmptyEtcdHosts
} else if len(c.Key) == 0 {
return errors.New("empty etcd key")
return errEmptyEtcdKey
} else {
return nil
}
+36
View File
@@ -44,3 +44,39 @@ func TestConfig(t *testing.T) {
}
}
}
func TestEtcdConf_HasAccount(t *testing.T) {
tests := []struct {
EtcdConf
hasAccount bool
}{
{
EtcdConf: EtcdConf{
Hosts: []string{"any"},
Key: "key",
},
hasAccount: false,
},
{
EtcdConf: EtcdConf{
Hosts: []string{"any"},
Key: "key",
User: "foo",
},
hasAccount: false,
},
{
EtcdConf: EtcdConf{
Hosts: []string{"any"},
Key: "key",
User: "foo",
Pass: "bar",
},
hasAccount: true,
},
}
for _, test := range tests {
assert.Equal(t, test.hasAccount, test.EtcdConf.HasAccount())
}
}
-47
View File
@@ -1,47 +0,0 @@
package discov
import (
"github.com/tal-tech/go-zero/core/discov/internal"
"github.com/tal-tech/go-zero/core/logx"
)
type (
Facade struct {
endpoints []string
registry *internal.Registry
}
FacadeListener interface {
OnAdd(key, val string)
OnDelete(key string)
}
)
func NewFacade(endpoints []string) Facade {
return Facade{
endpoints: endpoints,
registry: internal.GetRegistry(),
}
}
func (f Facade) Client() internal.EtcdClient {
conn, err := f.registry.GetConn(f.endpoints)
logx.Must(err)
return conn
}
func (f Facade) Monitor(key string, l FacadeListener) {
f.registry.Monitor(f.endpoints, key, listenerAdapter{l})
}
type listenerAdapter struct {
l FacadeListener
}
func (la listenerAdapter) OnAdd(kv internal.KV) {
la.l.OnAdd(kv.Key, kv.Val)
}
func (la listenerAdapter) OnDelete(kv internal.KV) {
la.l.OnDelete(kv.Key)
}
+75
View File
@@ -0,0 +1,75 @@
package internal
import (
"crypto/tls"
"crypto/x509"
"io/ioutil"
"sync"
)
var (
accounts = make(map[string]Account)
tlsConfigs = make(map[string]*tls.Config)
lock sync.RWMutex
)
// Account holds the username/password for an etcd cluster.
type Account struct {
User string
Pass string
}
// AddAccount adds the username/password for the given etcd cluster.
func AddAccount(endpoints []string, user, pass string) {
lock.Lock()
defer lock.Unlock()
accounts[getClusterKey(endpoints)] = Account{
User: user,
Pass: pass,
}
}
// AddTLS adds the tls cert files for the given etcd cluster.
func AddTLS(endpoints []string, certFile, certKeyFile, caFile string, insecureSkipVerify bool) error {
cert, err := tls.LoadX509KeyPair(certFile, certKeyFile)
if err != nil {
return err
}
caData, err := ioutil.ReadFile(caFile)
if err != nil {
return err
}
pool := x509.NewCertPool()
pool.AppendCertsFromPEM(caData)
lock.Lock()
defer lock.Unlock()
tlsConfigs[getClusterKey(endpoints)] = &tls.Config{
Certificates: []tls.Certificate{cert},
RootCAs: pool,
InsecureSkipVerify: insecureSkipVerify,
}
return nil
}
// GetAccount gets the username/password for the given etcd cluster.
func GetAccount(endpoints []string) (Account, bool) {
lock.RLock()
defer lock.RUnlock()
account, ok := accounts[getClusterKey(endpoints)]
return account, ok
}
// GetTLS gets the tls config for the given etcd cluster.
func GetTLS(endpoints []string) (*tls.Config, bool) {
lock.RLock()
defer lock.RUnlock()
cfg, ok := tlsConfigs[getClusterKey(endpoints)]
return cfg, ok
}
@@ -0,0 +1,34 @@
package internal
import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/zeromicro/go-zero/core/stringx"
)
func TestAccount(t *testing.T) {
endpoints := []string{
"192.168.0.2:2379",
"192.168.0.3:2379",
"192.168.0.4:2379",
}
username := "foo" + stringx.Rand()
password := "bar"
anotherPassword := "any"
_, ok := GetAccount(endpoints)
assert.False(t, ok)
AddAccount(endpoints, username, password)
account, ok := GetAccount(endpoints)
assert.True(t, ok)
assert.Equal(t, username, account.User)
assert.Equal(t, password, account.Pass)
AddAccount(endpoints, username, anotherPassword)
account, ok = GetAccount(endpoints)
assert.True(t, ok)
assert.Equal(t, username, account.User)
assert.Equal(t, anotherPassword, account.Pass)
}
+3 -1
View File
@@ -1,13 +1,15 @@
//go:generate mockgen -package internal -destination etcdclient_mock.go -source etcdclient.go EtcdClient
package internal
import (
"context"
"go.etcd.io/etcd/clientv3"
clientv3 "go.etcd.io/etcd/client/v3"
"google.golang.org/grpc"
)
// EtcdClient interface represents an etcd client.
type EtcdClient interface {
ActiveConnection() *grpc.ClientConn
Close() error
+4 -3
View File
@@ -6,10 +6,11 @@ package internal
import (
context "context"
gomock "github.com/golang/mock/gomock"
clientv3 "go.etcd.io/etcd/clientv3"
grpc "google.golang.org/grpc"
reflect "reflect"
gomock "github.com/golang/mock/gomock"
clientv3 "go.etcd.io/etcd/client/v3"
grpc "google.golang.org/grpc"
)
// MockEtcdClient is a mock of EtcdClient interface
+2 -1
View File
@@ -1,5 +1,6 @@
package internal
// Listener interface wraps the OnUpdate method.
type Listener interface {
OnUpdate(keys []string, values []string, newKey string)
OnUpdate(keys, values []string, newKey string)
}
+74 -29
View File
@@ -9,49 +9,63 @@ import (
"sync"
"time"
"github.com/tal-tech/go-zero/core/contextx"
"github.com/tal-tech/go-zero/core/lang"
"github.com/tal-tech/go-zero/core/logx"
"github.com/tal-tech/go-zero/core/syncx"
"github.com/tal-tech/go-zero/core/threading"
"go.etcd.io/etcd/clientv3"
"github.com/zeromicro/go-zero/core/contextx"
"github.com/zeromicro/go-zero/core/lang"
"github.com/zeromicro/go-zero/core/logx"
"github.com/zeromicro/go-zero/core/syncx"
"github.com/zeromicro/go-zero/core/threading"
clientv3 "go.etcd.io/etcd/client/v3"
)
var (
registryInstance = Registry{
registry = Registry{
clusters: make(map[string]*cluster),
}
connManager = syncx.NewResourceManager()
)
// A Registry is a registry that manages the etcd client connections.
type Registry struct {
clusters map[string]*cluster
lock sync.Mutex
}
// GetRegistry returns a global Registry.
func GetRegistry() *Registry {
return &registryInstance
return &registry
}
func (r *Registry) getCluster(endpoints []string) *cluster {
// GetConn returns an etcd client connection associated with given endpoints.
func (r *Registry) GetConn(endpoints []string) (EtcdClient, error) {
c, _ := r.getCluster(endpoints)
return c.getClient()
}
// Monitor monitors the key on given etcd endpoints, notify with the given UpdateListener.
func (r *Registry) Monitor(endpoints []string, key string, l UpdateListener) error {
c, exists := r.getCluster(endpoints)
// if exists, the existing values should be updated to the listener.
if exists {
kvs := c.getCurrent(key)
for _, kv := range kvs {
l.OnAdd(kv)
}
}
return c.monitor(key, l)
}
func (r *Registry) getCluster(endpoints []string) (c *cluster, exists bool) {
clusterKey := getClusterKey(endpoints)
r.lock.Lock()
defer r.lock.Unlock()
c, ok := r.clusters[clusterKey]
if !ok {
c, exists = r.clusters[clusterKey]
if !exists {
c = newCluster(endpoints)
r.clusters[clusterKey] = c
}
return c
}
func (r *Registry) GetConn(endpoints []string) (EtcdClient, error) {
return r.getCluster(endpoints).getClient()
}
func (r *Registry) Monitor(endpoints []string, key string, l UpdateListener) error {
return r.getCluster(endpoints).monitor(key, l)
return
}
type cluster struct {
@@ -90,6 +104,21 @@ func (c *cluster) getClient() (EtcdClient, error) {
return val.(EtcdClient), nil
}
func (c *cluster) getCurrent(key string) []KV {
c.lock.Lock()
defer c.lock.Unlock()
var kvs []KV
for k, v := range c.values[key] {
kvs = append(kvs, KV{
Key: k,
Val: v,
})
}
return kvs
}
func (c *cluster) handleChanges(key string, kvs []KV) {
var add []KV
var remove []KV
@@ -193,14 +222,12 @@ func (c *cluster) load(cli EtcdClient, key string) {
}
var kvs []KV
c.lock.Lock()
for _, ev := range resp.Kvs {
kvs = append(kvs, KV{
Key: string(ev.Key),
Val: string(ev.Value),
})
}
c.lock.Unlock()
c.handleChanges(key, kvs)
}
@@ -256,26 +283,34 @@ func (c *cluster) reload(cli EtcdClient) {
}
func (c *cluster) watch(cli EtcdClient, key string) {
for {
if c.watchStream(cli, key) {
return
}
}
}
func (c *cluster) watchStream(cli EtcdClient, key string) bool {
rch := cli.Watch(clientv3.WithRequireLeader(c.context(cli)), makeKeyPrefix(key), clientv3.WithPrefix())
for {
select {
case wresp, ok := <-rch:
if !ok {
logx.Error("etcd monitor chan has been closed")
return
return false
}
if wresp.Canceled {
logx.Error("etcd monitor chan has been canceled")
return
logx.Errorf("etcd monitor chan has been canceled, error: %v", wresp.Err())
return false
}
if wresp.Err() != nil {
logx.Error(fmt.Sprintf("etcd monitor chan error: %v", wresp.Err()))
return
return false
}
c.handleWatchEvents(key, wresp.Events)
case <-c.done:
return
return true
}
}
}
@@ -288,15 +323,25 @@ func (c *cluster) watchConnState(cli EtcdClient) {
watcher.watch(cli.ActiveConnection())
}
// DialClient dials an etcd cluster with given endpoints.
func DialClient(endpoints []string) (EtcdClient, error) {
return clientv3.New(clientv3.Config{
cfg := clientv3.Config{
Endpoints: endpoints,
AutoSyncInterval: autoSyncInterval,
DialTimeout: DialTimeout,
DialKeepAliveTime: dialKeepAliveTime,
DialKeepAliveTimeout: DialTimeout,
RejectOldCluster: true,
})
}
if account, ok := GetAccount(endpoints); ok {
cfg.Username = account.User
cfg.Password = account.Pass
}
if tlsCfg, ok := GetTLS(endpoints); ok {
cfg.TLS = tlsCfg
}
return clientv3.New(cfg)
}
func getClusterKey(endpoints []string) string {
+16 -10
View File
@@ -7,11 +7,12 @@ import (
"github.com/golang/mock/gomock"
"github.com/stretchr/testify/assert"
"github.com/tal-tech/go-zero/core/contextx"
"github.com/tal-tech/go-zero/core/logx"
"github.com/tal-tech/go-zero/core/stringx"
"go.etcd.io/etcd/clientv3"
"go.etcd.io/etcd/mvcc/mvccpb"
"github.com/zeromicro/go-zero/core/contextx"
"github.com/zeromicro/go-zero/core/lang"
"github.com/zeromicro/go-zero/core/logx"
"github.com/zeromicro/go-zero/core/stringx"
"go.etcd.io/etcd/api/v3/mvccpb"
clientv3 "go.etcd.io/etcd/client/v3"
)
var mockLock sync.Mutex
@@ -32,9 +33,10 @@ func setMockClient(cli EtcdClient) func() {
}
func TestGetCluster(t *testing.T) {
c1 := GetRegistry().getCluster([]string{"first"})
c2 := GetRegistry().getCluster([]string{"second"})
c3 := GetRegistry().getCluster([]string{"first"})
AddAccount([]string{"first"}, "foo", "bar")
c1, _ := GetRegistry().getCluster([]string{"first"})
c2, _ := GetRegistry().getCluster([]string{"second"})
c3, _ := GetRegistry().getCluster([]string{"first"})
assert.Equal(t, c1, c3)
assert.NotEqual(t, c1, c2)
}
@@ -202,11 +204,13 @@ func TestClusterWatch_RespFailures(t *testing.T) {
restore := setMockClient(cli)
defer restore()
ch := make(chan clientv3.WatchResponse)
cli.EXPECT().Watch(gomock.Any(), "any/", gomock.Any()).Return(ch)
cli.EXPECT().Watch(gomock.Any(), "any/", gomock.Any()).Return(ch).AnyTimes()
cli.EXPECT().Ctx().Return(context.Background()).AnyTimes()
c := new(cluster)
c.done = make(chan lang.PlaceholderType)
go func() {
ch <- resp
close(c.done)
}()
c.watch(cli, "any")
})
@@ -220,11 +224,13 @@ func TestClusterWatch_CloseChan(t *testing.T) {
restore := setMockClient(cli)
defer restore()
ch := make(chan clientv3.WatchResponse)
cli.EXPECT().Watch(gomock.Any(), "any/", gomock.Any()).Return(ch)
cli.EXPECT().Watch(gomock.Any(), "any/", gomock.Any()).Return(ch).AnyTimes()
cli.EXPECT().Ctx().Return(context.Background()).AnyTimes()
c := new(cluster)
c.done = make(chan lang.PlaceholderType)
go func() {
close(ch)
close(c.done)
}()
c.watch(cli, "any")
}
+1
View File
@@ -1,4 +1,5 @@
//go:generate mockgen -package internal -destination statewatcher_mock.go -source statewatcher.go etcdConn
package internal
import (
+2 -1
View File
@@ -6,9 +6,10 @@ package internal
import (
context "context"
reflect "reflect"
gomock "github.com/golang/mock/gomock"
connectivity "google.golang.org/grpc/connectivity"
reflect "reflect"
)
// MocketcdConn is a mock of etcdConn interface
+3
View File
@@ -1,12 +1,15 @@
//go:generate mockgen -package internal -destination updatelistener_mock.go -source updatelistener.go UpdateListener
package internal
type (
// A KV is used to store an etcd entry with key and value.
KV struct {
Key string
Val string
}
// UpdateListener wraps the OnAdd and OnDelete methods.
UpdateListener interface {
OnAdd(kv KV)
OnDelete(kv KV)
+2 -1
View File
@@ -5,8 +5,9 @@
package internal
import (
gomock "github.com/golang/mock/gomock"
reflect "reflect"
gomock "github.com/golang/mock/gomock"
)
// MockUpdateListener is a mock of UpdateListener interface
+8 -3
View File
@@ -3,17 +3,22 @@ package internal
import "time"
const (
// Delimiter is a separator that separates the etcd path.
Delimiter = '/'
autoSyncInterval = time.Minute
coolDownInterval = time.Second
dialTimeout = 5 * time.Second
dialKeepAliveTime = 5 * time.Second
requestTimeout = 3 * time.Second
Delimiter = '/'
endpointsSeparator = ","
)
var (
DialTimeout = dialTimeout
// DialTimeout is the dial timeout.
DialTimeout = dialTimeout
// RequestTimeout is the request timeout.
RequestTimeout = requestTimeout
NewClient = DialClient
// NewClient is used to create etcd clients.
NewClient = DialClient
)
@@ -0,0 +1,64 @@
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: "etcd"
namespace: discov
labels:
app: "etcd"
spec:
serviceName: "etcd"
replicas: 5
template:
metadata:
name: "etcd"
labels:
app: "etcd"
spec:
volumes:
- name: etcd-pvc
persistentVolumeClaim:
claimName: etcd-pvc
containers:
- name: "etcd"
image: quay.io/coreos/etcd:latest
ports:
- containerPort: 2379
name: client
- containerPort: 2380
name: peer
env:
- name: CLUSTER_SIZE
value: "5"
- name: SET_NAME
value: "etcd"
- name: VOLNAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.name
volumeMounts:
- name: etcd-pvc
mountPath: /var/lib/etcd
subPathExpr: $(VOLNAME) # data mounted respectively in each pod
command:
- "/bin/sh"
- "-ecx"
- |
chmod 700 /var/lib/etcd
IP=$(hostname -i)
PEERS=""
for i in $(seq 0 $((${CLUSTER_SIZE} - 1))); do
PEERS="${PEERS}${PEERS:+,}${SET_NAME}-${i}=http://${SET_NAME}-${i}.${SET_NAME}:2380"
done
exec etcd --name ${HOSTNAME} \
--listen-peer-urls http://0.0.0.0:2380 \
--listen-client-urls http://0.0.0.0:2379 \
--advertise-client-urls http://${HOSTNAME}.${SET_NAME}.discov:2379 \
--initial-advertise-peer-urls http://${HOSTNAME}.${SET_NAME}:2380 \
--initial-cluster ${PEERS} \
--initial-cluster-state new \
--logger zap \
--data-dir /var/lib/etcd \
--auto-compaction-retention 1
+35 -10
View File
@@ -1,18 +1,20 @@
package discov
import (
"github.com/tal-tech/go-zero/core/discov/internal"
"github.com/tal-tech/go-zero/core/lang"
"github.com/tal-tech/go-zero/core/logx"
"github.com/tal-tech/go-zero/core/proc"
"github.com/tal-tech/go-zero/core/syncx"
"github.com/tal-tech/go-zero/core/threading"
"go.etcd.io/etcd/clientv3"
"github.com/zeromicro/go-zero/core/discov/internal"
"github.com/zeromicro/go-zero/core/lang"
"github.com/zeromicro/go-zero/core/logx"
"github.com/zeromicro/go-zero/core/proc"
"github.com/zeromicro/go-zero/core/syncx"
"github.com/zeromicro/go-zero/core/threading"
clientv3 "go.etcd.io/etcd/client/v3"
)
type (
PublisherOption func(client *Publisher)
// PubOption defines the method to customize a Publisher.
PubOption func(client *Publisher)
// A Publisher can be used to publish the value to an etcd cluster on the given key.
Publisher struct {
endpoints []string
key string
@@ -26,7 +28,11 @@ type (
}
)
func NewPublisher(endpoints []string, key, value string, opts ...PublisherOption) *Publisher {
// NewPublisher returns a Publisher.
// endpoints is the hosts of the etcd cluster.
// key:value are a pair to be published.
// opts are used to customize the Publisher.
func NewPublisher(endpoints []string, key, value string, opts ...PubOption) *Publisher {
publisher := &Publisher{
endpoints: endpoints,
key: key,
@@ -43,6 +49,7 @@ func NewPublisher(endpoints []string, key, value string, opts ...PublisherOption
return publisher
}
// KeepAlive keeps key:value alive.
func (p *Publisher) KeepAlive() error {
cli, err := internal.GetRegistry().GetConn(p.endpoints)
if err != nil {
@@ -61,14 +68,17 @@ func (p *Publisher) KeepAlive() error {
return p.keepAliveAsync(cli)
}
// Pause pauses the renewing of key:value.
func (p *Publisher) Pause() {
p.pauseChan <- lang.Placeholder
}
// Resume resumes the renewing of key:value.
func (p *Publisher) Resume() {
p.resumeChan <- lang.Placeholder
}
// Stop stops the renewing and revokes the registration.
func (p *Publisher) Stop() {
p.quit.Close()
}
@@ -135,8 +145,23 @@ func (p *Publisher) revoke(cli internal.EtcdClient) {
}
}
func WithId(id int64) PublisherOption {
// WithId customizes a Publisher with the id.
func WithId(id int64) PubOption {
return func(publisher *Publisher) {
publisher.id = id
}
}
// WithPubEtcdAccount provides the etcd username/password.
func WithPubEtcdAccount(user, pass string) PubOption {
return func(pub *Publisher) {
RegisterAccount(pub.endpoints, user, pass)
}
}
// WithPubEtcdTLS provides the etcd CertFile/CertKeyFile/CACertFile.
func WithPubEtcdTLS(certFile, certKeyFile, caFile string, insecureSkipVerify bool) PubOption {
return func(pub *Publisher) {
logx.Must(RegisterTLS(pub.endpoints, certFile, certKeyFile, caFile, insecureSkipVerify))
}
}
+21 -4
View File
@@ -4,12 +4,15 @@ import (
"errors"
"sync"
"testing"
"time"
"github.com/golang/mock/gomock"
"github.com/stretchr/testify/assert"
"github.com/tal-tech/go-zero/core/discov/internal"
"github.com/tal-tech/go-zero/core/logx"
"go.etcd.io/etcd/clientv3"
"github.com/zeromicro/go-zero/core/discov/internal"
"github.com/zeromicro/go-zero/core/lang"
"github.com/zeromicro/go-zero/core/logx"
"github.com/zeromicro/go-zero/core/stringx"
clientv3 "go.etcd.io/etcd/client/v3"
)
func init() {
@@ -28,7 +31,8 @@ func TestPublisher_register(t *testing.T) {
ID: id,
}, nil)
cli.EXPECT().Put(gomock.Any(), makeEtcdKey("thekey", id), "thevalue", gomock.Any())
pub := NewPublisher(nil, "thekey", "thevalue")
pub := NewPublisher(nil, "thekey", "thevalue",
WithPubEtcdAccount(stringx.Rand(), "bar"))
_, err := pub.register(cli)
assert.Nil(t, err)
}
@@ -152,3 +156,16 @@ func TestPublisher_keepAliveAsyncPause(t *testing.T) {
pub.Pause()
wg.Wait()
}
func TestPublisher_Resume(t *testing.T) {
publisher := new(Publisher)
publisher.resumeChan = make(chan lang.PlaceholderType)
go func() {
publisher.Resume()
}()
go func() {
time.Sleep(time.Minute)
t.Fail()
}()
<-publisher.resumeChan
}
+40 -19
View File
@@ -4,31 +4,36 @@ import (
"sync"
"sync/atomic"
"github.com/tal-tech/go-zero/core/discov/internal"
"github.com/tal-tech/go-zero/core/syncx"
"github.com/zeromicro/go-zero/core/discov/internal"
"github.com/zeromicro/go-zero/core/logx"
"github.com/zeromicro/go-zero/core/syncx"
)
type (
subOptions struct {
exclusive bool
}
SubOption func(opts *subOptions)
// SubOption defines the method to customize a Subscriber.
SubOption func(sub *Subscriber)
// A Subscriber is used to subscribe the given key on a etcd cluster.
Subscriber struct {
items *container
endpoints []string
exclusive bool
items *container
}
)
// NewSubscriber returns a Subscriber.
// endpoints is the hosts of the etcd cluster.
// key is the key to subscribe.
// opts are used to customize the Subscriber.
func NewSubscriber(endpoints []string, key string, opts ...SubOption) (*Subscriber, error) {
var subOpts subOptions
for _, opt := range opts {
opt(&subOpts)
}
sub := &Subscriber{
items: newContainer(subOpts.exclusive),
endpoints: endpoints,
}
for _, opt := range opts {
opt(sub)
}
sub.items = newContainer(sub.exclusive)
if err := internal.GetRegistry().Monitor(endpoints, key, sub.items); err != nil {
return nil, err
}
@@ -36,19 +41,35 @@ func NewSubscriber(endpoints []string, key string, opts ...SubOption) (*Subscrib
return sub, nil
}
// AddListener adds listener to s.
func (s *Subscriber) AddListener(listener func()) {
s.items.addListener(listener)
}
// Values returns all the subscription values.
func (s *Subscriber) Values() []string {
return s.items.getValues()
}
// exclusive means that key value can only be 1:1,
// Exclusive means that key value can only be 1:1,
// which means later added value will remove the keys associated with the same value previously.
func Exclusive() SubOption {
return func(opts *subOptions) {
opts.exclusive = true
return func(sub *Subscriber) {
sub.exclusive = true
}
}
// WithSubEtcdAccount provides the etcd username/password.
func WithSubEtcdAccount(user, pass string) SubOption {
return func(sub *Subscriber) {
RegisterAccount(sub.endpoints, user, pass)
}
}
// WithSubEtcdTLS provides the etcd CertFile/CertKeyFile/CACertFile.
func WithSubEtcdTLS(certFile, certKeyFile, caFile string, insecureSkipVerify bool) SubOption {
return func(sub *Subscriber) {
logx.Must(RegisterTLS(sub.endpoints, certFile, certKeyFile, caFile, insecureSkipVerify))
}
}
@@ -100,9 +121,9 @@ func (c *container) addKv(key, value string) ([]string, bool) {
if early {
return previous, true
} else {
return nil, false
}
return nil, false
}
func (c *container) addListener(listener func()) {
+28 -1
View File
@@ -1,10 +1,12 @@
package discov
import (
"sync/atomic"
"testing"
"github.com/stretchr/testify/assert"
"github.com/tal-tech/go-zero/core/discov/internal"
"github.com/zeromicro/go-zero/core/discov/internal"
"github.com/zeromicro/go-zero/core/stringx"
)
const (
@@ -198,3 +200,28 @@ func TestContainer(t *testing.T) {
}
}
}
func TestSubscriber(t *testing.T) {
sub := new(Subscriber)
Exclusive()(sub)
sub.items = newContainer(sub.exclusive)
var count int32
sub.AddListener(func() {
atomic.AddInt32(&count, 1)
})
sub.items.notifyChange()
assert.Empty(t, sub.Values())
assert.Equal(t, int32(1), atomic.LoadInt32(&count))
}
func TestWithSubEtcdAccount(t *testing.T) {
endpoints := []string{"localhost:2379"}
user := stringx.Rand()
WithSubEtcdAccount(user, "bar")(&Subscriber{
endpoints: endpoints,
})
account, ok := internal.GetAccount(endpoints)
assert.True(t, ok)
assert.Equal(t, user, account.User)
assert.Equal(t, "bar", account.Pass)
}
+6 -1
View File
@@ -2,14 +2,19 @@ package errorx
import "sync/atomic"
// AtomicError defines an atomic error.
type AtomicError struct {
err atomic.Value // error
}
// Set sets the error.
func (ae *AtomicError) Set(err error) {
ae.err.Store(err)
if err != nil {
ae.err.Store(err)
}
}
// Load returns the error.
func (ae *AtomicError) Load() error {
if v := ae.err.Load(); v != nil {
return v.(error)
+9
View File
@@ -17,6 +17,15 @@ func TestAtomicError(t *testing.T) {
assert.Equal(t, errDummy, err.Load())
}
func TestAtomicErrorSetNil(t *testing.T) {
var (
errNil error
err AtomicError
)
err.Set(errNil)
assert.Equal(t, errNil, err.Load())
}
func TestAtomicErrorNil(t *testing.T) {
var err AtomicError
assert.Nil(t, err.Load())
+10 -3
View File
@@ -3,6 +3,7 @@ package errorx
import "bytes"
type (
// A BatchError is an error that can hold multiple errors.
BatchError struct {
errs errorArray
}
@@ -10,12 +11,16 @@ type (
errorArray []error
)
func (be *BatchError) Add(err error) {
if err != nil {
be.errs = append(be.errs, err)
// Add adds errs to be, nil errors are ignored.
func (be *BatchError) Add(errs ...error) {
for _, err := range errs {
if err != nil {
be.errs = append(be.errs, err)
}
}
}
// Err returns an error that represents all errors.
func (be *BatchError) Err() error {
switch len(be.errs) {
case 0:
@@ -27,10 +32,12 @@ func (be *BatchError) Err() error {
}
}
// NotNil checks if any error inside.
func (be *BatchError) NotNil() bool {
return len(be.errs) > 0
}
// Error returns a string that represents inside errors.
func (ea errorArray) Error() string {
var buf bytes.Buffer
+1
View File
@@ -1,5 +1,6 @@
package errorx
// Chain runs funs one by one until an error occurred.
func Chain(fns ...func() error) error {
for _, fn := range fns {
if err := fn(); err != nil {
+1 -1
View File
@@ -8,7 +8,7 @@ import (
)
func TestChain(t *testing.T) {
var errDummy = errors.New("dummy")
errDummy := errors.New("dummy")
assert.Nil(t, Chain(func() error {
return nil
}, func() error {
+10
View File
@@ -5,8 +5,12 @@ import "time"
const defaultBulkTasks = 1000
type (
// BulkOption defines the method to customize a BulkExecutor.
BulkOption func(options *bulkOptions)
// A BulkExecutor is an executor that can execute tasks on either requirement meets:
// 1. up to given size of tasks
// 2. flush interval time elapsed
BulkExecutor struct {
executor *PeriodicalExecutor
container *bulkContainer
@@ -18,6 +22,7 @@ type (
}
)
// NewBulkExecutor returns a BulkExecutor.
func NewBulkExecutor(execute Execute, opts ...BulkOption) *BulkExecutor {
options := newBulkOptions()
for _, opt := range opts {
@@ -36,25 +41,30 @@ func NewBulkExecutor(execute Execute, opts ...BulkOption) *BulkExecutor {
return executor
}
// Add adds task into be.
func (be *BulkExecutor) Add(task interface{}) error {
be.executor.Add(task)
return nil
}
// Flush forces be to flush and execute tasks.
func (be *BulkExecutor) Flush() {
be.executor.Flush()
}
// Wait waits be to done with the task execution.
func (be *BulkExecutor) Wait() {
be.executor.Wait()
}
// WithBulkTasks customizes a BulkExecutor with given tasks limit.
func WithBulkTasks(tasks int) BulkOption {
return func(options *bulkOptions) {
options.cachedTasks = tasks
}
}
// WithBulkInterval customizes a BulkExecutor with given flush interval.
func WithBulkInterval(duration time.Duration) BulkOption {
return func(options *bulkOptions) {
options.flushInterval = duration
+10
View File
@@ -5,8 +5,12 @@ import "time"
const defaultChunkSize = 1024 * 1024 // 1M
type (
// ChunkOption defines the method to customize a ChunkExecutor.
ChunkOption func(options *chunkOptions)
// A ChunkExecutor is an executor to execute tasks when either requirement meets:
// 1. up to given chunk size
// 2. flush interval elapsed
ChunkExecutor struct {
executor *PeriodicalExecutor
container *chunkContainer
@@ -18,6 +22,7 @@ type (
}
)
// NewChunkExecutor returns a ChunkExecutor.
func NewChunkExecutor(execute Execute, opts ...ChunkOption) *ChunkExecutor {
options := newChunkOptions()
for _, opt := range opts {
@@ -36,6 +41,7 @@ func NewChunkExecutor(execute Execute, opts ...ChunkOption) *ChunkExecutor {
return executor
}
// Add adds task with given chunk size into ce.
func (ce *ChunkExecutor) Add(task interface{}, size int) error {
ce.executor.Add(chunk{
val: task,
@@ -44,20 +50,24 @@ func (ce *ChunkExecutor) Add(task interface{}, size int) error {
return nil
}
// Flush forces ce to flush and execute tasks.
func (ce *ChunkExecutor) Flush() {
ce.executor.Flush()
}
// Wait waits the execution to be done.
func (ce *ChunkExecutor) Wait() {
ce.executor.Wait()
}
// WithChunkBytes customizes a ChunkExecutor with the given chunk size.
func WithChunkBytes(size int) ChunkOption {
return func(options *chunkOptions) {
options.chunkSize = size
}
}
// WithFlushInterval customizes a ChunkExecutor with the given flush interval.
func WithFlushInterval(duration time.Duration) ChunkOption {
return func(options *chunkOptions) {
options.flushInterval = duration
+4 -1
View File
@@ -4,9 +4,10 @@ import (
"sync"
"time"
"github.com/tal-tech/go-zero/core/threading"
"github.com/zeromicro/go-zero/core/threading"
)
// A DelayExecutor delays a tasks on given delay interval.
type DelayExecutor struct {
fn func()
delay time.Duration
@@ -14,6 +15,7 @@ type DelayExecutor struct {
lock sync.Mutex
}
// NewDelayExecutor returns a DelayExecutor with given fn and delay.
func NewDelayExecutor(fn func(), delay time.Duration) *DelayExecutor {
return &DelayExecutor{
fn: fn,
@@ -21,6 +23,7 @@ func NewDelayExecutor(fn func(), delay time.Duration) *DelayExecutor {
}
}
// Trigger triggers the task to be executed after given delay, safe to trigger more than once.
func (de *DelayExecutor) Trigger() {
de.lock.Lock()
defer de.lock.Unlock()
+6 -2
View File
@@ -3,15 +3,17 @@ package executors
import (
"time"
"github.com/tal-tech/go-zero/core/syncx"
"github.com/tal-tech/go-zero/core/timex"
"github.com/zeromicro/go-zero/core/syncx"
"github.com/zeromicro/go-zero/core/timex"
)
// A LessExecutor is an executor to limit execution once within given time interval.
type LessExecutor struct {
threshold time.Duration
lastTime *syncx.AtomicDuration
}
// NewLessExecutor returns a LessExecutor with given threshold as time interval.
func NewLessExecutor(threshold time.Duration) *LessExecutor {
return &LessExecutor{
threshold: threshold,
@@ -19,6 +21,8 @@ func NewLessExecutor(threshold time.Duration) *LessExecutor {
}
}
// DoOrDiscard executes or discards the task depends on if
// another task was executed within the time interval.
func (le *LessExecutor) DoOrDiscard(execute func()) bool {
now := timex.Now()
lastTime := le.lastTime.Load()
+1 -1
View File
@@ -5,7 +5,7 @@ import (
"time"
"github.com/stretchr/testify/assert"
"github.com/tal-tech/go-zero/core/timex"
"github.com/zeromicro/go-zero/core/timex"
)
func TestLessExecutor_DoOrDiscard(t *testing.T) {
+13 -7
View File
@@ -6,17 +6,17 @@ import (
"sync/atomic"
"time"
"github.com/tal-tech/go-zero/core/lang"
"github.com/tal-tech/go-zero/core/proc"
"github.com/tal-tech/go-zero/core/syncx"
"github.com/tal-tech/go-zero/core/threading"
"github.com/tal-tech/go-zero/core/timex"
"github.com/zeromicro/go-zero/core/lang"
"github.com/zeromicro/go-zero/core/proc"
"github.com/zeromicro/go-zero/core/syncx"
"github.com/zeromicro/go-zero/core/threading"
"github.com/zeromicro/go-zero/core/timex"
)
const idleRound = 10
type (
// A type that satisfies executors.TaskContainer can be used as the underlying
// TaskContainer interface defines a type that can be used as the underlying
// container that used to do periodical executions.
TaskContainer interface {
// AddTask adds the task into the container.
@@ -28,6 +28,7 @@ type (
RemoveAll() interface{}
}
// A PeriodicalExecutor is an executor that periodically execute tasks.
PeriodicalExecutor struct {
commander chan interface{}
interval time.Duration
@@ -43,6 +44,7 @@ type (
}
)
// NewPeriodicalExecutor returns a PeriodicalExecutor with given interval and container.
func NewPeriodicalExecutor(interval time.Duration, container TaskContainer) *PeriodicalExecutor {
executor := &PeriodicalExecutor{
// buffer 1 to let the caller go quickly
@@ -51,7 +53,7 @@ func NewPeriodicalExecutor(interval time.Duration, container TaskContainer) *Per
container: container,
confirmChan: make(chan lang.PlaceholderType),
newTicker: func(d time.Duration) timex.Ticker {
return timex.NewTicker(interval)
return timex.NewTicker(d)
},
}
proc.AddShutdownListener(func() {
@@ -61,6 +63,7 @@ func NewPeriodicalExecutor(interval time.Duration, container TaskContainer) *Per
return executor
}
// Add adds tasks into pe.
func (pe *PeriodicalExecutor) Add(task interface{}) {
if vals, ok := pe.addAndCheck(task); ok {
pe.commander <- vals
@@ -68,6 +71,7 @@ func (pe *PeriodicalExecutor) Add(task interface{}) {
}
}
// Flush forces pe to execute tasks.
func (pe *PeriodicalExecutor) Flush() bool {
pe.enterExecution()
return pe.executeTasks(func() interface{} {
@@ -77,12 +81,14 @@ func (pe *PeriodicalExecutor) Flush() bool {
}())
}
// Sync lets caller to run fn thread-safe with pe, especially for the underlying container.
func (pe *PeriodicalExecutor) Sync(fn func()) {
pe.lock.Lock()
defer pe.lock.Unlock()
fn()
}
// Wait waits the execution to be done.
func (pe *PeriodicalExecutor) Wait() {
pe.Flush()
pe.wgBarrier.Guard(func() {
+1 -1
View File
@@ -8,7 +8,7 @@ import (
"time"
"github.com/stretchr/testify/assert"
"github.com/tal-tech/go-zero/core/timex"
"github.com/zeromicro/go-zero/core/timex"
)
const threshold = 10
+1
View File
@@ -4,4 +4,5 @@ import "time"
const defaultFlushInterval = time.Second
// Execute defines the method to execute tasks.
type Execute func(tasks []interface{})
+4 -2
View File
@@ -7,6 +7,7 @@ import (
const bufSize = 1024
// FirstLine returns the first line of the file.
func FirstLine(filename string) (string, error) {
file, err := os.Open(filename)
if err != nil {
@@ -17,6 +18,7 @@ func FirstLine(filename string) (string, error) {
return firstLine(file)
}
// LastLine returns the last line of the file.
func LastLine(filename string) (string, error) {
file, err := os.Open(filename)
if err != nil {
@@ -69,11 +71,11 @@ func lastLine(filename string, file *os.File) (string, error) {
if buf[n-1] == '\n' {
buf = buf[:n-1]
n -= 1
n--
} else {
buf = buf[:n]
}
for n -= 1; n >= 0; n-- {
for n--; n >= 0; n-- {
if buf[n] == '\n' {
return string(append(buf[n+1:], last...)), nil
}
+1 -1
View File
@@ -5,7 +5,7 @@ import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/tal-tech/go-zero/core/fs"
"github.com/zeromicro/go-zero/core/fs"
)
const (
+3
View File
@@ -5,12 +5,15 @@ import (
"os"
)
// OffsetRange represents a content block of a file.
type OffsetRange struct {
File string
Start int64
Stop int64
}
// SplitLineChunks splits file into chunks.
// The whole line are guaranteed to be split in the same chunk.
func SplitLineChunks(filename string, chunks int) ([]OffsetRange, error) {
info, err := os.Stat(filename)
if err != nil {
+1 -1
View File
@@ -5,7 +5,7 @@ import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/tal-tech/go-zero/core/fs"
"github.com/zeromicro/go-zero/core/fs"
)
func TestSplitLineChunks(t *testing.T) {
+4
View File
@@ -3,8 +3,11 @@ package filex
import "gopkg.in/cheggaaa/pb.v1"
type (
// A Scanner is used to read lines.
Scanner interface {
// Scan checks if has remaining to read.
Scan() bool
// Text returns next line.
Text() string
}
@@ -14,6 +17,7 @@ type (
}
)
// NewProgressScanner returns a Scanner with progress indicator.
func NewProgressScanner(scanner Scanner, bar *pb.ProgressBar) Scanner {
return &progressScanner{
Scanner: scanner,
+7 -1
View File
@@ -5,12 +5,17 @@ import (
"os"
)
// errExceedFileSize indicates that the file size is exceeded.
var errExceedFileSize = errors.New("exceed file size")
// A RangeReader is used to read a range of content from a file.
type RangeReader struct {
file *os.File
start int64
stop int64
}
// NewRangeReader returns a RangeReader, which will read the range of content from file.
func NewRangeReader(file *os.File, start, stop int64) *RangeReader {
return &RangeReader{
file: file,
@@ -19,6 +24,7 @@ func NewRangeReader(file *os.File, start, stop int64) *RangeReader {
}
}
// Read reads the range of content into p.
func (rr *RangeReader) Read(p []byte) (n int, err error) {
stat, err := rr.file.Stat()
if err != nil {
@@ -26,7 +32,7 @@ func (rr *RangeReader) Read(p []byte) (n int, err error) {
}
if rr.stop < rr.start || rr.start >= stat.Size() {
return 0, errors.New("exceed file size")
return 0, errExceedFileSize
}
if rr.stop-rr.start < int64(len(p)) {
+1 -1
View File
@@ -5,7 +5,7 @@ import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/tal-tech/go-zero/core/fs"
"github.com/zeromicro/go-zero/core/fs"
)
func TestRangeReader(t *testing.T) {
+1
View File
@@ -1,3 +1,4 @@
//go:build windows
// +build windows
package fs
+2
View File
@@ -1,3 +1,4 @@
//go:build linux || darwin
// +build linux darwin
package fs
@@ -7,6 +8,7 @@ import (
"syscall"
)
// CloseOnExec makes sure closing the file on process forking.
func CloseOnExec(file *os.File) {
if file != nil {
syscall.CloseOnExec(int(file.Fd()))

Some files were not shown because too many files have changed in this diff Show More