XeLabs/sonic
2
0
Fork 0
mirror of https://github.com/ii64/sonic.git synced 2026-06-21 00:46:43 +08:00
Code Issues Projects Releases Wiki Activity
129 commits 1 branch 6 tags 7.4 MiB
Commit graph

129 commits

This branch
This branch
All branches
Author SHA1 Message Date
Yi Duan
b36771ba37
feat: support Go 1.19 (#275) 
* feat: support Go 1.19

* fuzz test refactor

* fix fuzz bug, generate pointer typ

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
 2022-08-09 12:54:11 +08:00
Yi Duan
8b51e75241
fix:(encoder) insufficient buffer size check for OP_i32 (#269) 2022-07-28 17:18:53 +08:00
Yi Duan
2eae594741
fix:(native) StreamDecoder returns error when run out of buffer while skiping value (#267) 
* fix:(native) StreamDecoder return error when run out of input buffer while skiping value

* refactor
 2022-07-25 15:17:52 +08:00
liu
34026b9c65
fix: useint64 in decode SetOption (#268) 
* fix: useint64 in decode SetOption

* doc: add comments on `sonic.Config`

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
 2022-07-25 14:26:18 +08:00
Yi Duan
755c0252a9
feat: (ast) support casting V_STRING to number (#262) 2022-07-12 16:27:27 +08:00
Yi Duan
07d7b867d4
feat (encoder): add encoder option NoNullSliceOrMap (#218) 
* feat (encoder): add encoder option `NoNullSliceOrMap`

* feat: add option on `sonic.Config`

* build: specify `self-host` to x64 machine
 2022-07-08 15:07:42 +08:00
Yi Duan
08c7640684
fix:(decoder) didn't check ',' before object key due to imperfect FSM (#261) 2022-07-08 11:12:34 +08:00
Yi Duan
a66db08b94
fix:(encoder) handle map like pointer when calling call_marshaler_v (#259) 2022-07-06 14:50:28 +08:00
Yi Duan
55e3d10619
fix: (decoder) return io.EOF when read nothing from io.Reader (#254) 
* fix: (decoder) return io.EOF when read nothing from io.Reader

* build:(CI) codeql-analysis only use x86 machine
 2022-07-01 17:51:42 +08:00
Yi Duan
fc6f3743fd
fix:(ast) checkRaw() check if node is nil pointer first (#250) 2022-06-28 14:16:10 +08:00
Yi Duan
2e751bf5db
feat:(ast) Add more strict casting API (#249) 
* feat:(ast) Add more strict API

* test: add cast test
 2022-06-28 13:27:07 +08:00
liu
f8fb04a184
fix(ub): marshal escape b f as encoding/json (#243) 
* fix(ub): marshal b f as encoding/json

* test: fix old test for escape b

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
 2022-06-10 11:21:35 +08:00
liu
410625ca9e
opt: remove bound and loop unrolling in quote (begin to use clang13) (#237) 
* opt: remove most bound check in quote

* opt: remove branch in copied escaped chars

* opt: use cache-friendly bool array

* opt: use loop unrolling in quote

* feat: add `-t` for bench.py

Change-Id: I3f41218187597126ef960eab09c6fa6fe252f347

* test:(bench) adjust repeat_time

* test: add Diff on CI

* test: update bench.py

* doc: update README.md

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
 2022-06-08 21:50:10 +08:00
liu
3717c62ec2
test: add generic benchmark (#239) 
* test: add generic benchmark

* test: add ci

* chore: adjust generic benchmark variable name

* ci: fix compare bug in bench.py

* build: adjust CI yaml

* test: use sonic.Config

* chore: generic test

* test: add interface type bench

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
 2022-06-07 13:26:28 +08:00
Yi Duan
ced28302a9
feat: add compatible API (#233) 
* feat: add compatible API

* test: add api compatibility test

* fmt

* test: adjust CI

* fix: no trailing '\n' for std `Marshal()`

* doc: update README.md

* test: add windows CI

* doc: update README.md

* doc: update README.md

* fmt: add comments and refator
 2022-05-25 14:16:56 +08:00
liu
8917a0a7d9
chore: add script to bench different git branches (#232) 
Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
 2022-05-24 14:13:07 +08:00
Yi Duan
f9d033d6f8
feat: support streaming IO (#225) 
* feat (decoder): support streaming IO

* opt: fine tune buffer size factors

* feat (encoder): support streaming IO

* doc: refactor format and add comments

* fix: io.EOF dead loop

* build: adjust CI files

* doc: update README.md
 2022-05-24 10:38:43 +08:00
Ii64人
75b728ba51
feat: support for Windows (#228) 
* feat: support for Windows

* fix (loader): default loader now has build tag: for linux and darwin
* feat (loader): add memory allocator support for Windows

* feat: add Windows CI

Added Windows CI for GitHub Actions
Go version: 1.15.x, 1.16.x, 1.17.x, 1.18.x

* fix (loader): remove `amd64` build constraint from Windows

* fix: TSAN error on Windows CI

Temporary remove `-race` flag

Link: https://github.com/golang/go/issues/46099

Co-authored-by: Yi Duan <duanyi.aster@bytedance.com>
 2022-05-23 12:55:45 +08:00
Yi Duan
308b76b44f
fix (encoder): spill RL (buffer length) register while calling internal encoders in case of it got scratched (#230) 
Change-Id: Ie15c589b739a7b3feb93c5a7b65e82c114d7ab80
 2022-05-23 12:27:29 +08:00
Yi Duan
5086cb21fe
fix: add nil pointer check (#224) 
* fix: add nil pointer check

* panicNilPointerOfNonEmptyString
 2022-05-19 13:24:57 +08:00
liu
d923a03329
fix: some typos (#221) 
Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
 2022-05-19 11:41:01 +08:00
Yi Duan
9aab4704ef
fix: issux213 (#215) 
* fix (decoder): use AVX mode only on base64x
 2022-04-21 15:29:52 +08:00
liu
2c8299dd44
chore: update readme (#220) 
Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
 2022-04-21 14:14:24 +08:00
liu
57dea971c6
chore: update readme (#219) 2022-04-21 03:35:32 +00:00
liu
57989f38ba
opt: skip number in json.Number parsing (#209) 
* opt: skip number in json.Number  parsing

* fix: generic use skip_number

* test: add json.Number decoding benchmarks

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
 2022-04-01 17:45:06 +08:00
Yi Duan
fe2497d01e
fix:(loader) init _Moduledata.gcdatapointer to avoid panic on plugin-mode (#188) 
* fix:(loader) init `_Moduledata.gcdata`pointer to avoid panic on plugin-mode

* fix: freeze gcdata pointer

* feat: go115

* update test

* fix: use unique bytes pointer

* test: drop go1.15 race test

* feat: support go1.18
 2022-04-01 16:46:07 +08:00
liu
d9b9980291
test: add fuzz test in Go 1.18 (#208) 2022-04-01 15:04:54 +08:00
Yi Duan
cb1702dc8c
feat: support Go1.18 (#205) 
* feat: support Go1.18

* fix: re-encoding `moduledata.pclntab`

* test: add generic func wrap test
 2022-03-28 20:43:43 +08:00
Yi Duan
94232a7b2e
fix: ignore overflow error when decoding json.Number (#207) 2022-03-24 16:00:08 +08:00
Yi Duan
c381c8b679
feat: add API MarshalString() (#197) 2022-03-01 15:09:09 +08:00
Yi Duan
f087445eee
fix: allow string value decoded as json.Number (#196) 
* fix: allow string value decoded as `json.Number`

* test: move -race flag from Go1.15 since it reports "race: limit on 8128 simultaneously alive goroutines is exceeded, dying"

* test: add benchmarks
 2022-02-28 20:14:38 +08:00
Yi Duan
b66168fa77
feat:(decoder) add option CopyString() (#192) 
* feat: add option `CopyString()`

* test: diff copy and nocopy decoding bench

* doc: update README.md

* opt: use global link for `escape_string()`

* feat: copy JSON on `Unmarshal()`

* test: add norace test

* fix: nocopy field key

* test: add generic test

* fix: remove useless instruction
 2022-02-28 18:46:25 +08:00
Yi Duan
8caa4eef05
fix: check nil pointer for Marshaler (#194) 2022-02-25 18:00:57 +08:00
liu
ce41dd9636
fix: correct htmlescape chars check (#190) 
* fix: correct htmlescape chars check

* fix: continuous html escape chars

* tmp: compile clang13

* fix: check single xe2 char

* fix: htmlescape no mem test

* fix: htmlescape zero check bug

* fix: htmlescape bug

* build: clang12

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
 2022-02-22 11:43:58 +08:00
liu
7475b256ce
support JSON validate (#189) 
* fix: check unescaped control chars in decode

* feat: add utf8 validate func

* feat: validate utf8 in json string

* feat: add validateone api

* fix: check unicode pointer for surrogate

* clang12 compile

* feat: Import `Valid()` and `Skip()`

* opt: use looktable

* fix utf-8 validate performance problem

* fix: utf-8 validate bug

* clang12 build

* feat: (encoder) accelerate validating json from `json.Marshaler`

chore!:
- `encoder.NoCompactMarshaler`changes to `encoder.CompactMarshaler`, which means compacting operation is not open by default

* fix: only one json value is `Valid()`

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
 2022-02-21 16:35:53 +08:00
Yi Duan
5be8dafe41
fix: premature GC on decoding string (#187) 
* fix: cast `[]byte` to `string` for `Unmarshal()`

* fix: store `Decoder.s` to `_Stack` to avoid premature GC

* fix: add gcwritebarrir for `op_str`
 2022-02-15 16:41:53 +08:00
Yi Duan
526c46883e
fix: support writeBarrier for Go 1.15 (#181) 2022-01-27 17:37:42 +08:00
liu
bf4e9e1982
fix: check number ending in skip number (#179) 
* fix: check number ending in skip number

* build: clang12

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
 2022-01-25 15:21:00 +08:00
Yi Duan
3dde76677e
doc: update benchmarks and dependency repos (#180) 
* doc: update benchmarks and dependency repos

* doc: add 'Optimization Options' on README

* fix some possible typos

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
 2022-01-25 15:09:01 +08:00
Yi Duan
f8efc2f6e7
Update README.md (#173) 2022-01-18 14:27:19 +08:00
liu
d75ce3f730
opt: use simd to optimize htmlescape (#171) 
* opt: use simd to optimize htmlescape

* opt: reuse escaped buffer

* feat: cmake with Clang12

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
 2022-01-18 11:30:28 +08:00
Yi Duan
14121d64f1
feat:(encoder) add optimazing options (#168) 
* feat:(encoder) add option `NoCompactMarshaler` and `NoEscapeTextMarshaler`

* feat: add `EscapeHTML` to align with std lib
 2022-01-05 14:20:46 +08:00
Yi Duan
fcfe1c4317
fix:(encoder) keep arguement buf alive (#166) 2021-12-31 17:15:35 +08:00
liu
1443eb3bcf
fix: move large-size local array to _Stack (#162) 
* fix: move large-size local array to _Stack

* fix: adjust jsonstate and add alwaysinline

* fix: initialize types.JsonState's dbuf at prologue

* fix: replace `always_inline` with `inline`

* fix: remove unused types

* feat: update asm2asm

* fix: check stack size befor call JIT function

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
 2021-12-30 14:51:23 +08:00
Yi Duan
c3cb5de704
feat:(ast) support sorting node keys (#164) 
* feat: ast.Node support ForEach() iteration (DFS)

Change-Id: Ia53f1db2814036e12b760dfbb7a21094a6abd541

* feat: support Node's key sorting

Change-Id: I0b93d9b4feada853fa2ca9f48277da71948a95be

* fmt

Change-Id: I1a53170959c08f1a32f02b0f163207254a87362c

* test: forbid `checkptr`

Change-Id: I6b34f74ee3bad883f515728300bf735a9e10b0d6

* fmt: add comments

Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
 2021-12-28 20:29:09 +08:00
liu
8dfaa13d3e
fix: out-of-bounds accessing in advance_string() (#161) 
* fix: out-of-bounds accessing in advance_string()

* fix: use Clang12 as C compiler

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
 2021-12-21 20:03:01 +08:00
Pure White
188e829dd7
chore(ci): setup self-hosted runner for ci (#158) 2021-12-20 19:56:07 +08:00
Yi Duan
8203bbac35
fix: remove stack writeBarrier for encoder (#159) 
Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
 2021-12-20 17:26:19 +08:00
Yi Duan
22229eefc3
fix: relocate stack pointers (#156) 
* fix: relocate stack pointer _VAR_sv to keep it alive

* fix: add stack pointer _VAR_vk to defend `encoding.TextUnmarshaler`

* fix: align faker func's stack with JIT func's

* fix: clear _Stack memory when err returned

* fix: clear stack pointer before return

* fix: relimit stack-overflow check at `_ValueDecoder`

Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
 2021-12-20 16:19:12 +08:00
Yi Duan
e80837a84d
fix:(ast) quote string when node.MarshalJSON() (#155) 
Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
 2021-12-15 19:20:18 +08:00