XeLabs/sonic
2
0
Fork 0
mirror of https://github.com/ii64/sonic.git synced 2026-06-21 00:46:43 +08:00
Code Issues Projects Releases Wiki Activity
183 commits 1 branch 6 tags 7.4 MiB
Commit graph

74 commits

Author SHA1 Message Date
liu
eaa70d4c25
fix: initialize the elem when slice grow in decode (#415) 
* fix: check mismatch error after recusive

* fix: initialize the elem when slice grow in decode

---------

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
Co-authored-by: Yi Duan <duanyi.aster@bytedance.com>
 2023-05-16 14:13:04 +08:00
liu
ebbe7589ca
fix: check mismatch error after recursive (#407) 
* fix: check mismatch error after recusive

* fix: add tests

* format

---------

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
 2023-05-06 16:20:23 +08:00
liu
869fa1b2b9
fix: unmarshal recursive pointer type (#400) 2023-04-13 15:06:13 +08:00
liu
7e82ad0a45
fix: unmarshal into defined ptr type (#392) 2023-04-03 17:25:57 +08:00
Yi Duan
8dbcce341b
fix: decimal map key may be truncated and populated unexpectedly (#382) 
* fix:(StreamDecoder) Integer map key may be truncated and populated unexpectedly

* feat: expose option `DefaultBufferSize`

* test adjust

* test adjust

* feat: add options to make default buffer size adjustable
 2023-03-14 13:49:27 +08:00
liu
02865de676
feat: repl invalid utf8 in serde by option (#357) 2023-02-10 18:55:27 +08:00
Yi Duan
e7ac2f25fc
feat: more complete function loader (#354) 
* follow complete implementation of go symtab
* support go1.20
 2023-02-08 14:35:00 +08:00
Yi Duan
32877b6d27
fix:(decoder) not clear memory after decoding failed (#346) 
* fix: not clear memory after decoding failed

* doc: update sonic ast benchmark data
 2023-01-10 14:19:32 +08:00
Yi Duan
134fba2c1d
feat: ignore SIGPROF while calling native funcs (#342) 2023-01-03 15:15:51 +08:00
Yi Duan
f421ee8530
fix:(encoder) pass pv through compiler recursively (#336) 
* pass pv

* test reflect indirect

* fix: pass `pv` throught compiler recursively
 2022-12-13 14:26:58 +08:00
liu
dda5da9f64
fix: support decode json-encoded base64 in JIT (#329) 
Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
 2022-11-15 16:38:08 +08:00
Yi Duan
518110bd99
feat:(decoder) clear memory when decoding failed (#320) 
* feat:(decoder) clear memory when decoding failed

* fix test case

* fix: skip empty string for `,string` option

* test map text key

* support skip mismatched key-value of map
 2022-11-07 11:04:19 +08:00
Yi Duan
02fe88266f
feat:(decoder) support skip mismatche-typed value (#325) 
* feat:(decoder) support skip mismatche-typed value

* change test cases

* refactor: add type check down into `CompilePrimitive()` to avoid repeat `null` check

* opt call skip()

* bench: add  option `--repeat_times`

* test: omit check primitive

* opt: inline primitive check into its OP

* implement on Go1.15

* fix: support skip json.Numer

* fix: OP_go_skip

* update README.md
 2022-11-04 14:27:21 +08:00
liu
6deed01a90
opt: reduce memory pool size (#302) 
* fix: reduce state stack size

* fix types

* opt: shrink encoder stack size

* opt: reduce decoder stack size

* add bench.py loops

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
 2022-10-28 16:40:14 +08:00
Chen Yannan
5e54c02172
fmt: move thrid-party tests into another module (#299) 
* fix:move benchmarkTests and unitTests related to external json library

* remove extra spaces

* merge go.mod files

* refactor: add external_jsonlib_test into go.work and CI

Co-authored-by: Yi Duan <duanyi.aster@bytedance.com>
 2022-09-20 12:02:41 +08:00
Yi Duan
a48cad8488
fix:(decoder) more loose repeatable condition for StreamDecoder (#296) 2022-09-08 16:16:38 +08:00
Yi Duan
6e979df0d3
fix:(decoder) StreamDecoder unexpectedly stop while skipping incomplete number (#295) 2022-09-07 19:54:13 +08:00
Yi Duan
94f95f0479
feat:(option) add option MaxInlineDepth for addjust compilation inline depth (#287) 
* feat: make compilation depth changeable

* feat: add option `DefaultMaxInlineDepth`

* add recurse depth = 10

* refactor

* doc: readme and comment

* opt: add `_MAX_FIELDS` to limit the inlining of big struct

* update license

* fix typo
 2022-08-22 15:45:49 +08:00
cyn6315
a8311c6cde
fix: adapts to Mac M1 (#280) 
* fix: adapts to Mac M1
* feat: add `noavx` mode
* fix: fix SSE4 segfault with latest asm2asm
* chore: remove excessive spaces
* chore: more compact assembly
* chore: re-indent with 4 spaces
* chore: remove excessive space

Co-authored-by: chenzhuoyu <chenzhuoyu@bytedance.com>
 2022-08-15 14:37:20 +08:00
liu
de2dc2c35a
add ValidateString option into decode (#253) 
* fix: add escape validate

* feat: add validatestring option

* add print debug

* add jit debug in decoder

* fix go1.16 decoder debug

* fix: not change others

* fix generic parse bug

* remove debug info

* test twitter json

* fix: option typos

* test: add bug test

* fix validate bugs

* fix: validate

* remove files

* re-compile

* update license

* add flags on `skip_xxx()`

* fix internal native tests

* re-compile

* add validstring for perftest

* modify api

* fix readme

* fix comment

* add license

* rename to ValidateString

* fix xprintf.h

* add debug

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
 2022-08-11 19:06:11 +08:00
Yi Duan
b36771ba37
feat: support Go 1.19 (#275) 
* feat: support Go 1.19

* fuzz test refactor

* fix fuzz bug, generate pointer typ

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
 2022-08-09 12:54:11 +08:00
Yi Duan
2eae594741
fix:(native) StreamDecoder returns error when run out of buffer while skiping value (#267) 
* fix:(native) StreamDecoder return error when run out of input buffer while skiping value

* refactor
 2022-07-25 15:17:52 +08:00
liu
34026b9c65
fix: useint64 in decode SetOption (#268) 
* fix: useint64 in decode SetOption

* doc: add comments on `sonic.Config`

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
 2022-07-25 14:26:18 +08:00
Yi Duan
08c7640684
fix:(decoder) didn't check ',' before object key due to imperfect FSM (#261) 2022-07-08 11:12:34 +08:00
Yi Duan
55e3d10619
fix: (decoder) return io.EOF when read nothing from io.Reader (#254) 
* fix: (decoder) return io.EOF when read nothing from io.Reader

* build:(CI) codeql-analysis only use x86 machine
 2022-07-01 17:51:42 +08:00
Yi Duan
ced28302a9
feat: add compatible API (#233) 
* feat: add compatible API

* test: add api compatibility test

* fmt

* test: adjust CI

* fix: no trailing '\n' for std `Marshal()`

* doc: update README.md

* test: add windows CI

* doc: update README.md

* doc: update README.md

* fmt: add comments and refator
 2022-05-25 14:16:56 +08:00
Yi Duan
f9d033d6f8
feat: support streaming IO (#225) 
* feat (decoder): support streaming IO

* opt: fine tune buffer size factors

* feat (encoder): support streaming IO

* doc: refactor format and add comments

* fix: io.EOF dead loop

* build: adjust CI files

* doc: update README.md
 2022-05-24 10:38:43 +08:00
Yi Duan
9aab4704ef
fix: issux213 (#215) 
* fix (decoder): use AVX mode only on base64x
 2022-04-21 15:29:52 +08:00
liu
57989f38ba
opt: skip number in json.Number parsing (#209) 
* opt: skip number in json.Number  parsing

* fix: generic use skip_number

* test: add json.Number decoding benchmarks

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
 2022-04-01 17:45:06 +08:00
Yi Duan
cb1702dc8c
feat: support Go1.18 (#205) 
* feat: support Go1.18

* fix: re-encoding `moduledata.pclntab`

* test: add generic func wrap test
 2022-03-28 20:43:43 +08:00
Yi Duan
94232a7b2e
fix: ignore overflow error when decoding json.Number (#207) 2022-03-24 16:00:08 +08:00
Yi Duan
c381c8b679
feat: add API MarshalString() (#197) 2022-03-01 15:09:09 +08:00
Yi Duan
f087445eee
fix: allow string value decoded as json.Number (#196) 
* fix: allow string value decoded as `json.Number`

* test: move -race flag from Go1.15 since it reports "race: limit on 8128 simultaneously alive goroutines is exceeded, dying"

* test: add benchmarks
 2022-02-28 20:14:38 +08:00
Yi Duan
b66168fa77
feat:(decoder) add option CopyString() (#192) 
* feat: add option `CopyString()`

* test: diff copy and nocopy decoding bench

* doc: update README.md

* opt: use global link for `escape_string()`

* feat: copy JSON on `Unmarshal()`

* test: add norace test

* fix: nocopy field key

* test: add generic test

* fix: remove useless instruction
 2022-02-28 18:46:25 +08:00
liu
7475b256ce
support JSON validate (#189) 
* fix: check unescaped control chars in decode

* feat: add utf8 validate func

* feat: validate utf8 in json string

* feat: add validateone api

* fix: check unicode pointer for surrogate

* clang12 compile

* feat: Import `Valid()` and `Skip()`

* opt: use looktable

* fix utf-8 validate performance problem

* fix: utf-8 validate bug

* clang12 build

* feat: (encoder) accelerate validating json from `json.Marshaler`

chore!:
- `encoder.NoCompactMarshaler`changes to `encoder.CompactMarshaler`, which means compacting operation is not open by default

* fix: only one json value is `Valid()`

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
 2022-02-21 16:35:53 +08:00
Yi Duan
5be8dafe41
fix: premature GC on decoding string (#187) 
* fix: cast `[]byte` to `string` for `Unmarshal()`

* fix: store `Decoder.s` to `_Stack` to avoid premature GC

* fix: add gcwritebarrir for `op_str`
 2022-02-15 16:41:53 +08:00
Yi Duan
526c46883e
fix: support writeBarrier for Go 1.15 (#181) 2022-01-27 17:37:42 +08:00
Yi Duan
fcfe1c4317
fix:(encoder) keep arguement buf alive (#166) 2021-12-31 17:15:35 +08:00
liu
1443eb3bcf
fix: move large-size local array to _Stack (#162) 
* fix: move large-size local array to _Stack

* fix: adjust jsonstate and add alwaysinline

* fix: initialize types.JsonState's dbuf at prologue

* fix: replace `always_inline` with `inline`

* fix: remove unused types

* feat: update asm2asm

* fix: check stack size befor call JIT function

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
 2021-12-30 14:51:23 +08:00
Yi Duan
22229eefc3
fix: relocate stack pointers (#156) 
* fix: relocate stack pointer _VAR_sv to keep it alive

* fix: add stack pointer _VAR_vk to defend `encoding.TextUnmarshaler`

* fix: align faker func's stack with JIT func's

* fix: clear _Stack memory when err returned

* fix: clear stack pointer before return

* fix: relimit stack-overflow check at `_ValueDecoder`

Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
 2021-12-20 16:19:12 +08:00
Yi Duan
7d3b22100f
feat: SyntaxError.Error() always calls Description() (#152) 
Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
 2021-12-09 13:11:31 +08:00
liu
d5104197c4
fix: 'string' option in struct tag as encoding/json (#148) 
Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
 2021-12-03 15:46:09 +08:00
Yi Duan
d2711a6af0
fix: use global var _KeepAlive to notice GC in shadow funcs (#145) 
Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
 2021-12-02 11:08:40 +08:00
liu
3f2bab552b
doc: update readme (#143) 
* doc: update readme

* test: add SetOne_XX() benchmarks

* doc: update bechmark charts

* doc: fix possible typos

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
 2021-12-01 18:03:19 +08:00
Yi Duan
49df11ef83
feat: adjust ABI for supporting go1.17 (#116) 
* feat: adjust encoder ABI for supporting go1.17

* binding

* generic

* opt: exchange R10/R11 with BX/BP when call c func

* fmt

Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
 2021-11-30 18:47:17 +08:00
liu
49fc705341
feat: Pretouch recursively for large/deep struct (#137) 2021-11-26 14:12:51 +08:00
Yi Duan
5b38f56cec
fix: go1.15 CANNOT support gcWriteBarrier due to linkname bug (#134) 
* fix: go1.15 CANNOT add gcWriteBarrier due to linkname bug

* add go1.15 push-check

Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
 2021-11-18 11:15:14 +08:00
Yi Duan
43e4a00e90
fix: add gc_write_barrier checks (#131) 
* fix: add gcWriteBarrier for decode

* add gcWriteBarrier for encoder


Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
 2021-11-17 11:21:20 +08:00
Yi Duan
ba4c2d2e55
fix: set _Stack.sp zero whenever put into pool (#132) 
Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
 2021-11-15 14:41:04 +08:00
Yi Duan
442ce696fb
fix: use stackmap of shadow func as jit func's (#127) 
* fix: use stackmap of shadow func as jit func's

* fix: use LoadWithFaker in decoder

* fix: LoadWithFaker support go115

* add 'runtime.' prefix on jit funcname to prevent preempt

* add parallel GC tests

* remove no_stack_pointer()

Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
 2021-11-09 11:50:40 +08:00