XeLabs/sonic
2
0
Fork 0
mirror of https://github.com/ii64/sonic.git synced 2026-06-21 00:46:43 +08:00
Code Issues Projects Releases Wiki Activity
189 commits 1 branch 6 tags 7.4 MiB
Commit graph

70 commits

Author SHA1 Message Date
liu
be00a52b0d
fix: decoder and encoder support fallback (#430) 2023-05-25 11:08:02 +08:00
liu
d83abb5435
opt: skip space use shuffle (#416) 2023-05-16 13:24:22 +08:00
Yi Duan
cdd8937c34
fix: incorrectly handle alias type of pointer in struct (#404) 2023-04-19 13:55:58 +08:00
liu
7e82ad0a45
fix: unmarshal into defined ptr type (#392) 2023-04-03 17:25:57 +08:00
liu
8639e93666
fix: validate on demand at first for GetByPath and NewRaw (#389) 
* feat(ast): validate the demanded fields for ast Get

* fix: add more tests

* fix: check invalid path type

* fix(arm): make return error when skip invalid exponent

* fix(arm): skip object and array with validate

* fix: check input json in NewRaw

* fix: validate the path in native c

* chore: add licenses

* fix: panic when invalid path

* fix: add Get Full json benchmark

* fix(arm): use fast skip besides the ondemand fields

* chore codes

---------

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
 2023-03-22 19:23:44 +08:00
liu
114b8168d4
fix: match empty string in path (#387) 2023-03-19 19:21:40 +08:00
liu
8f1e08ca4e
fix: validate empty object/array in ast Get (#383) 2023-03-14 13:05:23 +08:00
liu
18d2734975
fix: export not found error in ast get (#380) 2023-03-13 16:54:18 +08:00
liu
1f2f83e72e
fix(ast): out-of-bound for single unclosed string (#376) 
* fix(ast): out-of-bound for single unclosed string

* test: add more ast getbypath tests
 2023-03-07 15:52:34 +08:00
liu
10e45e906d
fix(ast): return error when get from single json value (#373) 
* fix(ast): return error when get from single json value

* fix: fallback impl of skipstring
 2023-03-02 12:45:35 +08:00
Yi Duan
6d60889e3b
build: omit checkptr detects on unsafe functions (#372) 
* build: omit `checkptr` detect on unsafe functions

* update CI
 2023-02-28 19:13:19 +08:00
Yi Duan
e60b9541cd
fix: add nocheckptr directive (#365) 2023-02-20 14:52:46 +08:00
Yi Duan
fa82a64ce2
feat: add env option SONIC_STOP_PROFILING to stop traceback on demands (#361) 2023-02-14 14:48:22 +08:00
liu
02865de676
feat: repl invalid utf8 in serde by option (#357) 2023-02-10 18:55:27 +08:00
Yi Duan
e7ac2f25fc
feat: more complete function loader (#354) 
* follow complete implementation of go symtab
* support go1.20
 2023-02-08 14:35:00 +08:00
liu
2dc405d750
opt: faster skip in ast (#345) 
* opt: faster skip in ast

* ci: remove excited repo
 2023-01-09 16:20:36 +08:00
Yi Duan
67cffb15bd
feat:(ast) add fallback api on not-amd64 env (#341) 
* feat:(ast) add  fallback api on `not-amd64` env

* test: add native `linux-arm64` CI

* opt: just skip number chars whne `decodeFloat64`

* fmt

* fix: check EOF
 2023-01-03 19:47:55 +08:00
Yi Duan
134fba2c1d
feat: ignore SIGPROF while calling native funcs (#342) 2023-01-03 15:15:51 +08:00
Yi Duan
f421ee8530
fix:(encoder) pass pv through compiler recursively (#336) 
* pass pv

* test reflect indirect

* fix: pass `pv` throught compiler recursively
 2022-12-13 14:26:58 +08:00
Yi Duan
02fe88266f
feat:(decoder) support skip mismatche-typed value (#325) 
* feat:(decoder) support skip mismatche-typed value

* change test cases

* refactor: add type check down into `CompilePrimitive()` to avoid repeat `null` check

* opt call skip()

* bench: add  option `--repeat_times`

* test: omit check primitive

* opt: inline primitive check into its OP

* implement on Go1.15

* fix: support skip json.Numer

* fix: OP_go_skip

* update README.md
 2022-11-04 14:27:21 +08:00
liu
6deed01a90
opt: reduce memory pool size (#302) 
* fix: reduce state stack size

* fix types

* opt: shrink encoder stack size

* opt: reduce decoder stack size

* add bench.py loops

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
 2022-10-28 16:40:14 +08:00
liu
56e81a633e
fix: use sse instead of sse4 (#305) 
* fix: use sse instead of sse4

* fix: use dispatch

* fix: remove lzero

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
 2022-09-26 12:45:01 +08:00
liu
ccc0f3f1e3
opt: faster ftoa (#291) 
* opt: faster f64toa

* feat:(option) add option `MaxInlineDepth` for addjust compilation inline depth (#287)

* feat: make compilation depth changeable

* feat: add option `DefaultMaxInlineDepth`

* add recurse depth = 10

* refactor

* doc: readme and comment

* opt: add `_MAX_FIELDS` to limit the inlining of big struct

* update license

* fix typo

* feat:(ast) support cast `null` to empty value (#278)

* fix: recompile

* test: add ftoa benchmarks

* feat: add f32toa

* fix: add license

* opt: optimize decimal

* fix benchmarks

* fix: poor performance in atof32

* test: add more integer test

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
Co-authored-by: Yi Duan <duanyi.aster@bytedance.com>
 2022-09-20 10:26:18 +08:00
cyn6315
a8311c6cde
fix: adapts to Mac M1 (#280) 
* fix: adapts to Mac M1
* feat: add `noavx` mode
* fix: fix SSE4 segfault with latest asm2asm
* chore: remove excessive spaces
* chore: more compact assembly
* chore: re-indent with 4 spaces
* chore: remove excessive space

Co-authored-by: chenzhuoyu <chenzhuoyu@bytedance.com>
 2022-08-15 14:37:20 +08:00
liu
de2dc2c35a
add ValidateString option into decode (#253) 
* fix: add escape validate

* feat: add validatestring option

* add print debug

* add jit debug in decoder

* fix go1.16 decoder debug

* fix: not change others

* fix generic parse bug

* remove debug info

* test twitter json

* fix: option typos

* test: add bug test

* fix validate bugs

* fix: validate

* remove files

* re-compile

* update license

* add flags on `skip_xxx()`

* fix internal native tests

* re-compile

* add validstring for perftest

* modify api

* fix readme

* fix comment

* add license

* rename to ValidateString

* fix xprintf.h

* add debug

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
 2022-08-11 19:06:11 +08:00
Yi Duan
b36771ba37
feat: support Go 1.19 (#275) 
* feat: support Go 1.19

* fuzz test refactor

* fix fuzz bug, generate pointer typ

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
 2022-08-09 12:54:11 +08:00
Yi Duan
2eae594741
fix:(native) StreamDecoder returns error when run out of buffer while skiping value (#267) 
* fix:(native) StreamDecoder return error when run out of input buffer while skiping value

* refactor
 2022-07-25 15:17:52 +08:00
liu
f8fb04a184
fix(ub): marshal escape b f as encoding/json (#243) 
* fix(ub): marshal b f as encoding/json

* test: fix old test for escape b

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
 2022-06-10 11:21:35 +08:00
liu
410625ca9e
opt: remove bound and loop unrolling in quote (begin to use clang13) (#237) 
* opt: remove most bound check in quote

* opt: remove branch in copied escaped chars

* opt: use cache-friendly bool array

* opt: use loop unrolling in quote

* feat: add `-t` for bench.py

Change-Id: I3f41218187597126ef960eab09c6fa6fe252f347

* test:(bench) adjust repeat_time

* test: add Diff on CI

* test: update bench.py

* doc: update README.md

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
 2022-06-08 21:50:10 +08:00
Ii64人
75b728ba51
feat: support for Windows (#228) 
* feat: support for Windows

* fix (loader): default loader now has build tag: for linux and darwin
* feat (loader): add memory allocator support for Windows

* feat: add Windows CI

Added Windows CI for GitHub Actions
Go version: 1.15.x, 1.16.x, 1.17.x, 1.18.x

* fix (loader): remove `amd64` build constraint from Windows

* fix: TSAN error on Windows CI

Temporary remove `-race` flag

Link: https://github.com/golang/go/issues/46099

Co-authored-by: Yi Duan <duanyi.aster@bytedance.com>
 2022-05-23 12:55:45 +08:00
liu
57989f38ba
opt: skip number in json.Number parsing (#209) 
* opt: skip number in json.Number  parsing

* fix: generic use skip_number

* test: add json.Number decoding benchmarks

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
 2022-04-01 17:45:06 +08:00
Yi Duan
fe2497d01e
fix:(loader) init _Moduledata.gcdatapointer to avoid panic on plugin-mode (#188) 
* fix:(loader) init `_Moduledata.gcdata`pointer to avoid panic on plugin-mode

* fix: freeze gcdata pointer

* feat: go115

* update test

* fix: use unique bytes pointer

* test: drop go1.15 race test

* feat: support go1.18
 2022-04-01 16:46:07 +08:00
Yi Duan
cb1702dc8c
feat: support Go1.18 (#205) 
* feat: support Go1.18

* fix: re-encoding `moduledata.pclntab`

* test: add generic func wrap test
 2022-03-28 20:43:43 +08:00
liu
ce41dd9636
fix: correct htmlescape chars check (#190) 
* fix: correct htmlescape chars check

* fix: continuous html escape chars

* tmp: compile clang13

* fix: check single xe2 char

* fix: htmlescape no mem test

* fix: htmlescape zero check bug

* fix: htmlescape bug

* build: clang12

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
 2022-02-22 11:43:58 +08:00
liu
7475b256ce
support JSON validate (#189) 
* fix: check unescaped control chars in decode

* feat: add utf8 validate func

* feat: validate utf8 in json string

* feat: add validateone api

* fix: check unicode pointer for surrogate

* clang12 compile

* feat: Import `Valid()` and `Skip()`

* opt: use looktable

* fix utf-8 validate performance problem

* fix: utf-8 validate bug

* clang12 build

* feat: (encoder) accelerate validating json from `json.Marshaler`

chore!:
- `encoder.NoCompactMarshaler`changes to `encoder.CompactMarshaler`, which means compacting operation is not open by default

* fix: only one json value is `Valid()`

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
 2022-02-21 16:35:53 +08:00
Yi Duan
526c46883e
fix: support writeBarrier for Go 1.15 (#181) 2022-01-27 17:37:42 +08:00
liu
bf4e9e1982
fix: check number ending in skip number (#179) 
* fix: check number ending in skip number

* build: clang12

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
 2022-01-25 15:21:00 +08:00
liu
d75ce3f730
opt: use simd to optimize htmlescape (#171) 
* opt: use simd to optimize htmlescape

* opt: reuse escaped buffer

* feat: cmake with Clang12

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
 2022-01-18 11:30:28 +08:00
liu
1443eb3bcf
fix: move large-size local array to _Stack (#162) 
* fix: move large-size local array to _Stack

* fix: adjust jsonstate and add alwaysinline

* fix: initialize types.JsonState's dbuf at prologue

* fix: replace `always_inline` with `inline`

* fix: remove unused types

* feat: update asm2asm

* fix: check stack size befor call JIT function

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
 2021-12-30 14:51:23 +08:00
liu
8dfaa13d3e
fix: out-of-bounds accessing in advance_string() (#161) 
* fix: out-of-bounds accessing in advance_string()

* fix: use Clang12 as C compiler

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
 2021-12-21 20:03:01 +08:00
Yi Duan
49df11ef83
feat: adjust ABI for supporting go1.17 (#116) 
* feat: adjust encoder ABI for supporting go1.17

* binding

* generic

* opt: exchange R10/R11 with BX/BP when call c func

* fmt

Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
 2021-11-30 18:47:17 +08:00
liu
c27120f2bc
fix: correct the field priority as encoding/json when unmarshal to struct (#142) 2021-11-30 10:28:50 +08:00
Yi Duan
442ce696fb
fix: use stackmap of shadow func as jit func's (#127) 
* fix: use stackmap of shadow func as jit func's

* fix: use LoadWithFaker in decoder

* fix: LoadWithFaker support go115

* add 'runtime.' prefix on jit funcname to prevent preempt

* add parallel GC tests

* remove no_stack_pointer()

Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
 2021-11-09 11:50:40 +08:00
liu
530d423243
fix: make it parse the UTF-16 surrogate pair after invalid unicode (#111) 2021-09-24 16:33:02 +08:00
liu
a7ace8cd03
fix: skip exponent number that begin with zero in skip_number() (#109) 2021-09-23 15:02:09 +08:00
chenzhuoyu
0e4b0b8ee1 fix: check character range before BTQ 2021-09-17 19:48:47 +08:00
liuqiang
9ba912d20a fix: make it return error when unmarshal infinity float number 2021-09-14 18:17:45 +08:00
liuqiang
ec15697761 fix: support IEEE-754 negative zero in both marshal and unmarshal 2021-09-14 15:06:47 +08:00
chenzhuoyu
514208a4d5 fix: potential ProgramCache OOM under high-load 2021-09-09 14:35:01 +08:00
Oxygen
6aff4b1ad6
fix: handling of surrogate-half characters within double-quoted strings (#89) 2021-09-07 14:14:06 +08:00