XeLabs/sonic
2
0
Fork 0
mirror of https://github.com/ii64/sonic.git synced 2026-06-21 00:46:43 +08:00
Code Issues Projects Releases Wiki Activity
184 commits 1 branch 6 tags 7.4 MiB
Commit graph

49 commits

Author SHA1 Message Date
liu
d83abb5435
opt: skip space use shuffle (#416) 2023-05-16 13:24:22 +08:00
liu
8639e93666
fix: validate on demand at first for GetByPath and NewRaw (#389) 
* feat(ast): validate the demanded fields for ast Get

* fix: add more tests

* fix: check invalid path type

* fix(arm): make return error when skip invalid exponent

* fix(arm): skip object and array with validate

* fix: check input json in NewRaw

* fix: validate the path in native c

* chore: add licenses

* fix: panic when invalid path

* fix: add Get Full json benchmark

* fix(arm): use fast skip besides the ondemand fields

* chore codes

---------

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
 2023-03-22 19:23:44 +08:00
liu
114b8168d4
fix: match empty string in path (#387) 2023-03-19 19:21:40 +08:00
liu
8f1e08ca4e
fix: validate empty object/array in ast Get (#383) 2023-03-14 13:05:23 +08:00
liu
18d2734975
fix: export not found error in ast get (#380) 2023-03-13 16:54:18 +08:00
liu
1f2f83e72e
fix(ast): out-of-bound for single unclosed string (#376) 
* fix(ast): out-of-bound for single unclosed string

* test: add more ast getbypath tests
 2023-03-07 15:52:34 +08:00
liu
10e45e906d
fix(ast): return error when get from single json value (#373) 
* fix(ast): return error when get from single json value

* fix: fallback impl of skipstring
 2023-03-02 12:45:35 +08:00
liu
02865de676
feat: repl invalid utf8 in serde by option (#357) 2023-02-10 18:55:27 +08:00
liu
2dc405d750
opt: faster skip in ast (#345) 
* opt: faster skip in ast

* ci: remove excited repo
 2023-01-09 16:20:36 +08:00
Yi Duan
67cffb15bd
feat:(ast) add fallback api on not-amd64 env (#341) 
* feat:(ast) add  fallback api on `not-amd64` env

* test: add native `linux-arm64` CI

* opt: just skip number chars whne `decodeFloat64`

* fmt

* fix: check EOF
 2023-01-03 19:47:55 +08:00
Yi Duan
02fe88266f
feat:(decoder) support skip mismatche-typed value (#325) 
* feat:(decoder) support skip mismatche-typed value

* change test cases

* refactor: add type check down into `CompilePrimitive()` to avoid repeat `null` check

* opt call skip()

* bench: add  option `--repeat_times`

* test: omit check primitive

* opt: inline primitive check into its OP

* implement on Go1.15

* fix: support skip json.Numer

* fix: OP_go_skip

* update README.md
 2022-11-04 14:27:21 +08:00
liu
6deed01a90
opt: reduce memory pool size (#302) 
* fix: reduce state stack size

* fix types

* opt: shrink encoder stack size

* opt: reduce decoder stack size

* add bench.py loops

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
 2022-10-28 16:40:14 +08:00
liu
56e81a633e
fix: use sse instead of sse4 (#305) 
* fix: use sse instead of sse4

* fix: use dispatch

* fix: remove lzero

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
 2022-09-26 12:45:01 +08:00
liu
ccc0f3f1e3
opt: faster ftoa (#291) 
* opt: faster f64toa

* feat:(option) add option `MaxInlineDepth` for addjust compilation inline depth (#287)

* feat: make compilation depth changeable

* feat: add option `DefaultMaxInlineDepth`

* add recurse depth = 10

* refactor

* doc: readme and comment

* opt: add `_MAX_FIELDS` to limit the inlining of big struct

* update license

* fix typo

* feat:(ast) support cast `null` to empty value (#278)

* fix: recompile

* test: add ftoa benchmarks

* feat: add f32toa

* fix: add license

* opt: optimize decimal

* fix benchmarks

* fix: poor performance in atof32

* test: add more integer test

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
Co-authored-by: Yi Duan <duanyi.aster@bytedance.com>
 2022-09-20 10:26:18 +08:00
cyn6315
a8311c6cde
fix: adapts to Mac M1 (#280) 
* fix: adapts to Mac M1
* feat: add `noavx` mode
* fix: fix SSE4 segfault with latest asm2asm
* chore: remove excessive spaces
* chore: more compact assembly
* chore: re-indent with 4 spaces
* chore: remove excessive space

Co-authored-by: chenzhuoyu <chenzhuoyu@bytedance.com>
 2022-08-15 14:37:20 +08:00
liu
de2dc2c35a
add ValidateString option into decode (#253) 
* fix: add escape validate

* feat: add validatestring option

* add print debug

* add jit debug in decoder

* fix go1.16 decoder debug

* fix: not change others

* fix generic parse bug

* remove debug info

* test twitter json

* fix: option typos

* test: add bug test

* fix validate bugs

* fix: validate

* remove files

* re-compile

* update license

* add flags on `skip_xxx()`

* fix internal native tests

* re-compile

* add validstring for perftest

* modify api

* fix readme

* fix comment

* add license

* rename to ValidateString

* fix xprintf.h

* add debug

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
 2022-08-11 19:06:11 +08:00
Yi Duan
2eae594741
fix:(native) StreamDecoder returns error when run out of buffer while skiping value (#267) 
* fix:(native) StreamDecoder return error when run out of input buffer while skiping value

* refactor
 2022-07-25 15:17:52 +08:00
liu
f8fb04a184
fix(ub): marshal escape b f as encoding/json (#243) 
* fix(ub): marshal b f as encoding/json

* test: fix old test for escape b

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
 2022-06-10 11:21:35 +08:00
liu
410625ca9e
opt: remove bound and loop unrolling in quote (begin to use clang13) (#237) 
* opt: remove most bound check in quote

* opt: remove branch in copied escaped chars

* opt: use cache-friendly bool array

* opt: use loop unrolling in quote

* feat: add `-t` for bench.py

Change-Id: I3f41218187597126ef960eab09c6fa6fe252f347

* test:(bench) adjust repeat_time

* test: add Diff on CI

* test: update bench.py

* doc: update README.md

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
 2022-06-08 21:50:10 +08:00
liu
57989f38ba
opt: skip number in json.Number parsing (#209) 
* opt: skip number in json.Number  parsing

* fix: generic use skip_number

* test: add json.Number decoding benchmarks

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
 2022-04-01 17:45:06 +08:00
liu
ce41dd9636
fix: correct htmlescape chars check (#190) 
* fix: correct htmlescape chars check

* fix: continuous html escape chars

* tmp: compile clang13

* fix: check single xe2 char

* fix: htmlescape no mem test

* fix: htmlescape zero check bug

* fix: htmlescape bug

* build: clang12

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
 2022-02-22 11:43:58 +08:00
liu
7475b256ce
support JSON validate (#189) 
* fix: check unescaped control chars in decode

* feat: add utf8 validate func

* feat: validate utf8 in json string

* feat: add validateone api

* fix: check unicode pointer for surrogate

* clang12 compile

* feat: Import `Valid()` and `Skip()`

* opt: use looktable

* fix utf-8 validate performance problem

* fix: utf-8 validate bug

* clang12 build

* feat: (encoder) accelerate validating json from `json.Marshaler`

chore!:
- `encoder.NoCompactMarshaler`changes to `encoder.CompactMarshaler`, which means compacting operation is not open by default

* fix: only one json value is `Valid()`

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
 2022-02-21 16:35:53 +08:00
liu
bf4e9e1982
fix: check number ending in skip number (#179) 
* fix: check number ending in skip number

* build: clang12

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
 2022-01-25 15:21:00 +08:00
liu
d75ce3f730
opt: use simd to optimize htmlescape (#171) 
* opt: use simd to optimize htmlescape

* opt: reuse escaped buffer

* feat: cmake with Clang12

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
 2022-01-18 11:30:28 +08:00
liu
1443eb3bcf
fix: move large-size local array to _Stack (#162) 
* fix: move large-size local array to _Stack

* fix: adjust jsonstate and add alwaysinline

* fix: initialize types.JsonState's dbuf at prologue

* fix: replace `always_inline` with `inline`

* fix: remove unused types

* feat: update asm2asm

* fix: check stack size befor call JIT function

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
 2021-12-30 14:51:23 +08:00
liu
8dfaa13d3e
fix: out-of-bounds accessing in advance_string() (#161) 
* fix: out-of-bounds accessing in advance_string()

* fix: use Clang12 as C compiler

Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
 2021-12-21 20:03:01 +08:00
Yi Duan
442ce696fb
fix: use stackmap of shadow func as jit func's (#127) 
* fix: use stackmap of shadow func as jit func's

* fix: use LoadWithFaker in decoder

* fix: LoadWithFaker support go115

* add 'runtime.' prefix on jit funcname to prevent preempt

* add parallel GC tests

* remove no_stack_pointer()

Co-authored-by: duanyi.aster <duanyi.aster@bytedance.com>
Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
 2021-11-09 11:50:40 +08:00
liu
530d423243
fix: make it parse the UTF-16 surrogate pair after invalid unicode (#111) 2021-09-24 16:33:02 +08:00
liu
a7ace8cd03
fix: skip exponent number that begin with zero in skip_number() (#109) 2021-09-23 15:02:09 +08:00
chenzhuoyu
0e4b0b8ee1 fix: check character range before BTQ 2021-09-17 19:48:47 +08:00
liuqiang
9ba912d20a fix: make it return error when unmarshal infinity float number 2021-09-14 18:17:45 +08:00
liuqiang
ec15697761 fix: support IEEE-754 negative zero in both marshal and unmarshal 2021-09-14 15:06:47 +08:00
Oxygen
6aff4b1ad6
fix: handling of surrogate-half characters within double-quoted strings (#89) 2021-09-07 14:14:06 +08:00
liu
365da9c1a8
fix: check prefix in native float parsing (#86) 2021-09-06 16:01:42 +08:00
chenzhuoyu
8383178c89 feat: supports map key-sorting via encoder options 2021-08-16 18:31:41 +08:00
liuqiang
ebac0a347a fix: parse min int64 number to float when UseInt64() 2021-07-29 21:27:05 +08:00
liu
83208b7ac4
feat: optimize float-to-str encoding through the Ryu algorithm (#60) 
Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
Co-authored-by: Oxygen <chenzhuoyu@users.noreply.github.com>
 2021-07-29 17:03:38 +08:00
liuqiang
e88411dafe fix: parsing to float64 when the integer(int64) overflow 2021-07-23 18:29:31 +08:00
liuqiang
f9632ab873 fix: enhance float parsing as Go encoding/json 2021-07-23 16:39:32 +08:00
liu
a5efd7e8b7
fix: support high precision for float parsing (#54) 
Co-authored-by: liuqiang <liuqiang.06@bytedance.com>
 2021-07-18 11:37:59 +08:00
chenzhuoyu
bcca8c2052 fix: fix addressing past valid memory for empty string 2021-07-02 18:25:50 +08:00
chenzhuoyu
5cb6f17944 feat: performance optimizations 2021-07-02 14:38:12 +08:00
chenzhuoyu
85a0d4219f feat: rewrite the generic decoder in assembly 2021-06-21 11:34:18 +08:00
chenzhuoyu
b2fbad7b4c feat: rewrite generic decoder in Go, using Finite State Machine 2021-06-16 23:05:21 +08:00
chenzhuoyu
34fa8d64a8 feat: CPU feature detection support 2021-06-10 17:14:25 +08:00
chenzhuoyu
e44e686d90 fix: clears rest of the array if the JSON is not long enough. This fixes #7. 2021-06-06 16:25:34 +08:00
chenzhuoyu
fc6fe804a7 fix: disable golink, it does not recognize my black-magic 2021-05-29 00:32:43 +08:00
chenzhuoyu
203b83a8cf fix: fix Copyright placements 2021-05-29 00:15:37 +08:00
chenzhuoyu
fbe0828190 chore: make it open-source 2021-05-28 23:58:58 +08:00