Conversation
- Cache the parsed validation tag values (field/rule/message) in a process level cache keyed by the tag value, as the tag values of the same struct definitions are static. The cache is enabled in default and can be disabled by gvalid.New(false) for dynamically generated rules, avoiding useless cache growth. - Use the messages of type map[string]string directly in doCheckValue instead of converting them by gconv.Map, and allocate the message containers lazily. - Replace the regexp based single rule item parsing with plain string scanning, which also fixes the panic when the name of a custom rule contains non-ASCII characters. - Add benchmarks for the struct validation path. Benchmarked on Linux (golang:1.25 container, -benchtime=2s) against e8d2252, measured at the branch tip where the util/gtag change is applied as well: Benchmark_CheckStruct_Data 8.18us -> 5.86us ( 97 -> 71 allocs) Benchmark_CheckStruct_Assoc 7.78us -> 5.33us ( 91 -> 65 allocs) Benchmark_CheckStruct_BigReq 28.30us -> 21.6us (251 -> 189 allocs)
…eholder
Parse replaces "{name}" style placeholders using a regexp, and it is a
hot path for struct tag retrieving (gstructs.Field.Tag and
TagPriorityName), while most tag values contain no placeholder at all.
Return the content directly if there is no char '{', which is equivalent
as the regexp `\{(.+?)\}` cannot match an input without it.
Clone created a new Validator through New and then overwrote every field with a shallow copy of the source, so the ruleFuncMap created by New was never referenced. Create the shallow copy directly instead: the copy already carries the source's ruleFuncMap, which keeps the behavior identical. Benchmarked on Linux (golang:1.25 container, interleaved, medians of 6 samples) against the previous commit: Benchmark_CheckStruct_Data 6.02us -> 5.75us ( 71 -> 67 allocs) Benchmark_CheckStruct_Assoc 5.49us -> 5.20us ( 65 -> 59 allocs) Benchmark_CheckStruct_BigReq 21.96us -> 20.90us (189 -> 185 allocs)
The parsed rule value cache is process level, while validation rules might be dynamically generated in user code, in which case the cache would grow without bound. Disable the cache in default so that the behavior of existing user code keeps unchanged, and enable it explicitly for the HTTP request handling, of which the validation rules are all from struct tag and are static and bounded. Users who are sure their rules are static can enable it with gvalid.New(true) for the same performance gain.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
背景
标准 handler(
func(ctx, req) (res, err))的 Req 校验是每个请求都要付的固定开销。对gvalid做 profile 后发现,时间大量花在几处"每次都在重复算、但结果本来就不会变"的地方:map[string]string,却每次被gconv.Map反射转换 + 拷贝一份^([\w-]+):{0,1}(.*)拆规则名:参数gtag.Parse的正则,而绝大多数 tag 值根本没有{占位符改动
util/gvalid(4 处)field/rule/message),进程级、默认关闭。Rules([]string{...})),进程级缓存会随之无界增长;默认关闭保证用户现有代码的行为与旧版完全一致。ghttp的请求校验路径就是这么做的(见下);用户自己的调用也可以传gvalid.New(true)获得同样收益。ParseTagValue保持原有"不缓存"语义,直接调用方不会引入隐藏的内存增长。Messages为map[string]string时直接使用(只读,无拷贝),跳过gconv.Map的反射转换与逐项gconv.String;消息容器改为惰性分配。:、|都属于参数(例如regex:^\d+:\w+$)。顺带修复一个 panic:自定义规则名含非 ASCII 字符(例如注册中文规则)时,旧实现会index out of range。Clone()里的无效分配:原先Clone()先调用New()再整体覆盖,New()创建的ruleFuncMap从未被引用(每次 Clone 白分配一个 map);改为直接做浅拷贝,逐字段行为等价。util/gtagParse在内容不含{时直接返回。正则\{(.+?)\}必须先有字面量{才可能匹配,所以这是可证明的等价:无占位符时结果必然等于原文(返回的字符串本身,不产生拷贝)。gtag.Parse是所有读 struct tag 路径的必经之地(gstructs→gvalid/gconv/gdb/gcmd/goai),所以这个收益不限于 gvalid。net/ghttp请求校验的两个点(标准 handler 的单结构体与结构体切片,
ghttp_request_param.go)改用gvalid.New(true)显式开启上述缓存。这里的校验规则全部来自 struct tag,编译期即固定,缓存条目数与"代码里写过的 tag 条数"一致,不会随时间增长。结果
口径:Linux 容器(
golang:1.25),改动前后交错各跑 6 次取中位数,-benchmem,基线为提交前的e8d22526a。请求校验路径(
ghttp,缓存开启)基准以
gvalid.New(true)构造(与ghttp一致);基线侧用等价的gvalid.New()(基线代码本身没有缓存)。CheckStruct_Data(6 字段)CheckStruct_Assoc(标准 handler 的用法)CheckStruct_BigReq(24 字段 8 规则,贴近真实 Req)ParseTagValue(对照组,公开 API)用户默认路径(
gvalid.New(),不开启缓存)用户现有代码走的路径:缓存之外的优化(消息直通、规则项拆分、Clone、gtag 短路)对所有调用者生效,无需任何改动或选择。
CheckStruct_DataCheckStruct_AssocCheckStruct_BigReq对照组说明:公开 API
ParseTagValue的耗时与内存逐字节不变,说明语义没有被改动,收益全部落在 validator 路径上。兼容性
gvalid.New()的调用方式不变(新增的是一个可选参数),ParseTagValue行为不变。New()默认不缓存,用户现有代码的校验行为与旧版一致(同时仍获得上表"默认路径"的 -13%~-25% 提升,因为它来自与缓存无关的那几项优化)。ghttp的两个校验点已显式开启缓存,升级后标准 handler 的请求校验自动获得上表"请求校验路径"的提升,用户无需任何配置。gvalid.New(true)获得与请求路径相同的收益。regex/not-regex(不带参数)与后一项按|合并时,新旧实现解析出的规则名不同,但两种实现在该写法下都是恒通过,实际不可观测。测试
Test_New_Cache:默认构造不写缓存、New(true)才写(钉住"缓存 opt-in"这一契约);Test_CustomError_MapStringString:map[string]string消息直通,并断言输入 map 不被校验改写;Test_Rule_PatternContainingColon:regex:^\d+:\w+$这类参数含冒号的规则正常;Test_Rule_NonAsciiName:非 ASCII 规则名不再 panic;Test_Parse_NoPlaceholder:gtag.Parse短路的两条分支(无占位符 / 未注册占位符);Test_Validator_Clone_Independence:链式校验互不污染、基座实例不被派生操作改动(Clone()改写的契约)。util/gvalid/gvalid_z_bench_check_test.go,含 6 字段 / 24 字段 / Assoc 三个场景,以gvalid.New(true)构造(与ghttp一致),并带 sanity 测试保证基准真实触发校验。util/gvalid、util/gtag、os/gstructs、util/gconv、util/gmeta、util/gutil全部通过;net/ghttp(Request/Params 定向用例)与os/gcmd通过。复现