Skip to content

pkg/schema: optimize parsing of superset #1790

@bradfitz

Description

@bradfitz

This unmarshaling allocates new strings when JSON parsing, rather than using common interned string values for CamliType, ClaimType, etc.

func parseSuperset(r io.Reader) (*superset, error) {
	buf := pools.BytesBuffer()
	defer pools.PutBuffer(buf)

	n, err := io.CopyN(buf, r, MaxSchemaBlobSize+1)
	if err != nil && err != io.EOF {
		return nil, err
	}
	if n > MaxSchemaBlobSize {
		return nil, errSchemaBlobTooLarge
	}

	ss := new(superset)
	if err := json.Unmarshal(buf.Bytes(), ss); err != nil {
		return nil, err
	}
	return ss, nil
}

If we used json/v2, we could hook into the []byte decoding at various levels and re-use common string values to allocate less garbage. (useful during reindexes)

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions