Changelog

Features

• M3DB: Performance increases for block rotation by streamlining indexing lock contention (#2423)
• M3DB: Zero-copy of ID and fields on series index metadata re-indexing (#2423)
• M3Coordinator: Add ability to restrict and block incoming series based on tag matchers (#2430)

Bug Fixes

• M3DB: Fix an error where background compaction caused transient errors in queries (#2432)

Documentation

• M3Query: Update config settings and cleaned up documentation for per query limits (#2427)

Changelog

Features

• M3DB: Ability to set per-query block limit (#2415)
• M3DB: Ability to set global per-second query limit (#2405)

Bug Fixes

• M3DB: Fix duplicate ID insertions causing transient error when flushing index block (#2411)
• M3Coordinator: Mapping rules with drop policies now correctly apply to unaggregated metrics (#2262)
• M3Query: Fix incorrect starting boundaries on some temporal queries (#2413)
• M3Query: Fix bug in one to one matching in binary functions (#2417)
• M3DB: Fix to edge case index data consistency on flush (#2399)

Changelog

Bug Fixes

• M3DB: Fix require exhaustive propagation of require exhaustive option through RPC (#2409)

Changelog

Features

• M3DB: Add ability to return an error when max time series limit is hit instead of partial result and warning (#2400)
• M3Coordinator: Add support for namespace retention updates by API (#2383)

Bug Fixes

• M3Coordinator: Fix Content-Type for OpenAPI handler (#2403)
• Build: Build release binaries with goreleaser using Go 1.13 to match Go 1.13 docker images (#2397)

Misc

• M3DB: Report a histogram of series blocks fetched per query (#2381)

Changelog

Features

• M3Ctl: Add M3 command line tool for calling APIs and using YAML files to apply commands (#2097)
• M3Coordinator: Add public API to write annotations (i.e. arbitrary bytes), next to datapoints for things like storing exemplars (#2022, #2029, #2031)
• M3Coordinator: Add support for mapping rules, allowing metrics to be stored at different resolutions based on their labels/tags (#2036)
• M3Coordinator: Add Graphite mapping rule support (#2060) (#2063)
• M3Coordinator: Add community contributed InfluxDB write endpoint (at /api/v1/influxdb/write) (#2083)
• M3Coordinator: Add headers to pass along with request to remote write forward targets (#2249)
• M3Coordinator: Add retry to remote write forward targets (#2299)
• M3Coordinator: Add in-place M3Msg topic consumer updates with a PUT request (#2186)
• M3Coordinator: Add ability to rewrite tags for Prometheus remote write requests using header (#2255)
• M3Coordinator: Add config for multi-process launcher and SO_REUSEPORT listen servers for non-container based multi-process scaling (#2292)
• M3Query: Add Prometheus engine to compliment Prometheus Remote Read, improves performance by skipping serialization/deserialization/network overhead between Prometheus and M3Query (#2343, #2369)
• M3Query: Add header to support enforcing all queries in request to implicitly always include a given label/tag matcher (#2053)
• M3Query: Return headers indicating incomplete results for cross-regional fanout queries when remote fails or hits a limit (#2053)
• M3Query: Refactor query server to allow for custom handlers (#2073)
• M3Query: Add remote read debug parameters to look at raw data for a PromQL query and/or get results as JSON (#2276)
• M3Query: Add warnings for Prometheus queries to Prometheus query JSON response (#2265)
• M3Query: Add ability to set default query timeout by config (#2226)
• M3Aggregator: Add M3Msg aggregator client for high throughput point to point clustered buffered delivery of metrics to aggregator (#2171)
• M3Aggregator: Add rollup rule support for metrics aggregated with pre-existing timestamps, such as Prometheus metrics (#2251)
• M3Aggregator: Add aggregator passthrough functionality for aggregation in a local region forwarding to a remote region for storage (#2235)

Performance

• M3DB: Improve RSS memory management with madvise resulting in flat RSS usage with a steady workload as time passes block-over-block (#2037)
• M3DB: Improve bootstrapping performance by allowing bootstrapping to be performed in a single pass, now possible for a lot of bootstraps to take just minutes depending on retention (#1989)
• M3DB: Use zero-copy references to index data instead of copy-on-read index data for each query, substantially improving query throughput and performance (#1839)
• M3DB: Further improve peer bootstrapping performance by using a document builder rather than raw memory segments (#2078)
• M3DB: Concurrent indexing when building segments for newly inserted metrics (#2146)
• M3DB: Decode ReadBits decompression improvements (#2197)
• M3DB: Remove implicit cloning of time ranges to reduce allocs (#2178)
• M3Query: Substantially improve temporal function performance (#2049)
• M3Query: Improve datapoint decompression speed (#2176, #2185, #2190)
• M3Query: Read bits uses an optimized byte reader (#2205)
• M3Coordinator: Ensure coordinator not grow M3Msg buffer if message over max size (#2207)

Bug Fixes

• M3Aggregator: Take last value by wall clock timestamp not arrival time to avoid late arrivals overwriting actual later occuring values (#2199)
• M3DB: Validate indexable metrics for valid utf-8 prior to insert, also includes a utility for earlier M3DB versions to remove non-utf8 index data (#2046)
• M3DB: Remove incorrect error log message for missing schema with default non-protobuf namespaces (#2013)
• M3DB: Fixed memory leak causing index blocks to remain in memory after flushing (#2037)
• M3DB: Fix long standing possibility of double RLock acqusition (#2128)
• M3DB: Remove loop in fileset writer when previous fileset encountered an error writing out index files (#2058)
• M3DB: Instead of encountering an error skip entries for unowned shards in commit log bootstrapper (#2145)
• M3DB: Fix to avoid returning error when missing writable bucket with a cold flush (#2188)
• M3DB: Set defaults and expose configuration of TChannel timeouts, this avoids idle connection growth (#2173)
• M3DB: Account for Neg/Pos Offsets when building per field roaring bitmap posting lists (#2213)
• M3DB: Fix to build flush errors (#2229, #2217)
• M3Coordinator: Respect env and zone headers for topic API endpoints (#2159)
• M3Coordinator: Add support for Graphite Grafana plugin /find POST requests (#2153)
• M3Coordinator: Use tag options specified in config with M3Msg ingester (#2212)
• M3Coordinator: Only honor default aggregation policies if not matched by mapping rule (#2203)
• M3Query: Fix namespace resolve debug log not being written with multiple namespaces (#2211)
• M3Query: Fix to temporal function regression leading to inconsistent results (#2231)
• M3Query: Fix edge cases with cross-zonal query fanout and add verify utility (#1993)
• M3Query: Fix issue with histogram grouping (#2247)

Documentation

• M3Aggregator: Add M3 aggregator Grafana dashboard (#2064)
• M3Coordinator: Add documentation to write to multiple clusters from a single coordinator (#2187)
• M3DB: Add documentation about estimating number of unique time series (#2062)
• M3DB: Update namespace configuration documentation to use simpler duration specific keys (#2045)

Misc

• All: Upgrade to Go 1.13 and switch dependency management to Go modules (#2221)
• All: Add gauge metrics to measure the number of active routines for any worker pool (#2061)
• All: Allow for ${ENV_VAR_NAME} expansion with YAML configuration files (#2033)
• All: Add a utility for comparing performance and correctness across different versions of M3DB, enabling diffing the perf of different versions (#2044)
• All: Upgrade etcd client library to 3.4.3 (#2101)
• All: Include key name in watch errors (#2138)
• Development: Add HA Prometheus lab setup for dev M3 docker compose deployment (#2206)
• Development: Temporarily disable kubeval validation to allow builds on go 1.12 (#2241)
• Development: Add comparator value ingester for replaying functions against given data sets (#2224)
• Development: Logging improvements (#2222,#2225)
• M3Aggregator: Add a datasource variable and reuse it in all the panels of the aggregator dashboard (#2182)
• M3DB: Add client bad request/internal error distinction for metrics and sampled logs (#2201)
• M3DB: Add latency me...