Release v0.13.0-nightly-20250303

What's Changed

• docs: tsbs update by @discord9 in #5608
• chore: bump version to 0.13.0 by @WenyXu in #5611
• refactor: add pipeline concept to OTLP traces and remove OTLP over gRPC by @sunng87 in #5605
• feat: add vec_dim function by @xiaoniaoyouhuajiang in #5587
• fix: check physical region before use by @waynexia in #5612
• fix: update typos rules to fix ci by @yihong0618 in #5621

New Contributors

• @xiaoniaoyouhuajiang made their first contribution in #5587

Full Changelog: v0.12.0...v0.13.0-nightly-20250303

Release v0.12.0

v0.12.0

Release date: February 27, 2025

👍 Highlights

• Added PromQL subquery support, series count metrics, and new functions like sort and sort_desc.
• Introduced Jaeger Query API
• Improved performance for last_non_null merge mode (10x faster).
• Optimized metric table creation, deletion and alteration speeds (100x faster).
• Introduced new functions like vec_add, hll_state, and uddsketch for advanced analytics.
• Fixed PromQL-related issues, including unescaped matcher values and incorrect behavior for non-existent columns.

Breaking changes

• refactor!: unify the option names across all components by @fengjiachun in #5457
• refactor!: unify the option names across all components part2 by @fengjiachun in #5476
• feat!: unify all index creation grammars by @waynexia in #5486
• refactor!: remove datetime type by @fengjiachun in #5506
• feat!: support alter skipping index by @waynexia in #5538

🚀 Features

• feat: Address different Metrics for Prometheus queries in the Dashboard and fix typo in metric name by @Stephan3555 in #5441
• feat(metric-engine): support to write rows with sparse primary key encoding by @WenyXu in #5424
• feat: add CORS headers for http interfaces by @sunng87 in #5447
• feat: mirror insert request to flownode in async by @waynexia in #5444
• feat: expose http endpoint for flownode and metasrv by @waynexia in #5437
• feat: provide options to disable or customize http corss-origin settings by @sunng87 in #5450
• feat(cli): add proxy options by @discord9 in #5459
• feat: add vec_add function by @zhongzc in #5471
• feat: write memtable in parallel by @waynexia in #5456
• feat: update topic-region map when create and drop table by @CookiePieWw in #5423
• feat: move pipelines to the first-class endpoint by @waynexia in #5480
• feat: pipeline dispatcher part 2: execution by @sunng87 in #5409
• feat: add metadata method to puffin reader by @zhongzc in #5501
• feat: update dashboard to v0.7.9 by @ZonaHex in #5508
• feat: change puffin stager eviction policy by @zhongzc in #5511
• feat: update dashboard to v0.7.10-rc by @ZonaHex in #5512
• feat: support server-side keep-alive for mysql and pg protocols by @waynexia in #5496
• feat: implement Jaeger query APIs by @zyy17 in #5452
• feat: alias database matcher for promql by @sunng87 in #5522
• feat: speed up read/write cache and stager eviction by @evenyag in #5531
• feat: window sort supports where on fields and time index by @evenyag in #5527
• feat: add stager nofitier to collect metrics by @zhongzc in #5530
• feat(promql): add series count metrics by @waynexia in #5534
• feat: add snapshot seqs field to query context by @discord9 in #5477
• feat: drop noneffective regex filter by @waynexia in #5544
• feat(log-query): support specifying exclusive/inclusive for between filter by @waynexia in #5546
• feat: collect stager metrics by @evenyag in #5553
• feat: update dashboard to v0.7.10 by @ZonaHex in #5562
• feat: allow purging a given puffin file in staging area by @zhongzc in #5558
• feat(promql): supports sort, sort_desc etc. functions by @killme2008 in #5542
• feat(log-query): implement the first part of log query expr by @waynexia in #5548
• feat: submit node's cpu cores number to metasrv in heartbeat by @MichaelScofield in #5571
• feat: flow type on creating procedure by @fengjiachun in #5572
• feat: remap flow route address by @fengjiachun in #5565
• feat: enable gzip for prometheus query handlers and ignore NaN values in prometheus response by @WenyXu in #5576
• feat: implement uddsketch function to calculate percentile by @waynexia in #5574
• feat: support UNNEST by @waynexia in #5580
• feat: support to generate json output for explain analyze in http api by @waynexia in #5567
• feat: run sqlness in parallel by @waynexia in #5499
• feat: unify puffin name passed to stager by @zhongzc in #5564
• feat: remove default inverted index for physical table by @waynexia in #5583
• feat: impl hll_state, hll_merge and hll_calc for incremental distinct counting by @waynexia in #5579
• feat: update dashboard to v0.7.11 by @ZonaHex in #5597
• feat(promql): ignore invalid input in histogram plan by @waynexia in #5607
• feat(promql): implement subquery by @waynexia in #5606
• feat: introduce prom_round fn by @WenyXu in #5604
• feat(promql-planner): introduce vector matching binary operation by @WenyXu in #5578

🐛 Bug Fixes

• fix: unexpected warning on applying bloom by @zhongzc in #5431
• fix: arm actions test failed by @yihong0618 in #5433
• fix: install x86-64 protoc on android dev-builder by @zyy17 in #5443
• fix: drop unused numpy code since pyo3 rustpython do not support any more by @yihong0618 in #5442
• fix: flush table panic when table has interval column close #3235 by @yihong0618 in #5422
• fix: remove metric engine's internal column from promql's query by @waynexia in [#5032](#5032...

Release v0.12.0-nightly-20250224

What's Changed

• ci: use s5cmd upload artifacts by @daviderli614 in #5550
• ci: don't push nightly latest image by @daviderli614 in #5551
• feat(log-query): support specifying exclusive/inclusive for between filter by @waynexia in #5546
• feat: collect stager metrics by @evenyag in #5553
• fix: information_schema.cluster_info be covered by the same id by @fengjiachun in #5555
• fix: correct promql behavior on nonexistent columns by @waynexia in #5547
• fix: window sort support alias time index by @evenyag in #5543
• chore: improve grafana dashboard by @daviderli614 in #5559
• feat: update dashboard to v0.7.10 by @ZonaHex in #5562
• chore: format all toml files by @yihong0618 in #5529
• fix(promql-planner): remove le tag in ctx by @WenyXu in #5560
• feat: allow purging a given puffin file in staging area by @zhongzc in #5558
• fix(promql-planner): correct AND/UNLESS operator behavior by @WenyXu in #5557
• refactor: use global type alias for pipeline input by @sunng87 in #5568
• fix(promql-planner): update ctx field columns of OR operator by @WenyXu in #5556
• feat(promql): supports sort, sort_desc etc. functions by @killme2008 in #5542
• chore: support Loki's structured metadata for ingestion by @shuiyisong in #5541
• feat(log-query): implement the first part of log query expr by @waynexia in #5548
• feat: submit node's cpu cores number to metasrv in heartbeat by @MichaelScofield in #5571
• feat: flow type on creating procedure by @fengjiachun in #5572
• feat: remap flow route address by @fengjiachun in #5565
• feat: enable gzip for prometheus query handlers and ignore NaN values in prometheus response by @WenyXu in #5576
• docs: revise the author list by @beryl678 in #5575
• feat: implement uddsketch function to calculate percentile by @waynexia in #5574
• feat: support UNNEST by @waynexia in #5580
• feat: support to generate json output for explain analyze in http api by @waynexia in #5567
• fix: broken link in AUTHOR.md by @waynexia in #5581
• feat: run sqlness in parallel by @waynexia in #5499
• feat: unify puffin name passed to stager by @zhongzc in #5564
• fix: avoid run labeler job concurrently by @waynexia in #5584
• feat: remove default inverted index for physical table by @waynexia in #5583
• fix: correct inverted_indexed_column_ids behavior by @WenyXu in #5586

New Contributors

• @beryl678 made their first contribution in #5575

Full Changelog: v0.12.0-nightly-20250217...v0.12.0-nightly-20250224

Release v0.12.0-nightly-20250217

What's Changed

• fix: close issue #5466 by do not shortcut the drop command by @yihong0618 in #5467
• fix: refactor pgkv using prepare_cache about 10% better by @yihong0618 in #5497
• chore: revert "docs: add TM to logos" by @sunng87 in #5495
• refactor: change traversal order during index construction by @zhongzc in #5498
• feat: add metadata method to puffin reader by @zhongzc in #5501
• ci: safe ci using zizmor check by @yihong0618 in #5491
• feat: update dashboard to v0.7.9 by @ZonaHex in #5508
• fix: drop useless clone and for loop second by @yihong0618 in #5507
• feat: change puffin stager eviction policy by @zhongzc in #5511
• ci: run nightly jobs only on greptimedb repo by @MichaelScofield in #5505
• feat: update dashboard to v0.7.10-rc by @ZonaHex in #5512
• chore: set now as timestamp field default value by @paomian in #5502
• feat: support server-side keep-alive for mysql and pg protocols by @waynexia in #5496
• feat!: unify all index creation grammars by @waynexia in #5486
• refactor: refactor region server request handling by @WenyXu in #5504
• ci: docbot requires pull_request_target by @sunng87 in #5514
• fix: use fixed tonistiigi/binfmt:qemu-v7.0.0-28 image version instead of latest version to avoid segmentation fault by @zyy17 in #5516
• feat: implement Jaeger query APIs by @zyy17 in #5452
• fix(query_range): skip data field on errors by @WenyXu in #5520
• perf: better performance for LastNonNullIter close #5229 about 10x times faster by @yihong0618 in #5518
• perf: optimize table creation speed in metric engine by @WenyXu in #5503
• refactor!: remove datetime type by @fengjiachun in #5506
• feat: alias database matcher for promql by @sunng87 in #5522
• fix(promql): unescape matcher values by @WenyXu in #5521
• feat: speed up read/write cache and stager eviction by @evenyag in #5531
• chore: use the same version of chrono-tz by @yihong0618 in #5523
• ci: use the repository variable to pass to image-name by @daviderli614 in #5517
• feat: window sort supports where on fields and time index by @evenyag in #5527
• fix: old typo by @waynexia in #5532
• chore: add section marker for extenal dependencies by @sunng87 in #5536
• fix(jaeger): return error when no tracing table by @zyy17 in #5539
• fix(promql): ignore filters for non-existent labels by @WenyXu in #5519
• feat: add stager nofitier to collect metrics by @zhongzc in #5530
• feat(promql): add series count metrics by @waynexia in #5534
• refactor: refactor pg kvbackend impl in preparation for other rds kvbackend by @CookiePieWw in #5494
• perf: optimize table alteration speed in metric engine by @WenyXu in #5526
• fix: promql join operation won't consider time index by @waynexia in #5535
• feat: add snapshot seqs field to query context by @discord9 in #5477
• feat!: support alter skipping index by @waynexia in #5538
• feat: drop noneffective regex filter by @waynexia in #5544

Full Changelog: v0.12.0-nightly-20250210...v0.12.0-nightly-20250217

Release v0.12.0-nightly-20250210

What's Changed

• docs: the year is better to show in 2025 by @yihong0618 in #5468
• feat(cli): add proxy options by @discord9 in #5459
• refactor(mito): Allow creating multiple files in ParquetWriter by @v0y4g3r in #5291
• docs: fix memory perf command wrong by @yihong0618 in #5470
• fix: don't transform Limit in TypeConversionRule, StringNormalizationRule and DistPlannerAnalyzer by @evenyag in #5472
• refactor!: unify the option names across all components by @fengjiachun in #5457
• feat: add vec_add function by @zhongzc in #5471
• perf: optimize time series memtable ingestion by @waynexia in #5451
• feat: write memtable in parallel by @waynexia in #5456
• refactor: pull column filling logic out of mito worker loop by @waynexia in #5455
• fix: introduce gc task for metadata store by @WenyXu in #5461
• fix: lose decimal precision when using decimal type as tag by @evenyag in #5481
• fix: Delete statement not supported in metric engine close #4649 by @yihong0618 in #5473
• refactor: otlp logs insertion by @shuiyisong in #5479
• fix: unquote flow_name in create flow expr by @discord9 in #5483
• fix: force recycle region dir after gc duration by @evenyag in #5485
• build: Update Loki proto by @ozewr in #5484
• fix: cross compiling for aarch64 targets and allow customizing page size by @v0y4g3r in #5487
• chore: update pprof by @discord9 in #5488
• refactor!: unify the option names across all components part2 by @fengjiachun in #5476
• feat: update topic-region map when create and drop table by @CookiePieWw in #5423
• feat: move pipelines to the first-class endpoint by @waynexia in #5480
• feat: pipeline dispatcher part 2: execution by @sunng87 in #5409

Full Changelog: v0.12.0-nightly-20250203...v0.12.0-nightly-20250210

Release v0.12.0-nightly-20250203

What's Changed

• perf: optimize writing non-null primitive value by @waynexia in #5460
• ci: move components to flakes so it won't affect builders by @sunng87 in #5464

Full Changelog: v0.12.0-nightly-20250127...v0.12.0-nightly-20250203

Release v0.12.0-nightly-20250127

What's Changed

• ci: use arm builders for tests by @sunng87 in #5395
• feat(metric-engine): introduce experimental_sparse_primary_key_encoding to MetricEngineConfig by @WenyXu in #5373
• ci: revert coverage runner by @sunng87 in #5403
• fix: better sqlness show, replace the unwarp with better show message by @yihong0618 in #5400
• chore(http): change authorization header by @v0y4g3r in #5389
• refactor: use MetadataKey for kafka topic by @CookiePieWw in #5351
• feat(metric-engine): introduce index options from metric engine by @WenyXu in #5374
• fix: matches incorrectly uses byte len as char len by @zhongzc in #5411
• chore: extract service_name in OTLP traces by default by @shuiyisong in #5412
• feat: add column if not exists by @NiwakaDev in #5393
• chore: better error msg by @discord9 in #5415
• feat(grafana): update cluster dashboard to include flow-related metrics by @waynexia in #5419
• feat: bloom filter as fulltext index v2 (Part 1) by @zhongzc in #5406
• fix: vector function for PromQL need to ignore the time index also by @yihong0618 in #5398
• feat: support not-equal matcher for PromQL metric names by @killme2008 in #5385
• feat(metric-engine): introduce RowModifier for MetricEngine by @WenyXu in #5380
• fix: avoid suppress manual compaction by @v0y4g3r in #5399
• feat(remote-wal): introduce TopicRegionManager by @CookiePieWw in #5407
• feat!: Bump datafusion, prost, hyper, tonic, tower, axum by @evenyag in #5417
• feat: replace DensePrimaryKeyCodec with Arc<dyn PrimaryKeyCodec> by @WenyXu in #5408
• refactor: support to flatten json object in greptime_identity pipeline by @zyy17 in #5358
• ci: update dev-builder version by @zyy17 in #5435
• fix(log-query): panic on prometheus by @waynexia in #5429
• ci: allow skipping tests as required tasks by @sunng87 in #5436
• fix: unexpected warning on applying bloom by @zhongzc in #5431
• feat: Address different Metrics for Prometheus queries in the Dashboard and fix typo in metric name by @Stephan3555 in #5441
• ci: unify all protoc version to 29.3 by @zyy17 in #5434
• ci: switch to nix flakes for more reproducible builds by @sunng87 in #5426
• fix: arm actions test failed by @yihong0618 in #5433
• fix: install x86-64 protoc on android dev-builder by @zyy17 in #5443
• ci: update dev-builder version to fix build android image failed by @zyy17 in #5445
• feat(metric-engine): support to write rows with sparse primary key encoding by @WenyXu in #5424
• fix: drop unused numpy code since pyo3 rustpython do not support any more by @yihong0618 in #5442
• feat: add CORS headers for http interfaces by @sunng87 in #5447
• chore: update toolchain to 2024-12-25 by @shuiyisong in #5430
• feat: mirror insert request to flownode in async by @waynexia in #5444
• fix: flush table panic when table has interval column close #3235 by @yihong0618 in #5422
• feat: expose http endpoint for flownode and metasrv by @waynexia in #5437
• chore: avoid necessary cloning by @WenyXu in #5454
• fix: remove metric engine's internal column from promql's query by @waynexia in #5032
• fix: better error handler for the time range close #5449 by @yihong0618 in #5453
• feat: provide options to disable or customize http corss-origin settings by @sunng87 in #5450
• fix: no need for special case since datafusion updated by @yihong0618 in #5458

New Contributors

• @Stephan3555 made their first contribution in #5441

Full Changelog: v0.12.0-nightly-20250120...v0.12.0-nightly-20250127

Release v0.11.3

v0.11.3

Release date: January 24, 2025

This version fixes the following issues:

• Panics when processing matches() or invalid queries.
• Altering compaction time window doesn't take effect.

🚀 Features

• feat: set default compaction parallelism by @waynexia in #5371
• feat: overwrites inferred compaction window by region options by @evenyag in #5396

🐛 Bug Fixes

• fix: handle insert default value by @discord9 in #5307
• fix: matches incorrectly uses byte len as char len by @zhongzc in #5411
• fix: panic when received invalid query string by @waynexia in #5366
• fix: avoid suppress manual compaction by @v0y4g3r in #5399
• fix(log-query): panic on prometheus by @waynexia in #5429

🚜 Refactor

• refactor: optimize out partition split insert requests by @MichaelScofield in #5298

⚙️ Miscellaneous Tasks

• ci: disable docker/rust cache temporarily and merge docker compose files by @sunng87 in #5293
• ci: do not trigger tests when there is a merge conflict by @sunng87 in #5318
• ci: use mold for tests by @sunng87 in #5319
• ci: disable cache for some tasks, create cache in nightly build by @sunng87 in #5324
• ci: do not collect coverage from pull request any more by @sunng87 in #5364
• ci: automatically bump doc version when release GreptimeDB by @nicecui in #5343
• ci: use arm builders for tests by @sunng87 in #5395
• ci: revert coverage runner by @sunng87 in #5403

All Contributors

We would like to thank the following contributors from the GreptimeDB community:

@MichaelScofield, @discord9, @evenyag, @nicecui, @sunng87, @v0y4g3r, @waynexia, @zhongzc

Release v0.12.0-nightly-20250120

What's Changed

• chore: drop useless import raw_normalize_path in object-store lib by @yihong0618 in #5349
• fix: skip building indexer when indexed_column_ids are empty by @WenyXu in #5348
• refactor(elasticsearch): use _index as greptimedb table in log ingestion and add /${index}/_bulk API by @zyy17 in #5335
• feat: update dashboard to v0.7.7 by @ZonaHex in #5350
• refactor: Bump opendal to 0.51.1 by @Xuanwo in #5354
• fix: security fix, sqlx, hashbrown, idna and CI updates by @sunng87 in #5330
• ci: automatically bump doc version when release GreptimeDB by @nicecui in #5343
• feat: update dashboard to v0.7.8 by @ZonaHex in #5355
• fix: handle insert default value by @discord9 in #5307
• feat: more than one flow workers by @discord9 in #5315
• fix: improve error handling in pipeline and log in log store by @waynexia in #5357
• chore: avoid sending create table requests for already existing tables by @WenyXu in #5347
• feat: introduce PrimaryKeyEncoding by @WenyXu in #5312
• fix: panic and interval when do not have keyword interval by @yihong0618 in #5339
• feat: support pg_database for DBeaver. by @yihong0618 in #5362
• fix: better makefile help show, show fuzz and fuzz-ls also by @yihong0618 in #5363
• feat(flow): refill flow task def(Part 2) by @discord9 in #5317
• ci: do not collect coverage from pull request any more by @sunng87 in #5364
• feat: refine log query AST by @waynexia in #5316
• fix: redirect /dashboard to /dashboard/ by @sunng87 in #5369
• feat: introduce SparsePrimaryKeyCodec and SparsePrimaryKeyFilter by @WenyXu in #5365
• feat: update dashboard to v0.7.9-rc.1 by @ZonaHex in #5368
• feat: filter batch by sequence in memtable by @discord9 in #5367
• feat: Alter inverted index by @lyang24 in #5131
• feat: flow's http server by @discord9 in #5372
• chore: update proto rev by @CookiePieWw in #5379
• fix!: correct index cache config by @WenyXu in #5381
• feat: digest pipeline processor by @waynexia in #5323
• fix: panic when received invalid query string by @waynexia in #5366
• feat: set default compaction parallelism by @waynexia in #5371
• test: fix config api test by @evenyag in #5386
• fix: increase in progress scan gauge and adjust histogram buckets by @evenyag in #5370
• feat!: use indirect indices for bloom filter to reduce size by @zhongzc in #5377
• fix: make flow worker actually run in parallel by @discord9 in #5384
• fix: maybe double free from static str in Snafu by @yihong0618 in #5383
• ci: always build standard db by @evenyag in #5390
• refactor: drop useless clone for some better performance using static chek by @yihong0618 in #5388
• feat: pipeline dispatch part 1, add definition by @sunng87 in #5359
• feat: overwrites inferred compaction window by region options by @evenyag in #5396
• chore: change default compaction output size limit to 2GB by @v0y4g3r in #5378
• fix: drop unused pub fn using to cargo_workspace_unused by @yihong0618 in #5352
• fix(flow): deal with flow drop leftover by @discord9 in #5391

Full Changelog: v0.12.0-nightly-20250113...v0.12.0-nightly-20250120

Release v0.12.0-nightly-20250113

What's Changed

• feat: add Txn for pg kv backend by @CookiePieWw in #5266
• feat: impl COPY a query resultset to external file by @KKould in #5250
• fix(fuzz): ensure all regions leases are renewed by @WenyXu in #5294
• feat: support Loki JSON write by @shuiyisong in #5288
• feat: introduce ParallelFstValuesMapper by @WenyXu in #5276
• ci: disable docker/rust cache temporarily and merge docker compose files by @sunng87 in #5293
• feat(pipeline): allow coerce timestamp from integer values by @sunng87 in #5270
• chore: cleanup dockerfile by @discord9 in #5299
• fix: pass unknown encoding in decompression layer by @shuiyisong in #5300
• feat: update standalone grafana with new metric name by @waynexia in #5278
• fix(bloom-filter): filter rows with segment precision by @zhongzc in #5286
• fix(pg_backend): correct set idle_in_transaction_session_timeout statement by @WenyXu in #5304
• docs: Added C/C++ building essentials to the prerequisites list by @v0y4g3r in #5302
• refactor: refactor PgStore by @WenyXu in #5309
• chore(nightly-ci): remove pg_kvbackend feature gate in windows ci by @WenyXu in #5310
• fix: show index to display skipping index by @zhongzc in #5297
• fix: drop unused dep using udeps to minial the size by @yihong0618 in #5301
• feat(flow): flow refill state (Part 1) by @discord9 in #5295
• refactor: optimize out partition split insert requests by @MichaelScofield in #5298
• perf: parse Loki labels in protobuf write path by @shuiyisong in #5305
• ci: do not trigger tests when there is a merge conflict by @sunng87 in #5318
• feat: support select session_user; by @yihong0618 in #5313
• feat: auto detecting local IP to use as hostname by @WenyXu in #5314
• feat!: Remove script crate and python feature by @evenyag in #5321
• ci: use mold for tests by @sunng87 in #5319
• feat: support elasticsearch _bulk API to ingest logs by @zyy17 in #5261
• feat(config): make table name of pg backend configurable by @CookiePieWw in #5244
• ci: drop useless deadsnake by @yihong0618 in #5311
• feat: Add VEC_PRODUCT, VEC_ELEM_PRODUCT, VEC_NORM. by @linyihai in #5303
• fix: drop all python embedding code for docker and doc by @yihong0618 in #5325
• ci: disable cache for some tasks, create cache in nightly build by @sunng87 in #5324
• build: disable local IP detection feature in Android binary by @WenyXu in #5327
• fix: android build by @yihong0618 in #5329
• feat: add show search_path for pg by @yihong0618 in #5328
• feat: support alter add multiple columns by @NiwakaDev in #5262
• chore: update rustls by @discord9 in #5337
• feat: update dashboard to v0.7.4 by @ZonaHex in #5336
• feat: add set search_path to 'xxx' for pg by @yihong0618 in #5342
• feat: update dashboard to v0.7.6 by @ZonaHex in #5340
• fix(test): use different table name by @CookiePieWw in #5334

Full Changelog: v0.12.0-nightly-20250106...v0.12.0-nightly-20250113