v1.2.0

Major Features 🔥

• Introducing Session Window and Streaming Reduce Capability
• Ability to pass headers from Source to Sink
• Fallback Sink

🚨 Breaking Change 🚨

• SDKs have to be updated to the latest version
• Python SDK has interface changes.

New Features 🔥

• feat: health status implementation by @kohlisid in #1406
• feat: Session Window and Reduce Streaming by @yhl25 in #1384
• feat: update tcp client connections by @kohlisid in #1429
• feat: enhance autoscaling peeking logic by @akashjkhamkar in #1432
• feat: pipeline health status for UI by @veds-g in #1460
• feat: configure standardResources via controller configmap by @inishchith in #1490
• feat: add support for SASL SCRAM 256 and 512 for Kafka by @Dnomyar in #1518
• feat: noop persistence store for reduce by @yhl25 in #1532
• feat: unaligned wal by @yhl25 in #1511
• feat: numaflow package by @veds-g in #1579
• feat: support headers for message by @yhl25 in #1578
• feat: Fallback Sink by @yhl25 in #1669

Major Fixes 🐞

• fix: terminate reduce vertex pods when pausing pipeline by @dpadhiar in #1481
• fix: configmap const name by @whynowy in #1423
• fix: server-secrets-init container restart by @veds-g in #1433
• fix: Kafka source reads duplicated messages by @jy4096 in #1438
• fix: UI Filter by status for pipelines doesn't work as expected by @inishchith in #1444
• fix: GetDownstreamEdges is not cycle safe by @inishchith in #1447
• fix: memory leak inside session windower by @yhl25 in #1445
• fix: bug where dashed line is treated as permalink + spelling/formatting by @ayildirim21 in #1467
• fix(controller): incorrect cpu/mem resources calculation by @whynowy in #1477
• fix: add idle handler offset nil check by @jy4096 in #1489
• fix: unknown for ISB details in pipeline card by @veds-g in #1497
• fix: avoid panic when ctx is canceled by @yhl25 in #1515
• fix: idle manager refactor for multi partitions by @jy4096 in #1512
• fix: podSpec incorrectly configured in case template exist by @inishchith in #1516
• fix: incorrect json schema for tls config by @whynowy in #1520
• fix(controller): vertex template metadata nil check by @inishchith in #1527
• fix: break from retry loop when key is not found by @yhl25 in #1535
• fix: initialize inflightAcks channel to not nil channel by @afugazzotto in #1548
• fix(codegen): protobuf upgrade by @whynowy in #1558
• fix: error message that cause Buttons overflow in the UI by @a49ibrahim in #1591
• fix: flaky e2e tests by @vigith in #1590
• fix: dedup in user-defined source #1613
• fix: allow pipeline to start with redis isbsvc (Fixes: #1513) #1567
• fix: watermark progression during pods creation/deletion #1619
• fix: numaflow package style fixes in #1622
• fix: bug in late message handling for sliding window #1471
• fix: race condition while publishing wm inside reduce #1599
• fix: read-only view for UI #1628

New Contributors ❤️

• @akashjkhamkar made their first contribution in #1432
• @inishchith made their first contribution in #1444
• @ayildirim21 made their first contribution in #1467
• @syayi made their first contribution in #1510
• @Dnomyar made their first contribution in #1518
• @afugazzotto made their first contribution in #1548
• @a49ibrahim made their first contribution in #1591
• @QuentinFAIDIDE made their first contribution in #1567

Installation

kubectl create namespace numaflow-system

# Install numaflow controller and UI
kubectl apply -n numaflow-system -f https://github.com/numaproj/numaflow/releases/download/v1.2.0/install.yaml

# Install validating webhook (Optional)
kubectl apply -n numaflow-system -f https://github.com/numaproj/numaflow/releases/download/v1.2.0/validating-webhook-install.yaml

Full Changelog: v1.1.0...v1.2.0

v1.2.0-rc5

Major Changes

• Skip sdk compatibility check if server info does not contain sdk version (#1703)

Pull Requests

v1.2.0-rc4...v1.2.0-rc5

Installation

kubectl create namespace numaflow-system

# Install numaflow controller and UI
kubectl apply -n numaflow-system -f https://github.com/numaproj/numaflow/releases/download/v1.2.0-rc5/install.yaml

# Install validating webhook (Optional)
kubectl apply -n numaflow-system -f https://github.com/numaproj/numaflow/releases/download/v1.2.0-rc5/validating-webhook-install.yaml

v1.2.0-rc4

Major Features 🔥

• Fallback Sink #1660

🚨 Breaking Change 🚨

• SDKs have to be updated to the latest version, user-defined sink has a breaking change.

Other changes

• feat: expose controller leader election duration and renew opts #1657
• chore: remove pods/exec from UX server RBAC #1659
• chore: do not create init-secrets when auth is disabled #1661
• fix: routing fixes #1671
• fix: version downgrade for monaco-editor #1673
• chore: add version compatibility check #1683

Full Changelog: v1.2.0-rc3...v1.2.0-rc4

v1.2.0-rc3

Major Changes

• Fix: pass headers to custom UDSink (#1653)
• Fix: pass headers to Source Transformer (#1651)
• Fix: avoid publishing watermarks for duplicate messages (#1649)
• Fix: Dedup not working for multi-partitioned edge (#1639)
• Fix: readonly view (#1640)

Pull Requests

v1.2.0-rc2...v1.2.0-rc3

Installation

kubectl create namespace numaflow-system

# Install numaflow controller and UI
kubectl apply -n numaflow-system -f https://github.com/numaproj/numaflow/releases/download/v1.2.0-rc3/install.yaml

# Install validating webhook (Optional)
kubectl apply -n numaflow-system -f https://github.com/numaproj/numaflow/releases/download/v1.2.0-rc3/validating-webhook-install.yaml

v1.2.0-rc2

Major Changes

• Fix: dedup in user-defined source #1613
• Fix: allow pipeline to start with redis isbsvc (Fixes: #1513) #1567
• Fix: watermark progression during pods creation/deletion #1619
• Fix: numaflow package style fixes in #1622
• Fix: bug in late message handling for sliding window #1471
• Fix: race condition while publishing wm inside reduce #1599
• Feat: read-only view for UI #1628

Pull Requests

v1.2.0-rc1...v1.2.0-rc2

Installation

kubectl create namespace numaflow-system

# Install numaflow controller and UI
kubectl apply -n numaflow-system -f https://github.com/numaproj/numaflow/releases/download/v1.2.0-rc2/install.yaml

# Install validating webhook (Optional)
kubectl apply -n numaflow-system -f https://github.com/numaproj/numaflow/releases/download/v1.2.0-rc2/validating-webhook-install.yaml

v1.2.0-rc1

Major Features 🔥

• Introducing Session Window and Streaming Reduce Capability
• Ability to pass headers from Source to Sink

🚨 Breaking Change 🚨

• SDKs have to be updated to the latest version
• Python SDK has interface changes.

New Features 🔥

• feat: health status implementation by @kohlisid in #1406
• feat: Session Window and Reduce Streaming by @yhl25 in #1384
• feat: update tcp client connections by @kohlisid in #1429
• feat: enhance autoscaling peeking logic by @akashjkhamkar in #1432
• feat: pipeline health status for UI by @veds-g in #1460
• feat: terminate reduce vertex pods when pausing pipeline by @dpadhiar in #1481
• feat: configure standardResources via controller configmap by @inishchith in #1490
• feat: add support for SASL SCRAM 256 and 512 for Kafka by @Dnomyar in #1518
• feat: noop persistence store for reduce by @yhl25 in #1532
• feat: unaligned wal by @yhl25 in #1511
• feat: numaflow package by @veds-g in #1579
• feat: support headers for message by @yhl25 in #1578

Major Fixes 🐞

• fix: configmap const name by @whynowy in #1423
• fix: server-secrets-init container restart by @veds-g in #1433
• fix: Kafka source reads duplicated messages by @jy4096 in #1438
• fix: UI Filter by status for pipelines doesn't work as expected by @inishchith in #1444
• fix: GetDownstreamEdges is not cycle safe by @inishchith in #1447
• fix: memory leak inside session windower by @yhl25 in #1445
• fix: bug where dashed line is treated as permalink + spelling/formatting by @ayildirim21 in #1467
• fix(controller): incorrect cpu/mem resources calculation by @whynowy in #1477
• fix: add idle handler offset nil check by @jy4096 in #1489
• fix: unknown for ISB details in pipeline card by @veds-g in #1497
• fix: avoid panic when ctx is canceled by @yhl25 in #1515
• fix: idle manager refactor for multi partitions by @jy4096 in #1512
• fix: podSpec incorrectly configured in case template exist by @inishchith in #1516
• fix: incorrect json schema for tls config by @whynowy in #1520
• fix(controller): vertex template metadata nil check by @inishchith in #1527
• fix: break from retry loop when key is not found by @yhl25 in #1535
• fix: initialize inflightAcks channel to not nil channel by @afugazzotto in #1548
• fix(codegen): protobuf upgrade by @whynowy in #1558
• fix: error message that cause Buttons overflow in the UI by @a49ibrahim in #1591
• fix: flaky e2e tests by @vigith in #1590

New Contributors ❤️

• @akashjkhamkar made their first contribution in #1432
• @inishchith made their first contribution in #1444
• @ayildirim21 made their first contribution in #1467
• @syayi made their first contribution in #1510
• @Dnomyar made their first contribution in #1518
• @afugazzotto made their first contribution in #1548
• @a49ibrahim made their first contribution in #1591

Installation

kubectl create namespace numaflow-system

# Install numaflow controller and UI
kubectl apply -n numaflow-system -f https://github.com/numaproj/numaflow/releases/download/v1.2.0-rc1/install.yaml

# Install validating webhook (Optional)
kubectl apply -n numaflow-system -f https://github.com/numaproj/numaflow/releases/download/v1.2.0-rc1/validating-webhook-install.yaml

Full Changelog: v1.1.0...v1.2.0-rc1

v1.1.7

Major Changes

• Fixed a Kafka source issue to avoid potential dead lock (#1548)
• A temporary solution to support including message headers for Kafka source (#1569) - Be aware this might be changed later on.

Pull Requests

v1.1.6...v1.1.7

Installation

kubectl create namespace numaflow-system

# Install numaflow controller and UI
kubectl apply -n numaflow-system -f https://github.com/numaproj/numaflow/releases/download/v1.1.7/install.yaml

# Install validating webhook (Optional)
kubectl apply -n numaflow-system -f https://github.com/numaproj/numaflow/releases/download/v1.1.7/validating-webhook-install.yaml

v1.1.6

Major Changes

• Fixed controller config concurrent loding issue [#1499]
• Limit scale.max has to be > 0 [#1500]
• Fixed a controller bug about applying pod template [#1516]
• Fixed incorrect json tag for TLS configuration [#1520]
• Fixed a bug in reduce code logic after getting sigterm
• Fixed a nil check bug in the controller about vertex template metadata [#1527]
• Added SASL SCRAM 256 and 512 support for Kafka source [#1518]
• Upgrade sarama lib version to 1.43 [#1522]

Pull Requests

v1.1.5...v1.1.6

Installation

kubectl create namespace numaflow-system

# Install numaflow controller and UI
kubectl apply -n numaflow-system -f https://github.com/numaproj/numaflow/releases/download/v1.1.6/install.yaml

# Install validating webhook (Optional)
kubectl apply -n numaflow-system -f https://github.com/numaproj/numaflow/releases/download/v1.1.6/validating-webhook-install.yaml

v1.1.5

Major Changes

• Fixed cpu/mem resources calculation issue [#1477]

Pull Requests

v1.1.4...v1.1.5

Installation

kubectl create namespace numaflow-system

# Install numaflow controller and UI
kubectl apply -n numaflow-system -f https://github.com/numaproj/numaflow/releases/download/v1.1.5/install.yaml

# Install validating webhook (Optional)
kubectl apply -n numaflow-system -f https://github.com/numaproj/numaflow/releases/download/v1.1.5/validating-webhook-install.yaml

v1.1.4

Major Changes

• Fixed late data caused sliding window close issue [#1468]

Pull Requests

v1.1.3...v1.1.4

Installation

kubectl create namespace numaflow-system

# Install numaflow controller and UI
kubectl apply -n numaflow-system -f https://github.com/numaproj/numaflow/releases/download/v1.1.4/install.yaml

# Install validating webhook (Optional)
kubectl apply -n numaflow-system -f https://github.com/numaproj/numaflow/releases/download/v1.1.4/validating-webhook-install.yaml