#679 Add Iceberg and Delta table names to the catalog table name column in notifications #680

yruslan · 2025-12-16T07:56:34Z

Closes #679

Summary by CodeRabbit

New Features
- Enhanced tracking of Hive table updates reported after save operations for clearer visibility.
Tests
- Expanded test coverage for table name formatting across various catalog/database scenarios, including unescaped short names.
Style
- Updated success logging appearance (notification icon) for send operations.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

coderabbitai · 2025-12-16T07:56:45Z

Walkthrough

Added a method to produce unescaped short table identifiers, captured those identifiers for Iceberg/Delta outputs in TransformationJob save results, extended SaveResult to carry hive table updates, and changed a success log emoji in notification sending.

Changes

Cohort / File(s)	Summary
Table name retrieval `pramen/api/src/main/scala/za/co/absa/pramen/api/CatalogTable.scala`, `pramen/api/src/test/scala/za/co/absa/pramen/api/CatalogTableSuite.scala`	Added `getShortUnescapedTableName: String` to produce unescaped "database.table" (or "table") forms; added tests covering catalog/database combinations and existing `getFullTableName`.
Transformation result reporting `pramen/core/src/main/scala/za/co/absa/pramen/core/pipeline/TransformationJob.scala`	Extract output table identifiers for Iceberg (short unescaped name) and Delta (query string) into `outputCatalogTable`; pass these as `hiveTablesUpdates` when constructing `SaveResult`.
Notification logging `pramen/core/src/main/scala/za/co/absa/pramen/core/notify/Sendable.scala`	Replaced success log emoji constant from VOLTAGE to ROCKET in `doSend` (log message content change only).

Sequence Diagram(s)

sequenceDiagram
    participant Job as TransformationJob
    participant Catalog as CatalogTable
    participant Save as SaveResult
    participant Notify as Sendable

    Job->>Catalog: call getShortUnescapedTableName (for Iceberg)
    Job->>Job: build outputCatalogTable (Iceberg/Delta handling)
    Job->>Save: construct SaveResult(stats, warnings, hiveTablesUpdates)
    Job->>Notify: trigger notification with SaveResult
    Notify->>Notify: log success (ROCKET)

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~30 minutes

Review SaveResult constructor changes and all call sites for compatibility.
Verify TransformationJob logic for detecting Iceberg vs Delta outputs and handling multiple outputs or missing metadata.
Confirm CatalogTable tests cover edge cases and escaping behavior.

Poem

🐰 I nibble names both short and plain,
From database dot table to single name,
Iceberg, Delta, I gently track,
Results hop back upon the stack,
I cheer the change with a little ROCKET flame! 🚀

Pre-merge checks and finishing touches

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The PR title accurately describes the main change: adding Iceberg and Delta table names to notifications. It is concise, specific, and directly reflects the primary objective from issue #679.
Linked Issues check	✅ Passed	The PR implements the core requirement from #679: including output table names in transformer notifications for Iceberg and Delta tables. Changes include adding getShortUnescapedTableName method and propagating hiveTablesUpdates through SaveResult.
Out of Scope Changes check	✅ Passed	All changes are directly related to the stated objective. The emoji change in Sendable.scala is a minor notification enhancement aligned with the feature. No unrelated modifications detected.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Post copyable unit tests in a comment
Commit unit tests in branch feature/679-add-iceberg-and-delta-table-names-to-transformers

📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between af6fcc3 and b5bf57c.

📒 Files selected for processing (4)

pramen/api/src/main/scala/za/co/absa/pramen/api/CatalogTable.scala (1 hunks)
pramen/api/src/test/scala/za/co/absa/pramen/api/CatalogTableSuite.scala (1 hunks)
pramen/core/src/main/scala/za/co/absa/pramen/core/notify/Sendable.scala (1 hunks)
pramen/core/src/main/scala/za/co/absa/pramen/core/pipeline/TransformationJob.scala (2 hunks)

🚧 Files skipped from review as they are similar to previous changes (2)

pramen/api/src/test/scala/za/co/absa/pramen/api/CatalogTableSuite.scala
pramen/api/src/main/scala/za/co/absa/pramen/api/CatalogTable.scala

🧰 Additional context used

🧠 Learnings (1)

📚 Learning: 2025-06-18T08:27:21.504Z

Learnt from: yruslan
Repo: AbsaOSS/pramen PR: 611
File: pramen/core/src/main/scala/za/co/absa/pramen/core/metastore/model/MetastoreDependencyFactory.scala:29-29
Timestamp: 2025-06-18T08:27:21.504Z
Learning: In pramen/core/src/main/scala/za/co/absa/pramen/core/metastore/model/MetastoreDependencyFactory.scala, the constant variable name was changed from DATE_UNTIL_EXPR_KEY to DATE_TO_EXPR_KEY, but both constants hold the same string value "date.to". This is a variable name refactoring, not a configuration key change, so it doesn't affect backward compatibility.

Applied to files:

pramen/core/src/main/scala/za/co/absa/pramen/core/pipeline/TransformationJob.scala

🧬 Code graph analysis (1)

pramen/core/src/main/scala/za/co/absa/pramen/core/pipeline/TransformationJob.scala (4)

pramen/api/src/main/scala/za/co/absa/pramen/api/DataFormat.scala (1)

DataFormat (30-105)

pramen/api/src/main/scala/za/co/absa/pramen/api/Query.scala (1)

Query (28-56)

pramen/api/src/main/scala/za/co/absa/pramen/api/CatalogTable.scala (1)

getShortUnescapedTableName (33-36)

pramen/core/src/main/scala/za/co/absa/pramen/core/pipeline/SaveResult.scala (1)

SaveResult (21-26)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (8)

GitHub Check: Test Spark 3.3.4 on Scala 2.13.16
GitHub Check: Test Spark 2.4.8 on Scala 2.11.12
GitHub Check: Test Spark 3.5.5 on Scala 2.13.16
GitHub Check: Test Spark 3.3.4 on Scala 2.12.20
GitHub Check: Test Spark 3.4.4 on Scala 2.13.16
GitHub Check: Test Spark 3.5.5 on Scala 2.12.20
GitHub Check: Test Spark 3.4.4 on Scala 2.12.20
GitHub Check: Test Coverage on Scala 2.12.18

🔇 Additional comments (2)

pramen/core/src/main/scala/za/co/absa/pramen/core/pipeline/TransformationJob.scala (1)

22-22: LGTM!

The added imports for DataFormat and Query are necessary for the new catalog table name extraction logic.

pramen/core/src/main/scala/za/co/absa/pramen/core/notify/Sendable.scala (1)

89-89: Emoji change from VOLTAGE to ROCKET is cosmetic but valid.

This change swaps the emoji in the success log message from the lightning bolt (⚡ VOLTAGE) to the rocket (🚀 ROCKET), which is a minor cosmetic improvement. Both emojis are properly defined in the Emoji constants, and ROCKET is semantically more appropriate for signaling successful completion.

The core PR objectives—adding Iceberg and Delta table names to notifications—are implemented in TransformationJob.scala (lines 93–102), which extracts short unescaped table identifiers and passes them via the hiveTablesUpdates parameter in SaveResult. This file handles the email notification mechanics and is appropriately updated to reflect the improved success indicator.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

coderabbitai

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between d3cf452 and af6fcc3.

📒 Files selected for processing (4)

pramen/api/src/main/scala/za/co/absa/pramen/api/CatalogTable.scala (1 hunks)
pramen/api/src/test/scala/za/co/absa/pramen/api/CatalogTableSuite.scala (1 hunks)
pramen/core/src/main/scala/za/co/absa/pramen/core/notify/Sendable.scala (1 hunks)
pramen/core/src/main/scala/za/co/absa/pramen/core/pipeline/TransformationJob.scala (2 hunks)

🧰 Additional context used

🧬 Code graph analysis (2)

pramen/api/src/test/scala/za/co/absa/pramen/api/CatalogTableSuite.scala (1)

pramen/api/src/main/scala/za/co/absa/pramen/api/CatalogTable.scala (4)

CatalogTable (19-37)

CatalogTable (39-60)

getFullTableName (27-31)

getShortUnescapedTableName (33-36)

pramen/core/src/main/scala/za/co/absa/pramen/core/pipeline/TransformationJob.scala (4)

pramen/api/src/main/scala/za/co/absa/pramen/api/DataFormat.scala (1)

DataFormat (30-105)

pramen/api/src/main/scala/za/co/absa/pramen/api/Query.scala (1)

Query (28-56)

pramen/api/src/main/scala/za/co/absa/pramen/api/CatalogTable.scala (1)

getShortUnescapedTableName (33-36)

pramen/core/src/main/scala/za/co/absa/pramen/core/pipeline/SaveResult.scala (1)

SaveResult (21-26)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (8)

GitHub Check: Test Coverage on Scala 2.12.18
GitHub Check: Test Spark 3.4.4 on Scala 2.13.16
GitHub Check: Test Spark 3.5.5 on Scala 2.12.20
GitHub Check: Test Spark 3.5.5 on Scala 2.13.16
GitHub Check: Test Spark 2.4.8 on Scala 2.11.12
GitHub Check: Test Spark 3.3.4 on Scala 2.12.20
GitHub Check: Test Spark 3.3.4 on Scala 2.13.16
GitHub Check: Test Spark 3.4.4 on Scala 2.12.20

🔇 Additional comments (4)

pramen/core/src/main/scala/za/co/absa/pramen/core/notify/Sendable.scala (1)

89-89: LGTM!

The emoji change from VOLTAGE to ROCKET better represents the action of sending an email notification.

pramen/api/src/main/scala/za/co/absa/pramen/api/CatalogTable.scala (1)

33-36: LGTM!

The new getShortUnescapedTableName method correctly returns the short form (database.table or just table) without escape characters or catalog prefix, fulfilling the PR objective to provide table names for notifications.

pramen/api/src/test/scala/za/co/absa/pramen/api/CatalogTableSuite.scala (1)

74-124: LGTM!

Comprehensive test coverage for both getFullTableName and getShortUnescapedTableName methods, covering all combinations of catalog and database presence. The assertions correctly validate the expected behavior for escaped and unescaped table name formats.

pramen/core/src/main/scala/za/co/absa/pramen/core/pipeline/TransformationJob.scala (1)

22-22: LGTM!

The additional imports for DataFormat and Query are necessary for the new table name extraction logic.

pramen/core/src/main/scala/za/co/absa/pramen/core/pipeline/TransformationJob.scala

…mn in notifications.

github-actions · 2025-12-16T08:09:18Z

Unit Test Coverage

Overall Project	83.21% `-0.02%`	🍏
Files changed	66.67%	❌

Module	Coverage
pramen:core Jacoco Report	85.12% `-0.03%`	❌

Files

Module	File	Coverage
pramen:core Jacoco Report	TransformationJob.scala	90.8% `-6.23%`	❌
	Sendable.scala	14.02%	🍏

coderabbitai bot reviewed Dec 16, 2025

View reviewed changes

pramen/core/src/main/scala/za/co/absa/pramen/core/pipeline/TransformationJob.scala Outdated Show resolved Hide resolved

#679 Add Iceberg and Delta table names to the catalog table name colu…

b5bf57c

…mn in notifications.

yruslan force-pushed the feature/679-add-iceberg-and-delta-table-names-to-transformers branch from af6fcc3 to b5bf57c Compare December 16, 2025 08:02

yruslan merged commit 81aa3c2 into main Dec 16, 2025
9 checks passed

yruslan deleted the feature/679-add-iceberg-and-delta-table-names-to-transformers branch December 16, 2025 08:20

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

#679 Add Iceberg and Delta table names to the catalog table name column in notifications #680

#679 Add Iceberg and Delta table names to the catalog table name column in notifications #680

Uh oh!

yruslan commented Dec 16, 2025 •

edited by coderabbitai bot

Loading

Uh oh!

coderabbitai bot commented Dec 16, 2025 •

edited

Loading

Uh oh!

coderabbitai bot left a comment

Uh oh!

Uh oh!

github-actions bot commented Dec 16, 2025 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

#679 Add Iceberg and Delta table names to the catalog table name column in notifications #680

#679 Add Iceberg and Delta table names to the catalog table name column in notifications #680

Uh oh!

Conversation

yruslan commented Dec 16, 2025 • edited by coderabbitai bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Uh oh!

coderabbitai bot commented Dec 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Sequence Diagram(s)

Estimated code review effort

Poem

Pre-merge checks and finishing touches

Uh oh!

coderabbitai bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions bot commented Dec 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Unit Test Coverage

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

yruslan commented Dec 16, 2025 •

edited by coderabbitai bot

Loading

coderabbitai bot commented Dec 16, 2025 •

edited

Loading

github-actions bot commented Dec 16, 2025 •

edited

Loading