Releases · AbsaOSS/atum-service

@salamonpavel

New Features 🎉

Create GET API endpoint returning the information about server's version @salamonpavel in #396
- Introduces GET /build-info endpoint returning application version
Add DB related metrics to Prometheus @lsulak in #420
- Adding database configuration to be able to fine-tune database connection settings and connection pool
- Monitoring enhancements: sending database metrics to Prometheus, to be able to see them in Grafana, as a new GET REST API endpoint /hikari-metrics.
Read-only user for data copy / analysis @lsulak in #422
- Adding read-only user for Atum database called atum_reader (SELECT only)

Infrastructure ⚙️

🔁 PR: Update release_draft.yml @ABLL526 in #418
- Changed python to 3.14 from 3.11
🔁 PR: chore(deps): bump coursier/cache-action from 8.0.0 to 8.0.1 @dependabot[bot] in #424
🔁 PR: chore(deps): bump softprops/action-gh-release from 2.5.0 to 2.6.0 @dependabot[bot] in #425
🔁 PR: chore(deps): bump softprops/action-gh-release from 2.6.0 to 2.6.1 @dependabot[bot] in #426
🔁 PR: chore(deps): bump coursier/cache-action from 8.0.1 to 8.1.0 @dependabot[bot] in #427

Full Changelog

v0.6.2...v0.7.0

@lsulak

Infrastructure ⚙️

Set up run of e2e tests before merge @lsulak in #307
- Adding and enabling integration tests into the CI
- Part of DEV deployment and also can be triggered manually
Update release_draft.yml @ABLL526 in #418
- Bump PYTHON from 3.11 to 3.14
Bump coursier/cache-action from 7.0.0 to 8.0.0 in #416
Bump AbsaOSS/generate-release-notes from 1.0.0 to 1.1.0 in #417

Full Changelog

v0.6.1...v0.6.2

@lsulak

Infrastructure ⚙️

Set up run of e2e tests before merge @lsulak in #307
- Adding and enabling integration tests into the CI
- Part of DEV deployment and also can be triggered manually

Full Changelog

v0.6.0...v0.6.1

@salamonpavel

New Features 🎉

Epic: Atum checkpoint with additional properties (#333)
- Reader: Include checkpoint properties @salamonpavel (#352)
  - Reader module now returns optional checkpoint properties.
- API: Accept & return checkpoint properties @salamonpavel (#350)
  - Introduces include-properties optional parameter for retrieving checkpoint properties.
- Database: Persist checkpoint properties @salamonpavel (#349)
  - Adds SQL storage & retrieval logic.
- Agent: Allow specifying checkpoint properties to Agent library @salamonpavel (#351)
  - Agent library can now create checkpoints with optional properties.
Epic: Adjust Agent to use v2 REST API (#271)
- Switch atumContext spawning to API v2 @salamonpavel (#347)
  - createPartitioning in HttpDispatcher now uses v2 endpoints.
Epic: Improve Atum relation to Spark (#379)
- Build against Spark 3.5 @salamonpavel (#380)
  - Spark dependency upgraded to 3.5.5
  - spark‑commons upgraded to 0.6.3
Rotated credentials auto‑re-fetching with fallback @salamonpavel (#413)
- Adds retry logic for rotated credentials with config fallback.

Security updates 👮

Increase security of DB functions in public schema @benedeki (#382)
- Hardens exposed database functions.

Bugfixes 🛠

Optional pagination parameters were mandatory @salamonpavel (#364)
Remove redundant data wrapper in additional‑data APIs @salamonpavel (#365)
- Fixes response structure for v2 GET/PATCH endpoints.ts
Aquasec: Replace action tags with SHA @lsulak (#404)
Fix API shot test + add CODEOWNER @lsulak (#389, #390)

Technical ⚙️

Run 1‑shot API tests against an ephemeral DB @salamonpavel (#392)
Add “Tech debt” issue template @benedeki (#391)
Update local env & env files @salamonpavel (#394, #398)
Remove obsolete GitHub Action @lsulak (#408)
Do not persist credentials in workflow checkout steps @hamc17 (#299)

Version Bumps 📦

Unify versions to eliminate shading issues @salamonpavel (#388)
- Multiple dependencies upgraded.
Upgrade fa-db dependency to 0.7.0 @salamonpavel (#402)
Bump AbsaOSS/version-tag-check from 0.3.0 to 1.0.0 (#410)
Bump actions/checkout from 6.0.1 to 6.0.2 (#411)
Bump actions/setup-python from 6.1.0 to 6.2.0 (#412)

Full Changelog

v0.5.1...v0.6.0

@benedeki

Security updates 👮

Increased security in database functions that exist in the public schema @benedeki in #382

Full Changelog

v0.5.0...v0.5.1

@benedeki

Breaking Changes ⚠️

*Dropped Support for Spark 2.4 developed by @benedeki, @lsulak in #196
- Users must now run Atum Service on Spark versions later than 2.4. This change improves compatibility with newer Spark features and enhances stability.

New Features 🎉

Created PATCH endpoint /api/v2/partitionings/{partitioning_id}/parents to patch the partitioning parents developed by @ABLL526, @lsulak, @benedeki in #273
- This enables users to update parent relationships of partitionings without full recreation, simplifying workflows and reducing overhead.
Created GET endpoint /api/v2/partitionings/{partitioning_id}/ancestors returns all ancestors, not just direct ones developed by @ABLL526, @lsulak, @benedeki in #305
- Users now have easy access to the full hierarchy of partitionings, supporting advanced lineage analysis and debugging.
Created endpoints for Kubernetes developed by @salamonpavel in #354
- These endpoints make Kubernetes integration smoother, enabling better orchestration, monitoring, and automation.

Bugfixes 🛠

Measures can be created with custom measureName developed by @salamonpavel, @benedeki in #342
- Missing support for private constructors for case classes in Scala 2.12 required measure classes to be refactored to standard classes.
- Restored protection when using custom measures

Technical ⚙️

Refactored Server's http package developed by @salamonpavel in #332
Refactored app structure with respect to API versioning developed by @salamonpavel in #335
Moved ErrorMonad type bound in Reader classes to evidence based developed by @salamonpavel in #339
Improved REST API test coverage to include ancestor and parent endpoints developed by @ABLL526 in #367
Made Atum Service releasable under the new Maven central repository developed by @ABLL526 in #370

Full Changelog

v0.4.1...v0.5.0

@ABLL526

New Features 🎉

2 new control functions added sum of truncated values developed by @ABLL526 in #314
- Created the aggregatedTruncTotal measure - only the whole part of the number is used in the control sum.
- Created the absAggregatedTruncTotal measure - only the absolute value of the whole part fo the number is used in control sum .

Bugfixes 🛠

Fixed has_more flag computation is SQL functions that support pagination. This affects the associated flag in REST API response data too developed by @salamonpavel in #337
Reader classes' signatures changed to receive MonadError instance as implicit evidence instead of type bound developed by @salamonpavel in #339

Silent Live 🤫

Endpoint GET /partitionings/{partId}/parents -> returns all ancestors of a partitioning` developed by @ABLL526 in #305

Known issues ⚠️

Measures can be created with custom measureName which saves the broken data causing eventual problems in #342
Dependency shading might be needed when using the Agent in Spark environment (especially when some Hadoop dependencies are in use as well, for example if you package the application that contains the Agent and use such JAR in Spark Submit command). Issue in #343. Here's the suggested project code snippet for Maven:

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-shade-plugin</artifactId>
                <version>${maven.shade.plugin.version}</version>
                <executions>
                    <execution>
                        <phase>package</phase>
                        <goals>
                            <goal>shade</goal>
                        </goals>
                    </execution>
                </executions>
                <configuration>
                    <relocations>
                        <relocation>
                            <pattern>okhttp3</pattern>
                            <shadedPattern>shaded.okhttp3</shadedPattern>
                        </relocation>
                        <relocation>
                            <pattern>okio</pattern>
                            <shadedPattern>shaded.okio</shadedPattern>
                        </relocation>
                        <relocation>
                            <pattern>sttp</pattern>
                            <shadedPattern>shaded.sttp</shadedPattern>
                        </relocation>
                        <relocation>
                            <pattern>cats</pattern>
                            <shadedPattern>shaded.cats</shadedPattern>
                        </relocation>
                        <relocation>
                            <pattern>shapeless</pattern>
                            <shadedPattern>shaded.shapeless</shadedPattern>
                        </relocation>
                        <relocation>
                            <pattern>kotlin</pattern>
                            <shadedPattern>shaded.kotlin</shadedPattern>
                        </relocation>
                    </relocations>
                    <filters>
                        <filter>
                            <artifact>*:*</artifact>
                            <excludes>
                                <exclude>META-INF/*.SF</exclude>
                                <exclude>META-INF/*.DSA</exclude>
                                <exclude>META-INF/*.RSA</exclude>
                            </excludes>
                        </filter>
                    </filters>
                </configuration>
            </plugin>

Full Changelog

v0.4.0...v0.4.1

@benedeki

New Features 🎉

Created a module named Reader, that reads information stored on the server in #243
- Added the ability to query REST endpoints from Reader module developed by @benedeki co-authored by @lsulak in #245
- Implement basics of ParitioningReader, a class to read Partitioning data developed by @benedeki in #246
- Implement basics of FlowReader, a class to read Flow data developed by @benedeki in #247
GetFlowCheckpoints endpoint refactored to reverse the order of data returned and to include partitioning data developed by @salamonpavel, @benedeki in #303
Change table runs.checkpoints to set column measured_by_atum_agent to have NOT NULL constraint with DEFAULT FALSE developed by @ABLL526 in #242
Server now includes AWS Sts dependency by @salamonpavel in #295
Several endpoints' Swagger documentation has been published (the endpoints had existed before, but they usage discouraged because of the chance of change)

Known Issues ⚠️

Dependency shading might be needed when using the Agent in Spark environment (especially when some Hadoop dependencies are in use as well, for example if you package the application that contains the Agent and use such JAR in Spark Submit command). Issue in #343. Here's the suggested project code snippet for Maven:

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-shade-plugin</artifactId>
                <version>${maven.shade.plugin.version}</version>
                <executions>
                    <execution>
                        <phase>package</phase>
                        <goals>
                            <goal>shade</goal>
                        </goals>
                    </execution>
                </executions>
                <configuration>
                    <relocations>
                        <relocation>
                            <pattern>okhttp3</pattern>
                            <shadedPattern>shaded.okhttp3</shadedPattern>
                        </relocation>
                        <relocation>
                            <pattern>okio</pattern>
                            <shadedPattern>shaded.okio</shadedPattern>
                        </relocation>
                        <relocation>
                            <pattern>sttp</pattern>
                            <shadedPattern>shaded.sttp</shadedPattern>
                        </relocation>
                        <relocation>
                            <pattern>cats</pattern>
                            <shadedPattern>shaded.cats</shadedPattern>
                        </relocation>
                        <relocation>
                            <pattern>shapeless</pattern>
                            <shadedPattern>shaded.shapeless</shadedPattern>
                        </relocation>
                        <relocation>
                            <pattern>kotlin</pattern>
                            <shadedPattern>shaded.kotlin</shadedPattern>
                        </relocation>
                    </relocations>
                    <filters>
                        <filter>
                            <artifact>*:*</artifact>
                            <excludes>
                                <exclude>META-INF/*.SF</exclude>
                                <exclude>META-INF/*.DSA</exclude>
                                <exclude>META-INF/*.RSA</exclude>
                            </excludes>
                        </filter>
                    </filters>
                </configuration>
            </plugin>

Full Changelog

v0.3.0...v0.4.0

@lsulak

Breaking Changes 💥

Additional data methods of AtumContext uses REST API v2 (incompatibility of Agent of 0.3.0+ with server 0.2.0) by @lsulak in #283
Full Flyway integration developed by @benedeki in #276

New Features 🎉

Atum server REST API v2 developed by @salamonpavel, @TebaleloS, @lsulak, @benedeki in #140
- GET /partitionings?partitioning=serializedPartitioning in #268
- GET /partitionings/{partId}/additional-data in #227
- PATCH /partitionings/{partId}/additional-data in #221
- there are many more, but they are silent live for now
Introduced response envelopes providing additional metadata (requestId) for REST API v2 endpoints by @salamonpavel in #197
Replaced Json4s and Jackson serialization libraries with Circe by @TebaleloS, @salamonpavel, @benedeki in #214
Introduced health API endpoint in a form StatusBoard projects expects by @salamonpavel in #282
Dockerfile and application configuration verified for deployment with ZIO and Http4s web server by @salamonpavel in #274
Dockerfile adjusted to ZIO framework and custom configuration now being passed during docker run, i.e. independent of the sbt build and docker build by @lsulak in #279

Silent Live 🤫

Introduced the Reader module to make reading of information stored in Atum server easy. by @benedeki in #248 (not publsiehd yet, only in code-base)
Atum server REST API v2 endpoints developed by @salamonpavel, @TebaleloS, @lsulak, @benedeki in #140
- There are numerous other endpoints implemented beside those mentioned above. We yet discourage from their usage though, as they are subject to change, particularly their payloads.

Known Issues ⚠️

Dependency shading might be needed when using the Agent in Spark environment (especially when some Hadoop dependencies are in use as well, for example if you package the application that contains the Agent and use such JAR in Spark Submit command). Here's the suggested project code snippet for Maven:

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-shade-plugin</artifactId>
                <version>${maven.shade.plugin.version}</version>
                <executions>
                    <execution>
                        <phase>package</phase>
                        <goals>
                            <goal>shade</goal>
                        </goals>
                    </execution>
                </executions>
                <configuration>
                    <relocations>
                        <relocation>
                            <pattern>okhttp3</pattern>
                            <shadedPattern>shaded.okhttp3</shadedPattern>
                        </relocation>
                        <relocation>
                            <pattern>okio</pattern>
                            <shadedPattern>shaded.okio</shadedPattern>
                        </relocation>
                        <relocation>
                            <pattern>sttp</pattern>
                            <shadedPattern>shaded.sttp</shadedPattern>
                        </relocation>
                        <relocation>
                            <pattern>cats</pattern>
                            <shadedPattern>shaded.cats</shadedPattern>
                        </relocation>
                        <relocation>
                            <pattern>shapeless</pattern>
                            <shadedPattern>shaded.shapeless</shadedPattern>
                        </relocation>
                        <relocation>
                            <pattern>kotlin</pattern>
                            <shadedPattern>shaded.kotlin</shadedPattern>
                        </relocation>
                    </relocations>
                    <filters>
                        <filter>
                            <artifact>*:*</artifact>
                            <excludes>
                                <exclude>META-INF/*.SF</exclude>
                                <exclude>META-INF/*.DSA</exclude>
                                <exclude>META-INF/*.RSA</exclude>
                            </excludes>
                        </filter>
                    </filters>
                </configuration>
            </plugin>

Full Changelog

v0.2.0...v0.3.0

@lsulak

This version fixes the configuration of the application in the Dockerized environment.

Bugfixes 🛠

Wrong format of application.properties file by @lsulak in #130
Renaming the application.properties file to be just a template and not the real one, the docker image MUST provide it by @lsulak in #131
Bugfix/remove hardcoded application.properties references from the code and adding ability to use config from SPRING_CONFIG_LOCATION env var by @lsulak in #132

Full Changelog: v0.1.0...v0.1.1

Releases: AbsaOSS/atum-service

v0.7.0

New Features 🎉

Infrastructure ⚙️

Full Changelog

Contributors

Uh oh!

v0.6.2

Infrastructure ⚙️

Full Changelog

Contributors

Uh oh!

v0.6.1

Infrastructure ⚙️

Full Changelog

Contributors

Uh oh!

v0.6.0

New Features 🎉

Security updates 👮

Bugfixes 🛠

Technical ⚙️

Version Bumps 📦

Full Changelog

Contributors

Uh oh!

v0.5.1

Security updates 👮

Full Changelog

Contributors

Uh oh!

v0.5.0

Breaking Changes ⚠️

New Features 🎉

Bugfixes 🛠

Technical ⚙️

Full Changelog

Contributors

Uh oh!

v0.4.1

New Features 🎉

Bugfixes 🛠

Silent Live 🤫

Known issues ⚠️

Full Changelog

Contributors

Uh oh!

v0.4.0

New Features 🎉

Known Issues ⚠️

Full Changelog

Contributors

Uh oh!

v0.3.0

Breaking Changes 💥

New Features 🎉

Silent Live 🤫

Known Issues ⚠️

Full Changelog

Contributors

Uh oh!

v0.1.1

Bugfixes 🛠

Contributors

Uh oh!