v1.13.11

What's Changed

• #718 Add support for "prefer.coalesce" when repartitioning metastore tables in #728
• #181 Add support for DynamoDB as the storage of bookeeping, schemas, locks, journal and offset information in #730
• #733 add numbers to duplicated sanitized cols in #734

Full Changelog: v1.13.10...v1.13.11

v1.13.9

What's Changed

• #722 Autoclose JDBC connections on job timeout in #723
• #724 Switch the behavior of Hive repair table on reruns. Do it only if explicitly asked in #725

Full Changelog: v1.13.8...v1.13.9

v1.13.8

What's Changed

• #720 Improve Hive table existence check to make it more efficient from the perspective of HMS pressure in #721

Full Changelog: v1.13.7...v1.13.8

v1.13.7

What's Changed

• #714 Release all locks when Pramen job exits abnormally in #715
• #716 Fix schema registry configuration for Kafka source and sink in #717
• Allow specifying starting offsets via the Kafka source options in #719

Full Changelog: v1.13.6...v1.13.7

v1.13.6

What's Changed

• #710 Make the schema change renderer nested schema friendly in #713

Full Changelog: v1.13.5...v1.13.6

v1.13.5

What's Changed

• #692 Refactor the usage of bookeeping database with Slick framework so it can be DB engine agnostic in #709
• #711 Add pipeline run attempt configuration and notification support in #712

Full Changelog: v1.13.4...v1.13.5

v1.13.4

What's Changed

• Move catalog utils in a separate utility object for reuse in #708
• Add support for RedShift driver in 7445150
• Make backoff times for reconnecting to bookeeping databases configurable in 7445150
• Tirm column names of leading and trailing spaces as part of column name sanitization in 1bbe96f

Full Changelog: v1.13.3...v1.13.4

v1.13.3

What's Changed

• #703 Fix the default value for backfill.days to be compatible with the previous Pramen scheduling behavior in #704
• #706 Add the ability to delete bookkeeping tables from the database via Bookeeper trait in #707

Full Changelog: v1.13.2...v1.13.3

v1.13.2

What's Changed

• #699 Add retries for Bookkeeping database, and make sure only 1 connection is active in #700
• #698 Fix the backfill logic for late jobs in #701

Full Changelog: v1.13.1...v1.13.2

v1.13.1

What's Changed

• #685 Fix when transformers and sinks are marked as not allowed to run in parallel have the behavior of transformations with self-dependencies in #688
• #686 Fix self-dependencies by making sure optional self-dependencies do not cause skips for informations dates if previous information dates had failed such dependencies in #689
• #690 Fix transient persistent metatable cache cleanup in #694
• #687 Add support for backfill days in #691
• #690 Add support for SQLite as the option for storing bookeeping data in #696
• #695 Add support for marking operations as 'critical' failure of which causes the pipeline to fail in #697
• Do not include stack traces of a job if a dependent lazy job failed because that stack trace is already included in #697
• Use final version tags for standard GH actions, and SHA1 commit references to community GH actions in #684

Full Changelog: v1.13.0...v1.13.1