Sling v1.5.1 (2025-12-07T10:54:11Z)

New Features

Directory-Based Runs

• Run all replications/pipelines in a directory: You can now pass a directory path to sling run, and it will automatically discover
  and execute all valid replication and pipeline YAML files within it
• Files are executed in order with clear visual separators showing progress
• Summary shows successes/failures after completion

API Specification Enhancements

• Modifier syntax for response rules/processors: New +rules, rules+, +processors, processors+ syntax allows prepending or appending
  rules and processors to API spec endpoints
• Setup/Teardown modifiers: Similar +setup, setup+, +teardown, teardown+ syntax for endpoint setup and teardown configurations
• JSON object/array detection in template variables: Template variables in API specs now properly detect and handle JSON objects and
  arrays

New Transform Functions

• regex_replace(string, pattern, replacement): Replace regex pattern matches in strings, supports backreferences (e.g., $1, $2)

Database Improvements

• ClickHouse keep-alive mechanism: Prevents idle connection timeouts during long bulk imports by periodically sending heartbeat
  queries
• DuckDB updated to v1.4.2: Latest DuckDB version with improvements and fixes
• DuckDB binary pre-download for threaded replications: Avoids race conditions when multiple threads need DuckDB simultaneously

Snowflake Enhancements

• MFA token caching: Improved support for Snowflake connections with MFA

OpenTelemetry Integration

• OTEL logging support: New OpenTelemetry dependencies for enhanced logging and observability
• Configure via SLING_OTEL_ENDPOINT environment variable

Pipeline Improvements

• Skip steps on previous error: Pipeline steps are now automatically skipped if a previous step errored
• Parameter rendering in hooks: Dynamic value handling in hook parameters

Bug Fixes

• Fixed table creation check: CREATE TABLE statements now properly check for existence before creation
• Removed deprecated --config CLI flag: Cleanup of legacy options
• Improved progress message formatting: Cleaner task execution output

Sling v1.5.0 (2025-11-30T16:59:21Z)

Key Features

API Specification Enhancements

• Endpoint-specific authentication: Refactored API authentication to allow individual endpoints to define and manage their own authentication, including endpoint-level auth expiry and re-authentication
• Dynamic endpoint rendering: Added Dynamic field to Endpoint for generated state indication
• New rule types: Added RuleTypeSkip to skip records and RuleTypeBreak to stop processing requests for current iteration
• Enhanced state management: Implemented contextMap for improved state management across endpoints
• Renamed AggregationTypeFlatten to AggregationTypeCollect for clearer semantics

Database & Connection Improvements

• DuckDB 1.4.1: Updated to latest DuckDB version
• Native pgx connection for Cloud SQL: Implemented native pgx connection for Cloud SQL COPY operations
• ClickHouse boolean handling: Fixed boolean-to-string conversion in CastColumnForSelect function
• DuckLake SQLite catalog: Automatically create parent directory for SQLite catalog if needed
• GCP credentials: Added base64 decoding for GCP credentials in BigQuery, MySQL, PostgreSQL, and Redshift connections

Replication & Pipeline Enhancements

• Wildcard hooks: Added support for wildcard hooks in replication configuration (e.g., pre.*, post.*)
• Dynamic SQL rendering: Enhanced evaluator integration for dynamic SQL rendering with JMESPath compatibility
• Incremental mode improvements: Added hasRange method and improved incremental mode warnings
• Periodic state updates: Implemented periodic state updates during replication execution
• Run state tracking: Added optional fields for incremental value and range in RunState

Environment & Configuration

• EnvFile refactoring: Refactored Variables to Env in EnvFile structure for improved clarity
• Legacy support: Improved handling for empty Env and Variables maps with backward compatibility
• LocalConnections: Initialize and populate LocalConnections map with connection data

Error Handling & Debugging

• Enhanced error messages: Improved error messages for CSV input errors, URL parsing failures, and transaction commit issues
• S3 region guidance: Added guidance for S3 region specification in DuckLake error messages
• Goroutine stack dumps: Dump goroutine stack on critical events (Ctrl+C, timeout) for debugging

SSH Enhancements

• NewSession method: Added NewSession method to create SSH sessions with proper error handling
• Comprehensive SSH tests: Added comprehensive SSH command tests with various scenarios

Concurrency Improvements

• Concurrent maps: Replaced standard maps with concurrent-map for LocalConnections and transforms to prevent race conditions
• Synchronized stopIters: Added synchronization during endpoint setup

Bug Fixes

• Fixed debug logging for HTTP stream initialization in DuckDB
• Fixed overwriting existing state in renderEndpointTemplate
• Fixed nil Pipeline handling in Context method of PipelineStepExecution
• Fixed replication step configuration and environment variable handling
• Fixed timeout duration to use seconds instead of milliseconds
• Removed redundant check for disabled streams during replication preparation
• Fixed missing filepath import in database_ducklake.go
• Removed unused duckURI variable in filesystem operations

Test Infrastructure

• Added comprehensive routine tests for logging, validation, file operations, and HTTP fetch
• Added test orchestration script for CLI and database testing
• Added dependency management for test cases with after field
• Added group classification for test organization
• Added MSSQL to Postgres decimal precision and cast_as tests
• Added nested JSON to SQLite test scripts
• Added test for wildcard streams with disabled table exclusion
• Added ClickHouse boolean-to-string conversion test
• Added test for {fields} placeholder with select parameter
• Added HTTP POST hooks test suite for JSON payloads

Sling v1.4.26 (2025-11-06T13:15:03Z)

See https://github.com/slingdata-io/sling-cli/ for more details.

Sling v1.4.25 (2025-11-06T11:06:26Z)

See https://github.com/slingdata-io/sling-cli/ for more details.

Sling v1.4.24 (2025-10-17T16:09:36Z)

See https://github.com/slingdata-io/sling-cli/ for more details.

Sling v1.4.23 (2025-09-26T18:42:03Z)

• MongoDB collection listing enhancements
• StarRocks validation fixes
• Cross-platform build support via GoReleaser
• Comprehensive test suite additions
• Various bugs and fixes

Sling v1.4.22 (2025-09-15T14:17:53Z)

• Fix Arrow Decimal truncation issue.

Sling v1.4.21 (2025-09-15T11:11:26Z)

New Features & Enhancements:

1. MongoDB ObjectID Support - Added proper ObjectID filtering capabilities for MongoDB connections with automatic hex string conversion
2. ClickHouse Improvements - Enhanced merge operations with retry logic for "Too many unfinished mutations" errors and lightweight delete support for v23+
3. Prometheus Enhancements - Better transform support and column type handling for Prometheus data sources
4. API Spec Fetching - New FetchSpec function for loading API specifications
5. Databricks Volume Management - Improved volume path handling and creation logic

Bug Fixes:

1. SQL Server Money Type - Fixed precision issues with MONEY and SMALLMONEY data types when replicating to StarRocks
2. Decimal Precision Handling - Improved decimal column typing with better precision and scale calculations
3. Snowflake Stage Management - Better internal stage creation and validation
4. Redshift Credentials - Enhanced AWS credential handling for COPY operations including support for IAM roles
5. PostgreSQL Timeout Support - Added statement_timeout parameter support

Sling v1.4.20 (2025-09-03T12:34:56Z)

Database and Connection Improvements

• New Database Support: Added Arrow Flight SQL (FlightSQL) connection type
• Connection URL Handling: Improved SFTP/FTP connection URL parsing with proper path handling
• Decimal Precision Handling: Enhanced decimal number processing with better precision and scale management

File Processing Enhancements

• XML File Support: Added comprehensive XML file parsing capabilities with custom decoder
• File Splitting: Improved file splitting functionality with better chunking and manifest generation
• Parquet Processing: Enhanced Parquet file handling with DuckDB integration

Data Type and Processing Improvements

• Decimal Type Refactoring: Switched from float64 to decimal.Decimal for better precision
• String Parsing Optimization: Refactored string parsing with better type detection and caching
• Number Precision: Enhanced handling of high-precision numbers and scientific notation

Database Operation Updates

• Merge Operations: Renamed "Upsert" operations to "Merge" across multiple database connectors
• PostgreSQL Merge Strategy: Implemented sophisticated merge strategy using temporary tables
• Transaction Handling: Improved transaction management for bulk operations

API Specification Enhancements

• Sling Platform API Updates: Enhanced API specifications with improved pagination logic and state management
• Stripe API Improvements: Fixed pagination issues with better handling of starting_after and ending_before parameters
• Dynamic Base URL Configuration: Added support for dynamic base URL configuration using secrets

New LLM Documentation Guides

• Pipeline Guide for LLMs: Comprehensive guide covering pipeline structure, task types, variables, control flow, and best practices
• Python Integration Guide: guide for Sling Python bindings with usage patterns and examples

Sling v1.4.19 (2025-08-26T00:56:46Z)

Transforms

• Improve Staged Transforms reliability and performance

MCP Integration

• Comprehensive documentation for AI/LLM usage of Sling connections
• Unified tool interfaces with action-based architecture
• Enhanced error handling and user guidance

Database Connectivity

• Native ClickHouse driver support with advanced configuration
• Improved DuckDB Arrow streaming capabilities
• Enhanced TIME datatype handling across multiple databases
• Better TLS and security configuration options
• BigQuery BIGNUMERIC Support