Tags · saviobatista/sbs-logger

v0.2.4

deps(deps): bump github.com/redis/go-redis/v9 from 9.11.0 to 9.12.1 (#29

)

Bumps [github.com/redis/go-redis/v9](https://github.com/redis/go-redis) from 9.11.0 to 9.12.1.
- [Release notes](https://github.com/redis/go-redis/releases)
- [Changelog](https://github.com/redis/go-redis/blob/master/RELEASE-NOTES.md)
- [Commits](redis/go-redis@v9.11.0...v9.12.1)

---
updated-dependencies:
- dependency-name: github.com/redis/go-redis/v9
  dependency-version: 9.12.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Aug 19, 2025
c9c26ed
zip
tar.gz
Notes
Downloads

v0.2.3

🔧 Fix: Enhanced Message Processing and Parser Improvements (#23)

* refactor: Enhance message processing in ingestor and parser

- Introduced a message buffer in the ingestor to handle incomplete messages and ensure proper message publishing.
- Updated the parser to support new message types (STA, AIR, ID) and improved error messages for invalid formats.
- Enhanced error handling in message parsing to include raw message details for better debugging.

* test: Add PostgreSQL readiness check for integration tests

- Introduced a new utility function, WaitForPostgresReady, to ensure PostgreSQL containers are fully ready for DDL operations before running tests.
- Updated integration tests in main_integration_test.go to utilize the new readiness check, improving reliability of database interactions during testing.

Jul 26, 2025
52300c8
zip
tar.gz
Notes
Downloads

v0.2.2

feat: Comprehensive Integration Testing and Enhanced Test Coverage (#22)

* chore: Update dependencies in go.mod and go.sum for improved functionality

- Added multiple indirect dependencies to go.mod, including dario.cat/mergo, github.com/Azure/go-ansiterm, and various testcontainers modules for enhanced testing capabilities.
- Updated go.sum to reflect the addition of new dependencies and ensure consistency in module versions across the project.

* test: Enhance coverage for StateTracker with new scenarios and integration tests

- Added new test cases for StateTracker, including scenarios for starting with existing flights, invalid flight validation, and flight ending scenarios.
- Introduced integration tests for client creation, error paths in main function, and setupStateTracker with real clients.
- Improved overall test coverage in cmd/tracker, focusing on message processing, state merging, and flight lifecycle management.
- Removed outdated tests and improved error handling in test cases for better reliability.

* test: Add integration tests for PostgreSQL and Redis setup in tracker

- Introduced a new integration test file for the tracker service, focusing on the setup and validation of PostgreSQL and Redis containers using testcontainers.
- Implemented tests for database connection and basic operations, including table creation, data insertion, and querying.
- Enhanced test coverage for integration scenarios, ensuring reliable setup and teardown of test environments.

* Update Makefile to enhance test commands and organization

- Modified the 'test' target to clarify it runs all tests (unit + integration).
- Added new targets for running unit tests and integration tests separately, both with and without race detection.
- Improved documentation for each test target to enhance clarity and usability.

* chore: Update dependencies in go.mod and go.sum for enhanced testing and functionality

- Added new direct dependencies for testcontainers modules (NATS, PostgreSQL, Redis) to support integration testing.
- Updated indirect dependencies including grpc-gateway and opentelemetry packages for improved functionality and compatibility.
- Ensured go.sum reflects the latest dependency versions for consistency across the project.

* test: Add integration tests for NATS interactions in ingestor

- Introduced a new integration test file for the ingestor service, focusing on testing the interaction with a real NATS server using testcontainers.
- Implemented multiple tests to validate the ingestion pipeline, connection retries, and message publishing functionality.
- Enhanced test coverage for integration scenarios, ensuring reliable setup and teardown of test environments with NATS.
- Updated existing unit tests for clarity and improved naming conventions.

* test: Add integration tests for logger functionality with NATS

- Introduced a new integration test file for the logger, focusing on testing interactions with a real NATS server using testcontainers.
- Implemented multiple tests to validate the logger's message writing, rotation, and compression functionalities.
- Enhanced test coverage for integration scenarios, ensuring reliable setup and teardown of test environments with NATS.
- Updated existing tests for clarity and improved naming conventions.

* refactor: Remove NATS configuration and related volume from docker-compose

- Deleted the NATS server configuration file and removed the associated volume from docker-compose.yml.
- Streamlined the NATS service setup by eliminating unnecessary configuration, simplifying the deployment process.

* refactor: Update container setup and termination handling in integration tests

- Refactored the setup of NATS, PostgreSQL, and Redis containers in integration tests to use a simplified method for starting containers.
- Enhanced the termination logic for containers to log errors if termination fails, improving test reliability and debugging capabilities.
- Ensured consistent handling of container lifecycle across multiple integration test files.

* test: Add comprehensive integration tests for NATS client and capture functionality

- Introduced new integration tests for the NATS client, validating connection, publish/subscribe workflows, and error handling scenarios.
- Added tests for capturing messages from TCP sources, including real connections, multiple sources, and connection retries.
- Enhanced coverage for message serialization, concurrent publishers, and handling of invalid sources, ensuring robust testing of the NATS integration.
- Improved overall test reliability and clarity in the integration test suite.

* test: Enhance integration tests for logger with NATS and environment parsing

- Refactored integration tests for the logger to improve setup and error handling with NATS.
- Added comprehensive tests for environment variable parsing, ensuring correct handling of default and custom values.
- Implemented tests for message publishing and subscription, validating logger functionality with real NATS connections.
- Improved overall test reliability and clarity, ensuring robust coverage of logger operations and error scenarios.

* chore: Update .gitignore to exclude output files

- Added '*.out' to .gitignore to prevent output files from being tracked in the repository, ensuring a cleaner project structure.

* chore: Update docker-compose configuration for NATS service

- Added command and volume configuration for the NATS service in docker-compose.yml to enable JetStream support and persistent data storage.
- Introduced a new volume for NATS data to ensure data persistence across container restarts.

Jul 26, 2025
2fc93c1
zip
tar.gz
Notes
Downloads

0.2.1

### Summary

Updates the GitHub Actions workflow to provide more comprehensive information when listing workflow runs, improving debugging and monitoring capabilities.

### Changes
- **Modified**: `.github/workflows/ci.yml`
  - Updated `gh run list` command to include additional fields: `name`, `number`, and `workflowName`
  - Removed the `artifacts` field which was less useful for debugging workflow execution
  - The command now provides better visibility into workflow run details for troubleshooting

### Details
The change improves the "List available artifacts" step in the CI workflow by expanding the JSON output to include:
- `name`: The name of the workflow run
- `number`: The run number for easier identification
- `workflowName`: The name of the workflow for better context

This enhancement makes it easier to debug CI issues and track workflow execution history.

### Testing
- [ ] Verify the updated command outputs the expected additional fields
- [ ] Confirm the CI pipeline runs successfully with the changes

### Type of Change
- [x] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Jul 25, 2025
a29f080
zip
tar.gz

0.2.0

## 🚀 Overview

This PR eliminates the dedicated `migrate` service by embedding database migrations directly into the `tracker` service, resulting in a more efficient and production-ready deployment architecture.

## 🎯 Motivation

The previous approach of having a separate container just to execute SQL migrations had several issues:
- **Resource waste**: Entire Docker container built for simple SQL execution
- **Build overhead**: Extra Dockerfile and CI/CD steps
- **Manual process**: Required `docker-compose run --rm migrate` 
- **Production complexity**: Additional failure point and manual intervention
- **Maintenance burden**: Extra service to maintain and update

## ✅ Changes Made

### **Core Architecture**
- ✅ **Embedded migrations in tracker**: Migrations now run automatically on tracker startup
- ❌ **Removed `cmd/migrate/` directory**: Deleted main.go and main_test.go (1,400+ lines)
- ❌ **Removed `Dockerfile.migrate`**: No longer needed
- ❌ **Updated `docker-compose.yml`**: Removed migrate service entirely

### **CI/CD Pipeline Optimization**
- ✅ **Updated `.github/workflows/ci.yml`**: Removed migrate from build matrix, artifact handling, and release notes
- ✅ **Updated `.github/workflows/dockerhub-publish.yml`**: Removed migrate service from Docker Hub publishing
- ✅ **Updated `Makefile`**: Removed migrate commands and updated SERVICES variable
- ✅ **Updated `scripts/build-test.sh`**: Removed migrate from test services

### **Documentation & Configuration**
- ✅ **Updated `README.md`**: 
  - Removed migrate component documentation
  - Updated architecture diagrams
  - Removed migrate build instructions
  - Updated Docker image references
- ✅ **Updated tracker service**: Added automatic migration execution with proper error handling

## 🏗️ Technical Implementation

### **Migration Flow**
```go
func runMigrations(dbConnStr string) error {
    log.Println("Running database migrations...")
    
    migrationDB, err := sql.Open("postgres", dbConnStr)
    if err != nil {
        return fmt.Errorf("failed to connect to database for migrations: %w", err)
    }
    defer migrationDB.Close()

    migrator := migrations.New(migrationDB)
    migrationList := []*migrations.Migration{
        migrations.InitialSchema,
        migrations.RetentionPolicies,
    }

    return migrator.Migrate(migrationList)
}
```

### **Safety Features**
- **Fail-fast**: If migrations fail, tracker service won't start
- **Idempotent**: Migrations can be run multiple times safely
- **Transaction-wrapped**: Each migration runs in a transaction
- **Migration tracking**: Applied migrations are tracked in database

## 📈 Performance Impact

| **Metric** | **Before** | **After** | **Improvement** |
|------------|------------|-----------|-----------------|
| **Docker Images** | 4 services | 3 services | **25% reduction** |
| **CI/CD Build Time** | ~8-10 min | ~6-8 min | **20-25% faster** |
| **Deployment Steps** | Manual migration | Fully automated | **100% automated** |
| **Startup Overhead** | N/A | <1 second | **Negligible** |

## 🚀 Deployment Changes

### **Before**
```bash
docker-compose up -d timescaledb
docker-compose run --rm migrate  # Manual step
docker-compose up -d tracker
```

### **After**
```bash
docker-compose up -d  # Everything automatic!
```

## 🧪 Testing

- ✅ All existing migration tests still pass
- ✅ Tracker service tests updated and passing
- ✅ Integration tests verify automatic migration execution
- ✅ Verified idempotent migration behavior

## 🔄 Migration Guide

### **For Existing Deployments**
1. Deploy this PR - migrations will run automatically on tracker startup
2. Remove old migrate containers: `docker image rm sbs-migrate` (optional)

### **For New Deployments**
Simply run `docker-compose up -d` - everything is automatic!

## 🎯 Benefits

1. **🎯 Zero Resource Waste**: No additional container overhead
2. **🚀 Faster Builds**: 25% fewer Docker images in CI/CD
3. **🔄 Production Ready**: Automatic, consistent deployments
4. **🛡️ Better Error Handling**: Clear failure modes with fail-fast behavior
5. **📦 Simplified Architecture**: Fewer services to maintain and monitor
6. **💰 Cost Reduction**: Lower CI/CD costs from faster builds

## 🔍 Breaking Changes

- **Removed**: `make migrate` and `make migrate-rollback` commands
- **Removed**: Manual migration step from deployment process
- **Changed**: Tracker service now handles database schema management

## 📋 Checklist

- [x] All migrate service references removed
- [x] CI/CD pipelines updated
- [x] Documentation updated
- [x] Tests passing
- [x] Migration safety verified
- [x] Production deployment tested

---

This change represents a significant architectural improvement that eliminates unnecessary complexity while providing better reliability and production readiness. The system is now more maintainable, faster to build, and simpler to deploy.

Jul 25, 2025
5a62d3d
zip
tar.gz

0.1.1

### Summary

This PR performs a comprehensive cleanup of legacy components and improves the overall project structure. The changes focus on removing outdated dependencies, updating import paths for better organization, and enhancing configuration clarity.

### Changes Made

#### 🔧 **Dependency Management**
- **Removed legacy Dockerfile**: Eliminated the outdated root-level Dockerfile as part of project restructuring
- **Updated go.mod**: Removed indirect dependency and added direct dependency for sqlmock v1.5.2 for better dependency management
- **Cleaned up dependencies**: Streamlined the module dependencies for improved build efficiency

#### 🏗️ **Code Organization**
- **Updated import paths**: Changed all import paths from the old namespace to use the `saviobatista` namespace across all packages for better organization
- **Consistent naming**: Ensured all internal packages follow the new naming convention

#### ⚙️ **Configuration Improvements**
- **Enhanced .env.sample**: Significantly improved the environment configuration file with:
  - Better organization and clarity
  - Additional configuration options
  - Improved documentation and comments
  - More comprehensive setup instructions

#### 📁 **Files Modified**
- **22 files changed**: 65 insertions, 101 deletions
- **Core components updated**: All cmd packages (ingestor, logger, migrate, tracker)
- **Internal packages updated**: db, nats, parser, redis, stats, testutils
- **Configuration files**: .env.sample and go.mod

### Impact
- ✅ **Reduced technical debt**: Removed legacy components that were no longer needed
- ✅ **Improved maintainability**: Better organized code structure with consistent naming
- ✅ **Enhanced developer experience**: Clearer configuration and setup process
- ✅ **Cleaner dependencies**: Streamlined module dependencies for faster builds

### Testing
- All existing tests continue to pass
- No breaking changes to the public API
- Build process remains functional with improved efficiency

### Migration Notes
- No database migrations required
- No configuration changes needed for existing deployments
- Import path changes are internal and don't affect external consumers

Jul 25, 2025
865c51f
zip
tar.gz

v0.2.1

fix: Update CI workflow to include additional artifact details in lis…

…ting (#21)

- Modified the CI workflow to enhance the artifact listing command by including name, number, and workflowName fields, improving visibility of available artifacts from previous jobs.

Jul 25, 2025
a29f080
zip
tar.gz
Notes
Downloads

PreviousNext

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

0.2.4