Skip to content

πŸ“ Modern C++20 async logging framework with 4.34M msg/sec throughput, 148ns latency, and modular architecture

License

Notifications You must be signed in to change notification settings

kcenon/logger_system

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

CI Sanitizers Benchmarks Code Coverage Static Analysis Docs

Logger System

Language: English | ν•œκ΅­μ–΄

Overview

A high-performance C++20 asynchronous logging framework designed for multithreaded applications. Built with a modular, interface-based architecture and seamless ecosystem integration.

Key Features:

  • πŸš€ Ultra-fast async logging: 4.34M messages/second, 148ns latency
  • πŸ”’ Thread-safe by design: Zero data races, production-proven
  • πŸ—οΈ Modular architecture: Interface-driven, pluggable components
  • πŸ›‘οΈ Production-grade: Comprehensive CI/CD, sanitizers, benchmarks
  • πŸ” Security-first: Path validation, secure storage, audit logging
  • 🌐 Cross-platform: Windows, Linux, macOS with GCC, Clang, MSVC

Quick Start

Basic Example

#include <kcenon/logger/core/logger_builder.h>
#include <kcenon/logger/writers/console_writer.h>
#include <kcenon/logger/writers/file_writer.h>

int main() {
    // Create logger using builder pattern with automatic validation
    auto result = kcenon::logger::logger_builder()
        .use_template("production")  // Predefined configuration
        .with_min_level(kcenon::logger::log_level::info)
        .add_writer("console", std::make_unique<kcenon::logger::console_writer>())
        .add_writer("file", std::make_unique<kcenon::logger::file_writer>("app.log"))
        .build();

    if (result.is_err()) {
        const auto& err = result.error();
        std::cerr << "Failed to create logger: " << err.message
                  << " (code: " << err.code << ")\n";
        return -1;
    }

    auto logger = std::move(result.value());

    // Log messages with error handling
    logger->log(kcenon::logger::log_level::info, "Application started");
logger->log(kcenon::logger::log_level::error, "Something went wrong");

return 0;
}

Need a quick reminder later? See the Result Handling Cheatsheet for canonical snippets that can be reused across the ecosystem.

Installation

Using CMake:

mkdir build && cd build
cmake ..
cmake --build .
cmake --build . --target install

Using in Your Project:

find_package(LoggerSystem REQUIRED)
target_link_libraries(your_app PRIVATE LoggerSystem::logger)

Requirements

Dependency Version Required Description
C++20 Compiler GCC 11+ / Clang 14+ / MSVC 2022+ / Apple Clang 14+ Yes C++20 features required
CMake 3.20+ Yes Build system
common_system latest Yes Common interfaces (ILogger, Result)
thread_system latest Optional Async logging with thread pool support
vcpkg latest Optional Package management
fmt latest Optional Formatting library (header-only mode available)

Dependency Flow

logger_system
β”œβ”€β”€ common_system (required)
└── thread_system (optional, for async logging with thread pool)
    └── common_system (required)

Building with Dependencies

# Clone dependencies
git clone https://github.com/kcenon/common_system.git
git clone https://github.com/kcenon/thread_system.git  # Optional

# Clone and build logger_system
git clone https://github.com/kcenon/logger_system.git
cd logger_system
cmake -B build -DLOGGER_USE_THREAD_SYSTEM=ON  # Enable thread_system integration
cmake --build build

Core Features

Asynchronous Logging

  • Non-blocking operations: Background thread handles I/O without blocking
  • Batched processing: Processes multiple log entries efficiently
  • Adaptive batching: Intelligent optimization based on queue utilization
  • Zero-copy design: Minimal allocations and overhead

Multiple Writer Types

  • Console Writer: ANSI colored output for different log levels
  • File Writer: Buffered file output with configurable settings
  • Rotating File Writer: Size/time-based rotation with compression
  • Network Writer: TCP/UDP remote logging
  • Critical Writer: Synchronous logging for critical messages
  • Hybrid Writer: Automatic async/sync switching based on log level

πŸ“š Detailed Writer Documentation β†’

Security Features (v3.0.0)

  • Secure Key Storage: RAII-based encryption key management with automatic cleanup
  • Path Validation: Protection against path traversal attacks
  • Signal Handler Safety: Emergency flush for crash scenarios
  • Security Audit Logging: Tamper-evident audit trail with HMAC-SHA256
  • Compliance Support: GDPR, PCI DSS, ISO 27001, SOC 2

πŸ”’ Complete Security Guide β†’


Performance Highlights

Benchmarked on Apple M1 (8-core) @ 3.2GHz, 16GB, macOS Sonoma

Throughput

Configuration Throughput vs spdlog
Single thread (async) 4.34M msg/s -19%
4 threads 1.07M msg/s +36%
8 threads 412K msg/s +72%
16 threads 390K msg/s +117%

Latency

Metric Logger System spdlog async Advantage
Average 148 ns 2,325 ns 15.7x faster
p99 312 ns 4,850 ns 15.5x faster
p99.9 487 ns ~7,000 ns 14.4x faster

Memory Efficiency

  • Baseline: 1.8 MB (vs spdlog: 4.2 MB, 57% less)
  • Peak: 2.4 MB
  • Allocations/msg: 0.12

Key Insights:

  • πŸƒ Multi-threaded advantage: Adaptive batching provides superior scaling
  • ⏱️ Ultra-low latency: Industry-leading 148ns average enqueue time
  • πŸ’Ύ Memory efficient: Minimal footprint with zero-copy design

⚑ Full Benchmarks & Methodology β†’


Architecture Overview

Modular Design

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                      Logger Core                            β”‚
β”‚  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”      β”‚
β”‚  β”‚   Builder    β”‚  β”‚  Async Queue β”‚  β”‚   Metrics    β”‚      β”‚
β”‚  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜      β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                        β”‚
        β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
        β”‚               β”‚               β”‚
        β–Ό               β–Ό               β–Ό
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚   Writers    β”‚ β”‚   Filters    β”‚ β”‚  Formatters  β”‚
β”‚              β”‚ β”‚              β”‚ β”‚              β”‚
β”‚ β€’ Console    β”‚ β”‚ β€’ Level      β”‚ β”‚ β€’ Plain Text β”‚
β”‚ β€’ File       β”‚ β”‚ β€’ Regex      β”‚ β”‚ β€’ JSON       β”‚
β”‚ β€’ Rotating   β”‚ β”‚ β€’ Function   β”‚ β”‚ β€’ Logfmt     β”‚
β”‚ β€’ Network    β”‚ β”‚ β€’ Composite  β”‚ β”‚ β€’ Custom     β”‚
β”‚ β€’ Critical   β”‚ β”‚              β”‚ β”‚              β”‚
β”‚ β€’ Hybrid     β”‚ β”‚              β”‚ β”‚              β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Key Components

  • Logger Core: Main async processing engine with builder pattern
  • Writers: Pluggable output destinations (file, console, network, etc.)
  • Filters: Conditional logging based on level, pattern, or custom logic
  • Formatters: Configurable output formats (plain, JSON, logfmt, custom)
  • Security: Path validation, secure storage, audit logging

πŸ›οΈ Detailed Architecture Guide β†’


Ecosystem Integration

Part of a modular C++ ecosystem with clean interface boundaries:

Dependencies

Required:

  • common_system: Core interfaces (ILogger, IMonitor, Result) with C++20 Concepts support

Optional:

Note: Since v3.1.0, thread_system is optional. The logger system uses a standalone std::jthread implementation by default. For advanced async processing, enable thread_system integration with -DLOGGER_USE_THREAD_SYSTEM=ON (see Issue #224).

Integration Pattern

#include <kcenon/logger/core/logger.h>
#include <kcenon/logger/core/logger_builder.h>
#include <kcenon/logger/writers/console_writer.h>

int main() {
    // Create logger using builder pattern (standalone mode, no thread_system required)
    auto logger = kcenon::logger::logger_builder()
        .use_template("production")
        .add_writer("console", std::make_unique<kcenon::logger::console_writer>())
        .build()
        .value();

    // Use logger anywhere in your application
    logger->log(kcenon::logger::log_level::info, "System initialized");

    return 0;
}

Note: When thread_system is available and LOGGER_HAS_THREAD_SYSTEM is defined (via -DLOGGER_USE_THREAD_SYSTEM=ON), additional integration features are enabled including shared thread pool for async processing. See thread_system integration for details.

Benefits:

  • Interface-only dependencies (no circular references)
  • Independent compilation and deployment
  • Runtime component injection via DI pattern
  • Clean separation of concerns

πŸ”— Ecosystem Integration Guide β†’


Documentation

Getting Started

Core Documentation

Advanced Topics

Development


Configuration Templates

The logger system provides predefined templates for common scenarios:

// Production: Optimized for production environments
auto logger = kcenon::logger::logger_builder()
    .use_template("production")
    .build()
    .value();

// Debug: Immediate output for development
auto logger = kcenon::logger::logger_builder()
    .use_template("debug")
    .build()
    .value();

// High-performance: Maximized throughput
auto logger = kcenon::logger::logger_builder()
    .use_template("high_performance")
    .build()
    .value();

// Low-latency: Minimized latency for real-time systems
auto logger = kcenon::logger::logger_builder()
    .use_template("low_latency")
    .build()
    .value();

Advanced Configuration

auto logger = kcenon::logger::logger_builder()
    // Core settings
    .with_min_level(kcenon::logger::log_level::info)
    .with_buffer_size(16384)
    .with_batch_size(200)
    .with_queue_size(20000)

    // Add multiple writers
    .add_writer("console", std::make_unique<kcenon::logger::console_writer>())
    .add_writer("file", std::make_unique<kcenon::logger::rotating_file_writer>(
        "app.log",
        10 * 1024 * 1024,  // 10MB per file
        5                   // Keep 5 files
    ))

    // Build with validation
    .build()
    .value();

πŸ“š Complete Configuration Guide β†’


Build Configuration

CMake Feature Flags

# Core Features
cmake -DLOGGER_USE_DI=ON              # Dependency injection (default: ON)
cmake -DLOGGER_USE_MONITORING=ON      # Monitoring support (default: ON)
cmake -DLOGGER_ENABLE_ASYNC=ON        # Async logging (default: ON)
cmake -DLOGGER_ENABLE_CRASH_HANDLER=ON # Crash handler (default: ON)

# Advanced Features
cmake -DLOGGER_ENABLE_STRUCTURED_LOGGING=ON # JSON logging (default: OFF)
cmake -DLOGGER_ENABLE_NETWORK_WRITER=ON # Network writer (default: OFF)
cmake -DLOGGER_ENABLE_FILE_ROTATION=ON  # File rotation (default: ON)

# Performance Tuning
cmake -DLOGGER_DEFAULT_BUFFER_SIZE=16384 # Buffer size in bytes
cmake -DLOGGER_DEFAULT_BATCH_SIZE=200    # Batch processing size
cmake -DLOGGER_DEFAULT_QUEUE_SIZE=20000  # Maximum queue size

# Quality Assurance
cmake -DLOGGER_ENABLE_SANITIZERS=ON   # Enable sanitizers
cmake -DLOGGER_ENABLE_COVERAGE=ON     # Code coverage
cmake -DLOGGER_WARNINGS_AS_ERRORS=ON  # Treat warnings as errors

πŸ”§ Complete Build Options β†’


Platform Support

Officially Supported

Platform Architecture Compilers Status
Ubuntu 22.04+ x86_64, ARM64 GCC 11+, Clang 14+ βœ… Fully tested
macOS Sonoma+ x86_64, ARM64 (M1/M2) Apple Clang 14+ βœ… Fully tested
Windows 11 x86_64 MSVC 2022 βœ… Fully tested

Minimum Requirements:

  • C++20 compiler
  • CMake 3.20+
  • fmt library

πŸ–₯️ Platform Details β†’


Testing

The logger system includes comprehensive testing infrastructure:

Test Coverage

  • Unit Tests: 150+ test cases (GTest)
  • Integration Tests: 30+ scenarios
  • Benchmarks: 20+ performance tests
  • Coverage: ~65% (growing)

Running Tests

# Build with tests
cmake -DBUILD_TESTS=ON ..
cmake --build .

# Run all tests
ctest --output-on-failure

# Run specific test suite
./build/bin/core_tests
./build/bin/writer_tests

# Run benchmarks
./build/bin/benchmarks

CI/CD Status

All pipelines green:

  • βœ… Multi-platform builds (Ubuntu, macOS, Windows)
  • βœ… Sanitizers (Thread, Address, UB)
  • βœ… Performance benchmarks
  • βœ… Code coverage
  • βœ… Static analysis (clang-tidy, cppcheck)

βœ… Production Quality Metrics β†’


Contributing

We welcome contributions! Please see our Contributing Guide for details.

Development Workflow

  1. Fork the repository
  2. Create your feature branch: git checkout -b feature/amazing-feature
  3. Commit your changes: git commit -m 'Add amazing feature'
  4. Push to the branch: git push origin feature/amazing-feature
  5. Open a Pull Request

Code Standards

  • Follow modern C++ best practices
  • Use RAII and smart pointers
  • Write comprehensive unit tests
  • Maintain consistent formatting (clang-format)
  • Document public APIs

🀝 Contributing Guidelines β†’


Support


License

This project is licensed under the BSD 3-Clause License - see the LICENSE file for details.


Acknowledgments

  • Inspired by modern logging frameworks (spdlog, Boost.Log, glog)
  • Built with C++20 features (GCC 11+, Clang 14+, MSVC 2022+) for maximum performance and safety
  • Maintained by kcenon@naver.com

Made with ❀️ by πŸ€β˜€πŸŒ•πŸŒ₯ 🌊

About

πŸ“ Modern C++20 async logging framework with 4.34M msg/sec throughput, 148ns latency, and modular architecture

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published