Skip to content

xixu-me/tzst

Tip

欢迎加入“Xget 开源与 AI 交流群”,一起交流开源项目、AI 应用、工程实践、效率工具和独立开发;如果你也在做产品、写代码、折腾项目或者对开源和 AI 感兴趣,欢迎进群认识更多认真做事、乐于分享的朋友。


codecov CodeQL CI/CD PyPI - Version PyPI - Downloads GitHub License Sponsor Documentation

🇺🇸 English | 🇨🇳 汉语 | 🇪🇸 español | 🇯🇵 日本語 | 🇦🇪 العربية | 🇷🇺 русский | 🇩🇪 Deutsch | 🇫🇷 français | 🇰🇷 한국어 | 🇧🇷 português

tzst is a Python 3.12+ library and CLI for creating, extracting, listing, and testing .tzst and .tar.zst archives. It combines tar compatibility, Zstandard compression, streaming support, atomic writes, and safe-by-default extraction in a compact, production-ready interface.

Features

  • High Compression: Zstandard compression for excellent compression ratios and speed
  • Tar Compatibility: Creates standard tar archives compressed with Zstandard
  • Command Line Interface: Intuitive CLI with streaming support and comprehensive options
  • Python API: Clean, Pythonic API for programmatic use
  • Cross-Platform: Works on Windows, macOS, and Linux
  • Multiple Extensions: Supports both .tzst and .tar.zst extensions
  • Memory Efficient: Streaming mode for handling large archives with minimal memory usage
  • Atomic Operations: Safe file operations with automatic cleanup on interruption
  • Secure by Default: Uses the 'data' filter for maximum security during extraction
  • Enhanced Error Handling: Clear error messages with helpful alternatives

Installation

From GitHub Releases

Download standalone executables that don't require Python installation:

Supported Platforms

Platform Architecture File
Linux x86_64 tzst-{version}-linux-amd64.zip
Linux ARM64 tzst-{version}-linux-arm64.zip
Windows x64 tzst-{version}-windows-amd64.zip
Windows ARM64 tzst-{version}-windows-arm64.zip
macOS Intel tzst-{version}-darwin-amd64.zip
macOS Apple Silicon tzst-{version}-darwin-arm64.zip

Installation Steps

  1. Download the appropriate archive for your platform from the latest releases page
  2. Extract the archive to get the tzst executable (or tzst.exe on Windows)
  3. Move the executable to a directory in your PATH:
    • Linux/macOS: sudo mv tzst /usr/local/bin/
    • Windows: Add the directory containing tzst.exe to your PATH environment variable
  4. Verify installation: tzst --help

Benefits of Binary Installation

  • No Python required - Standalone executable
  • Faster startup - No Python interpreter overhead
  • Easy deployment - Single file distribution
  • Consistent behavior - Bundled dependencies

From PyPI

Using pip:

pip install tzst

Or using uv (recommended):

uv tool install tzst

From Source

git clone https://github.com/xixu-me/tzst.git
cd tzst
pip install .

Development Installation

This project uses modern Python packaging standards:

git clone https://github.com/xixu-me/tzst.git
cd tzst
pip install -e .[dev]

Quick Start

Command Line Usage

# Create an archive
tzst a archive.tzst file1.txt file2.txt directory/

# Extract an archive
tzst x archive.tzst

# List archive contents
tzst l archive.tzst

# Test archive integrity
tzst t archive.tzst

Python API Usage

from tzst import create_archive, extract_archive, list_archive

# Create an archive
create_archive("archive.tzst", ["file1.txt", "file2.txt", "directory/"])

# Extract an archive
extract_archive("archive.tzst", "output_directory/")

# List archive contents
contents = list_archive("archive.tzst", verbose=True)
for item in contents:
    print(f"{item['name']}: {item['size']} bytes")

Command Line Interface

Archive Operations

Create Archive

# Basic usage
tzst a archive.tzst file1.txt file2.txt

# With compression level (1-22, default: 3)
tzst a archive.tzst files/ -l 15

# Alternative commands
tzst add archive.tzst files/
tzst create archive.tzst files/

Extract Archive

# Extract with full directory structure
tzst x archive.tzst

# Extract to specific directory
tzst x archive.tzst -o output/

# Extract specific files
tzst x archive.tzst file1.txt dir/file2.txt

# Extract without directory structure (flat)
tzst e archive.tzst -o output/

# Use streaming mode for large archives
tzst x archive.tzst --streaming -o output/

List Contents

# Simple listing
tzst l archive.tzst

# Verbose listing with details
tzst l archive.tzst -v

# Use streaming mode for large archives
tzst l archive.tzst --streaming -v

Test Integrity

# Test archive integrity
tzst t archive.tzst

# Test with streaming mode
tzst t archive.tzst --streaming

Command Reference

Command Aliases Description Streaming Support
a add, create Create or add to archive N/A
x extract Extract with full paths --streaming
e extract-flat Extract without directory structure --streaming
l list List archive contents --streaming
t test Test archive integrity --streaming

CLI Options

  • -v, --verbose: Enable verbose output
  • -o, --output DIR: Specify output directory (extract commands)
  • -l, --level LEVEL: Set compression level 1-22 (create command)
  • --streaming: Enable streaming mode for memory-efficient processing
  • --filter FILTER: Security filter for extraction (data/tar/fully_trusted)
  • --no-atomic: Disable atomic file operations (not recommended)

Security Filters

# Extract with maximum security (default)
tzst x archive.tzst --filter data

# Extract with standard tar compatibility
tzst x archive.tzst --filter tar

# Extract with full trust (dangerous - only for trusted archives)
tzst x archive.tzst --filter fully_trusted

Security Filter Options:

  • data (default): Most secure. Blocks dangerous files, absolute paths, and paths outside extraction directory
  • tar: Standard tar compatibility. Blocks absolute paths and directory traversal
  • fully_trusted: No security restrictions. Only use with completely trusted archives

Python API

TzstArchive Class

from tzst import TzstArchive

# Create a new archive
with TzstArchive("archive.tzst", "w", compression_level=5) as archive:
    archive.add("file.txt")
    archive.add("directory/", recursive=True)

# Read an existing archive
with TzstArchive("archive.tzst", "r") as archive:
    # List contents
    contents = archive.list(verbose=True)
    
    # Extract with security filter
    archive.extract("file.txt", "output/", filter="data")
    
    # Test integrity
    is_valid = archive.test()

# For large archives, use streaming mode
with TzstArchive("large_archive.tzst", "r", streaming=True) as archive:
    archive.extract(path="output/")

Important Limitations:

  • Append Mode Not Supported: Create multiple archives or recreate the entire archive instead

Convenience Functions

create_archive()

from tzst import create_archive

# Create with atomic operations (default)
create_archive(
    archive_path="backup.tzst",
    files=["documents/", "photos/", "config.txt"],
    compression_level=10
)

extract_archive()

from tzst import extract_archive

# Extract with security (default: 'data' filter)
extract_archive("backup.tzst", "restore/")

# Extract specific files
extract_archive("backup.tzst", "restore/", members=["config.txt"])

# Flatten directory structure
extract_archive("backup.tzst", "restore/", flatten=True)

# Use streaming for large archives
extract_archive("large_backup.tzst", "restore/", streaming=True)

list_archive()

from tzst import list_archive

# Simple listing
files = list_archive("backup.tzst")

# Detailed listing
files = list_archive("backup.tzst", verbose=True)

# Streaming for large archives
files = list_archive("large_backup.tzst", streaming=True)

test_archive()

from tzst import test_archive

# Basic integrity test
if test_archive("backup.tzst"):
    print("Archive is valid")

# Test with streaming
if test_archive("large_backup.tzst", streaming=True):
    print("Large archive is valid")

Advanced Features

File Extensions

The library automatically handles file extensions with intelligent normalization:

  • .tzst - Primary extension for tar+zstandard archives
  • .tar.zst - Alternative standard extension
  • Auto-detection when opening existing archives
  • Automatic extension addition when creating archives
# These all create valid archives
create_archive("backup.tzst", files)      # Creates backup.tzst
create_archive("backup.tar.zst", files)  # Creates backup.tar.zst  
create_archive("backup", files)          # Creates backup.tzst
create_archive("backup.txt", files)      # Creates backup.tzst (normalized)

Compression Levels

Zstandard compression levels range from 1 (fastest) to 22 (best compression):

  • Level 1-3: Fast compression, larger files
  • Level 3 (default): Good balance of speed and compression
  • Level 10-15: Better compression, slower
  • Level 20-22: Maximum compression, much slower

Streaming Mode

Use streaming mode for memory-efficient processing of large archives:

Benefits:

  • Significantly reduced memory usage
  • Better performance for archives that don't fit in memory
  • Automatic cleanup of resources

When to Use:

  • Archives larger than 100MB
  • Limited memory environments
  • Processing archives with many large files
# Example: Processing a large backup archive
from tzst import extract_archive, list_archive, test_archive

large_archive = "backup_500gb.tzst"

# Memory-efficient operations
is_valid = test_archive(large_archive, streaming=True)
contents = list_archive(large_archive, streaming=True, verbose=True)
extract_archive(large_archive, "restore/", streaming=True)

Atomic Operations

All file creation operations use atomic file operations by default:

  • Archives created in temporary files first, then atomically moved
  • Automatic cleanup if process is interrupted
  • No risk of corrupted or incomplete archives
  • Cross-platform compatibility
# Atomic operations enabled by default
create_archive("important.tzst", files)  # Safe from interruption

# Can be disabled if needed (not recommended)
create_archive("test.tzst", files, use_temp_file=False)

Error Handling

from tzst import TzstArchive
from tzst.exceptions import (
    TzstError,
    TzstArchiveError,
    TzstCompressionError,
    TzstDecompressionError,
    TzstFileNotFoundError
)

try:
    with TzstArchive("archive.tzst", "r") as archive:
        archive.extract()
except TzstDecompressionError:
    print("Failed to decompress archive")
except TzstFileNotFoundError:
    print("Archive file not found")
except KeyboardInterrupt:
    print("Operation interrupted by user")
    # Cleanup handled automatically

Performance and Comparison

Performance Tips

  1. Compression levels: Level 3 is optimal for most use cases
  2. Streaming: Use for archives larger than 100MB
  3. Batch operations: Add multiple files in single session
  4. File types: Already compressed files won't compress much further

vs Other Tools

vs tar + gzip:

  • Better compression ratios
  • Faster decompression
  • Modern algorithm

vs tar + xz:

  • Significantly faster compression
  • Similar compression ratios
  • Better speed/compression trade-off

vs zip:

  • Better compression
  • Preserves Unix permissions and metadata
  • Better streaming support

Requirements

  • Python 3.12 or higher (tested on 3.12-3.14)
  • zstandard >= 0.19.0

Development

Setting up Development Environment

This project uses modern Python packaging standards:

git clone https://github.com/xixu-me/tzst.git
cd tzst
pip install -e .[dev]

Running Tests

# Run tests with coverage
pytest --cov=tzst --cov-report=html

# Or use the simpler command (coverage settings are in pyproject.toml)
pytest

Code Quality

# Check code quality
ruff check src tests

# Format code
ruff format src tests

Contributing

We welcome contributions! Please read our Contributing Guide for:

  • Development setup and project structure
  • Code style guidelines and best practices
  • Testing requirements and writing tests
  • Pull request process and review workflow

Quick Start for Contributors

git clone https://github.com/xixu-me/tzst.git
cd tzst
pip install -e .[dev]
python -m pytest tests/

Types of Contributions Welcome

  • Bug fixes - Fix issues in existing functionality
  • Features - Add new capabilities to the library
  • Documentation - Improve or add documentation
  • Tests - Add or improve test coverage
  • Performance - Optimize existing code
  • Security - Address security vulnerabilities

Acknowledgments

License

Copyright © Xi Xu. All rights reserved.

Licensed under the BSD 3-Clause license.

About

Python 3.12+ library and CLI for creating, extracting, listing, and testing .tzst and .tar.zst archives

Topics

Resources

License

Code of conduct

Contributing

Security policy

Stars

Watchers

Forks

Sponsor this project

Contributors

Languages