This directory contains utility scripts for building and signing security tests in the F0RT1KA framework.

A comprehensive build utility for compiling security tests with cross-platform support.

• Cross-platform compilation (Windows, Linux, macOS)
• Multiple architecture support (amd64, 386, arm64)
• Build individual tests or all tests
• Validation of test structure
• Colored output and verbose logging

# List all available tests
./utils/gobuild list

# Build a specific test for Windows (default)
./utils/gobuild build tests_source/ecd2514c-512a-4251-a6f4-eb3aa834d401

# Build for Linux
./utils/gobuild --os linux build tests_source/ecd2514c-512a-4251-a6f4-eb3aa834d401

# Build all tests
./utils/gobuild build-all

# Build with custom output directory
./utils/gobuild --output dist/ build-all

# Clean build artifacts
./utils/gobuild clean

• Go 1.21 or higher
• Valid test structure in tests_source/

A utility for signing Windows executables using osslsigncode with support for certificate management.

• Interactive certificate selection
• Secure password prompting
• Batch signing of multiple binaries
• Signature verification
• Timestamping support
• Certificate listing

# List available certificates
./utils/codesign list-certs

# Sign a binary (interactive certificate selection)
./utils/codesign sign build/test-uuid/test-uuid.exe

# Sign with specific certificate
./utils/codesign --cert signing-certs/mycert.pfx sign build/test-uuid/test-uuid.exe

# Sign all binaries in build directory
./utils/codesign sign-all

# Sign with metadata
./utils/codesign --cert signing-certs/mycert.pfx \
    --description "F0RT1KA Security Test" \
    --url "https://github.com/ubercylon8/f0_library" \
    sign build/test-uuid/test-uuid.exe

# Verify a signature
./utils/codesign verify build/test-uuid/test-uuid.exe

• osslsigncode installed
• Code signing certificates in PFX/P12 format
• Certificates placed in signing-certs/ directory

# macOS
brew install osslsigncode

# Ubuntu/Debian
sudo apt-get install osslsigncode

# CentOS/RHEL
sudo yum install osslsigncode

A PowerShell utility that verifies Windows Defender protection status by examining registry settings targeted by malware like CyberEye RAT.

• Checks critical Windows Defender registry values
• Determines if host is protected or vulnerable
• Color-coded output for easy status identification
• Verifies Tamper Protection, Anti-Spyware, and Real-Time Protection settings
• Queries actual Defender status via PowerShell cmdlets

# Run with Administrator privileges
powershell -ExecutionPolicy Bypass -File ./utils/Check-DefenderProtection.ps1

• HKLM:\SOFTWARE\Microsoft\Windows Defender\Features\TamperProtection
• HKLM:\SOFTWARE\Policies\Microsoft\Windows Defender\DisableAntiSpyware
• HKLM:\SOFTWARE\Policies\Microsoft\Windows Defender\Real-Time Protection\DisableBehaviorMonitoring
• HKLM:\SOFTWARE\Policies\Microsoft\Windows Defender\Real-Time Protection\DisableOnAccessProtection
• HKLM:\SOFTWARE\Policies\Microsoft\Windows Defender\Real-Time Protection\DisableScanOnRealtimeEnable

• PROTECTED (Green): Registry values indicate Defender features are enabled
• VULNERABLE (Red): One or more Defender features are disabled via registry
• Not Set: Registry key doesn't exist (default protection applies)

• Windows PowerShell 5.0 or higher
• Administrator privileges
• Windows 10/11 or Windows Server 2016+

A validation utility for checking attack flow HTML diagrams for JavaScript syntax errors, specifically focusing on unescaped backticks that cause "Invalid or unexpected token" errors in template literals.

• Detects unescaped backticks in JavaScript template literals
• Validates PowerShell code block syntax
• Checks HTML structure and required components
• Verifies MITRE ATT&CK technique mappings
• Color-coded output with detailed error reporting
• Support for verbose and quiet modes

# Validate a single HTML file
./utils/validate-attack-flow-html.sh sample_attack_flow.html

# Verbose validation with detailed output
./utils/validate-attack-flow-html.sh -v tests_source/akira_attack_flow.html

# Quiet mode - only show errors
./utils/validate-attack-flow-html.sh -q *.html

# Validate multiple files
find tests_source/ -name "*.html" -exec ./utils/validate-attack-flow-html.sh {} \;

• Backtick Escaping: Detects unescaped backticks in template literals
• PowerShell Syntax: Validates PowerShell code blocks for proper escaping
• HTML Structure: Verifies presence of required script tags and functions
• MITRE Mappings: Checks for MITRE ATT&CK technique references
• CSS Classes: Validates required CSS classes for interactive components
• Template Literals: Identifies malformed JavaScript template literals

• 0: Validation passed (no errors)
• 1: Validation failed (errors found that need fixing)

• Bash 4.0 or higher
• grep, sed utilities

An automatic repair utility that fixes unescaped backticks in attack flow HTML files to prevent JavaScript syntax errors. Creates backups before making changes.

• Automatically escapes unescaped backticks in PowerShell code snippets
• Creates backup copies before making changes
• Dry-run mode to preview changes without modification
• Targeted fixing for JavaScript template literals
• Safe processing that preserves legitimate template literals
• Color-coded output showing changes made

# Fix backticks in a file (creates backup automatically)
./utils/fix-attack-flow-backticks.sh sample_attack_flow.html

# Dry run to see what would be changed
./utils/fix-attack-flow-backticks.sh --dry-run sample_attack_flow.html

# Verbose output with detailed processing
./utils/fix-attack-flow-backticks.sh -v tests_source/akira_attack_flow.html

# Custom backup directory
./utils/fix-attack-flow-backticks.sh --backup ./backups sample_attack_flow.html

• Identifies JavaScript template literals containing PowerShell commands
• Escapes unescaped backticks in PowerShell code snippets
• Preserves legitimate template literal syntax
• Handles backticks at line beginnings and within content
• Creates automatic backups for safety

• Default: Creates .bak file in same directory
• Custom: Use --backup to specify backup directory
• Automatic cleanup if no changes needed

• Bash 4.0 or higher
• sed, grep utilities
• Write permissions in target directory

A Node.js script that demonstrates the JavaScript template literal backtick escaping issue and shows the correct solution. This educational tool helps understand why the validation and fix utilities are necessary.

• Shows problematic PowerShell code patterns that break JavaScript
• Demonstrates correct backtick escaping techniques
• Provides working code examples
• Explains the root cause of "Invalid or unexpected token" errors
• Lists common PowerShell patterns that need escaping

# Run the demo to understand the issue and solution
node utils/example-backtick-fix.js

# Use Node.js if available
./utils/example-backtick-fix.js

• PowerShell line continuation backticks () must be escaped as (\\) inside JavaScript template literals
• Template literal delimiters (outer backticks) should NOT be escaped
• Only escape backticks that are CONTENT within the template literal
• Test generated HTML in browser console to verify syntax

• Node.js (any recent version)
• No additional dependencies

A Python utility for querying LimaCharlie sensor events specifically for F0RT1KA security test analysis with detailed reporting capabilities.

• Query events by test UUID and date range using LCQL
• Generate formatted tables with key event data
• Provide statistics on error codes and test outcomes
• Show endpoints tested with event timestamps
• Support multiple output formats (table, JSON, CSV)
• Flexible authentication via environment variables or .env files

# Query events using .env file credentials
python3 utils/lc_events_query.py --uuid "abc123def456" --date-range "last 24 hours"

# Query with explicit credentials
python3 utils/lc_events_query.py --uuid "abc123def456" --date-range "last 24 hours" -k API_KEY -o ORG_ID

# Show only endpoints tested
python3 utils/lc_events_query.py --uuid "abc123def456" --date-range "last 7 days" --hostnames

# Export to JSON for further processing
python3 utils/lc_events_query.py --uuid "abc123def456" --date-range "today" --output results.json

• Command line: -k API_KEY -o ORG_ID
• Environment variables: LIMACHARLIE_API_KEY, LIMACHARLIE_OID
• .env file: LC_API_KEY, LC_ORG_ID
• Custom .env file: --env-file path/to/.env

• Python 3.7+
• LimaCharlie sensor binary (lc-sensors) in PATH or F0_CST/bin/
• Valid LimaCharlie API key and Organization ID
• Optional: python-dotenv package for enhanced .env support

A comprehensive Python utility that correlates LimaCharlie sensor events with Microsoft Defender alerts for F0RT1KA security test analysis, providing unified visibility across security platforms.

• Correlates LimaCharlie events with Microsoft Defender alerts by hostname and timestamp
• Flexible hostname matching supports both FQDN and short hostnames
• Configurable time window for correlation matching (default: 5 minutes)
• Comprehensive correlation statistics and success rates
• Detailed analysis of unmatched events with time differences
• Multiple output formats (table, JSON)
• Dual authentication support for both LimaCharlie and Microsoft Defender

# Basic correlation analysis
python3 utils/combine_test_results.py --uuid "abc123def456" --date-range "last 24 hours"

# Export to JSON for further processing
python3 utils/combine_test_results.py --uuid "abc123def456" --date-range "today" --output results.json

# Custom time window for correlation
python3 utils/combine_test_results.py --uuid "abc123def456" --date-range "last 7 days" --time-window 10

# Use custom .env file for credentials
python3 utils/combine_test_results.py --uuid "abc123def456" --date-range "today" --env-file custom.env

LimaCharlie: API key and Organization ID via environment variables or .env file Microsoft Defender: Azure tenant ID, client ID, and client secret

• High Correlation Rates: Typically achieves 90%+ correlation between platforms
• Hostname Normalization: Matches DESKTOP-ABC123 with DESKTOP-ABC123.domain.com
• Time-based Correlation: Finds alerts within configurable time windows
• Unmatched Analysis: Explains why events didn't correlate (hostname mismatch, time difference)
• Comprehensive Statistics: Error code distribution, severity analysis, per-endpoint breakdown

• Python 3.7+
• Both lc_events_query.py and defender_alert_query.py utilities
• LimaCharlie credentials (API key, Org ID)
• Microsoft Defender credentials (Azure tenant, client ID, secret)
• Optional: python-dotenv package for .env file support

Wrapper script that checks dependencies and invokes validate_test_results.py.

./utils/validate build/<uuid>/test_execution_log.json   # Single file
./utils/validate --directory build/<uuid>/               # Directory
./utils/validate --all                                   # All results in build/

Validates test result JSON files against Schema v2.0 structure requirements.

python3 utils/validate_test_results.py build/<uuid>/test_execution_log.json
python3 utils/validate_test_results.py --all

Validates that README.md and info.md files use the correct score format patterns expected by the metadata extractor.

./utils/validate-score-format.sh              # All tests
./utils/validate-score-format.sh <test-uuid>  # Specific test

Validates URLs in _references.md files for HTTP status, MITRE ATT&CK technique correctness, and broken links.

python3 utils/validate-reference-urls.py                    # All tests
python3 utils/validate-reference-urls.py --uuid <test-uuid> # Specific test
python3 utils/validate-reference-urls.py --fix              # Auto-fix known broken URLs
python3 utils/validate-reference-urls.py --dry-run          # Preview fixes

Maps and fixes known broken URLs in _references.md files using predefined replacements (e.g., docs.microsoft.com → learn.microsoft.com migrations).

python3 utils/fix-broken-urls.py  # Operates on files in place

Synchronizes test metadata from tests_source/ to Elasticsearch for the enrichment pipeline.

source .venv/bin/activate
python3 utils/sync-test-catalog-to-elasticsearch.py
python3 utils/sync-test-catalog-to-elasticsearch.py --dry-run

Creates visualizations and dashboards for F0RT1KA test results in Kibana using the Saved Objects API.

python3 utils/create-kibana-dashboard.py
python3 utils/create-kibana-dashboard.py --dry-run
python3 utils/create-kibana-dashboard.py --delete  # Remove dashboards

Generates synthetic enriched RECEIPT events for Elasticsearch visualization experimentation.

python3 utils/generate-synthetic-test-data.py
python3 utils/generate-synthetic-test-data.py --count 500 --dry-run

Enriches historical test execution results with metadata and uploads them to Elasticsearch.

python3 utils/upload-legacy-results.py --input results.json
python3 utils/upload-legacy-results.py --input results.json --dry-run --verbose

Lightweight Go utility for collecting F0RT1KA test result JSON files from Windows endpoints and exporting to Elasticsearch with schema v2.0 validation and state management.

# Build and deploy
cd utils/f0_collector && go build -o f0_collector.exe .
# Deploy via PowerShell
powershell -File utils/f0_collector/deploy-collector-task.ps1

Bash helper library for resolving Elasticsearch profile identifiers to endpoints, indices, and API keys from elasticsearch-registry.json.

source utils/resolve_es.sh
resolve_es_to_endpoint "production"
resolve_es_to_index "production"
resolve_es_to_apikey "production"

Displays all F0RT1KA security tests in a paginated formatted table.

python3 utils/get_tests.py      # Page 1 (default)
python3 utils/get_tests.py 2    # Page 2

Analyzes LCQL NDJSON query results for test execution events, providing analysis of ERROR codes, STDOUT/STDERR patterns, and protection metrics.

python3 utils/analyze_test_results.py rga_all_tests.json tpsgl_all_tests.json
python3 utils/analyze_test_results.py --output-dir ./analysis_output results.json
python3 utils/analyze_test_results.py --console-only results.json

Queries Microsoft Defender 365 alerts for security testing workflows using Azure AD service principal authentication.

python3 utils/defender_alert_query.py --days 7
python3 utils/defender_alert_query.py --severity high --search-term "F0RT1KA"
python3 utils/defender_alert_query.py --output alerts.json --env-file .env

Updates severity assignments in test Go files based on MITRE ATT&CK technique mapping using F0RT1KA Severity Framework v2.

python3 utils/update-severity.py  # Auto-scans tests_source/

Installs the F0RT1KA code signing certificate to the Trusted Root store with embedded Base64-encoded certificate. Requires admin privileges.

powershell -ExecutionPolicy Bypass -File ./utils/Install-F0RT1KA-Certificate.ps1

Diagnostic tool for troubleshooting F0RT1KA certificate installation — checks execution context, Root Store, test logs, and certificate validity.

powershell -ExecutionPolicy Bypass -File ./utils/Check-F0RT1KA-Certificate.ps1

Bash helper library for resolving organization identifiers to certificate file paths from organization-registry.json.

source utils/resolve_org.sh
resolve_org_to_cert "sb"

Generates _references.md files for existing tests by extracting data from info.md References sections and Go metadata headers.

python3 utils/generate-retroactive-references.py
python3 utils/generate-retroactive-references.py --uuid <test-uuid>
python3 utils/generate-retroactive-references.py --dry-run

Monitors and displays recent Windows registry changes via WMI events, audit logs, and snapshot comparisons. Requires admin privileges.

powershell -ExecutionPolicy Bypass -File ./utils/Monitor-RegistryChanges.ps1

Runs all utility unit tests with colored pass/fail reporting.

./utils/run_tests.sh
./utils/run_tests.sh --verbose

Unit test suites for the codesign and gobuild utilities.

./utils/test_codesign.sh
./utils/test_gobuild.sh

Template automating the multi-stage build process where each ATT&CK technique is a separate signed binary embedded in the main orchestrator.

1. Build Tests:

   ./utils/gobuild build-all

2. Sign Binaries:

   ./utils/codesign sign-all

3. Verify Signatures:

   find build/ -name "*.exe" -exec ./utils/codesign verify {} \;

1. Validate Generated HTML:

   ./utils/validate-attack-flow-html.sh tests_source/your_attack_flow.html

2. Fix Backtick Issues (if needed):

   # Preview fixes first
   ./utils/fix-attack-flow-backticks.sh --dry-run tests_source/your_attack_flow.html
   
   # Apply fixes
   ./utils/fix-attack-flow-backticks.sh tests_source/your_attack_flow.html

3. Re-validate After Fixes:

   ./utils/validate-attack-flow-html.sh tests_source/your_attack_flow.html

4. Batch Validation of All HTML Files:

   find tests_source/ -name "*attack_flow*.html" -exec ./utils/validate-attack-flow-html.sh {} \;

1. Sync test catalog after creating/updating tests:

   source .venv/bin/activate
   python3 utils/sync-test-catalog-to-elasticsearch.py

2. Re-execute enrichment policy in Kibana Dev Tools:

   POST /_enrich/policy/f0rtika-test-enrichment/_execute
   

3. Create/update dashboards:

   python3 utils/create-kibana-dashboard.py

1. Validate test results:

   ./utils/validate --all

2. Check score formats:

   ./utils/validate-score-format.sh

3. Validate reference URLs:

   python3 utils/validate-reference-urls.py

• Place certificates in signing-certs/ directory
• Supported formats: .pfx, .p12
• Certificates should be password-protected

• Default target: Windows/amd64
• Default output: build/ directory
• Each test builds to its own subdirectory

See utils/setup_defender_api.md for Azure AD app registration, API permissions, and environment variable configuration.

Utilities include comprehensive error handling:

• Input validation
• File existence checks
• Tool availability verification
• Clear error messages with colored output

• Certificate Security: Store certificates securely and use strong passwords
• Password Handling: Passwords are prompted securely and not logged
• Binary Integrity: Always verify signatures after signing
• Access Control: Limit access to signing certificates
• API Credentials: Use .env files (gitignored) for API keys, never hardcode

1. Go not found: Ensure Go 1.21+ is installed and in PATH
2. osslsigncode not found: Install using package manager
3. Certificate not found: Check signing-certs/ directory
4. Permission denied: Ensure scripts are executable (chmod +x)
5. Build failures: Check test structure and go.mod files
6. ES sync fails: Verify .env has correct ES endpoint and API key
7. Defender query fails: Check Azure AD credentials in .env (see setup_defender_api.md)

Most utilities support --verbose or --dry-run flags for debugging.