Advanced Malware Analysis

Executable Formats and Techniques

This technical presentation explores the intricate world of malware analysis, focusing on executable file
formats and advanced analysis techniques. Designed for cybersecurity professionals and students, this
guide provides a comprehensive overview of the tools and methodologies essential for identifying,
understanding, and countering sophisticated malware threats.

preencoded.png
Presentation Agenda
1

Executable File Formats

Detailed examination of PE and ELF file structures, including headers, sections, and format-specific attributes that
malware analysts need to understand.

Malware Attack Lifecycle

Analysis of the complete attack chain from reconnaissance to actions on objectives, with emphasis on executable
behavior at each stage.

Analysis Methodologies
Comparison of static and dynamic analysis techniques, with practical applications for different malware types and anti-
analysis countermeasures.

Tools and Techniques

Overview of professional-grade tools for disassembly, debugging, and behavioral analysis, with demonstrations of their
application in real-world scenarios.

preencoded.png
PE File Structure: Windows Executables
DOS Header

First 64 bytes of the PE file, begins with magic number "MZ". Contains a pointer to the PE header and a DOS stub
program.

DOS Stub

Small DOS program that displays "This program cannot be run in DOS mode" when executed in DOS environment.
Often modified by malware authors.

NT Headers

Contains the PE signature "PE\0\0" that identifies the file as a PE format. Critical for loader validation.

COFF File Header

Defines machine type, number of sections, and timestamp. Malware often manipulates the timestamp to evade
detection.

Optional Header

Contains data directories and other essential information for the Windows loader, including entry point address.

preencoded.png

Understanding the PE (Portable Executable) file structure is fundamental for analyzing Windows-based malware. The structured format provides multiple locations where malware can hide malicious code or employ anti-analysis techniques.
PE File Sections: Code and Data Storage

.text .data
The code section containing executable instructions. Primary target Contains initialized data accessible throughout the program's
for analysis as it holds the malware's core functionality. Usually execution. Often holds hardcoded C2 addresses, encryption keys,
marked as read-only and executable. and configuration data.

Key indicators: Encrypted or heavily obfuscated code, unusual Suspicious elements: Encoded strings, unusual entropy patterns, and
instruction sequences, and API resolution techniques. hidden data structures.

.bss .symtab
Uninitialized data section (zero-initialized). Used for variables that Symbol table containing debugging information. Usually stripped in
don't have explicit initial values. Less common for hiding malicious malware to hinder analysis, but when present can reveal function
content but may contain space for decrypted payloads. names and variable information.

Malware authors frequently manipulate section properties, add custom sections, or hide code in non-standard locations. Section analysis often reveals
the first indicators of malicious intent through anomalous characteristics like high entropy in data sections or executable flags in non-code sections.

preencoded.png
Comparing Windows PE and Linux ELF Formats
Component Windows PE Linux ELF

Initial Headers DOS header with "MZ" signature ELF header with 0x7F 'ELF' signature

Structure Signature PE signature (PE\0\0) 16-byte ELF magic number

Main Headers COFF File HeaderOptional Header (Loader info) Program Header TableSection Header Table

Common Sections .text (code).data (initialized).bss (zero-initialized) .text (code).data (initialized).bss (zero-initialized)

Platform-Specific .rsrc (resources).rdata (read-only data) .plt (procedure linkage).got (global offset table)

While PE and ELF formats serve similar purposes as executable containers, they employ different structures optimized for their respective operating systems. Cross-
platform malware must account for these differences when targeting multiple environments, often implementing separate loaders for each format.

Both formats contain similar concepts of sections for segregating code and data, but implement them with different headers and loading mechanisms. Malware
analysts must be proficient in both formats to effectively respond to modern cross-platform threats.
preencoded.png
Malware Attack Lifecycle
Reconnaissance Weaponization
Gathering target intelligence. Executable artifacts include Creating exploit/payload combinations. Involves packers,
port scanners, network enumeration tools, and crypters, and obfuscation techniques to evade detection.
fingerprinting utilities. Analysis indicators: extensive Key indicators: high entropy sections, anti-disassembly
system queries and network discovery functions. tricks, and suspicious imports.

Command & Control Delivery

Communication with attacker infrastructure. Examine Transmission to victim. Common vectors include email
network traffic patterns, encryption routines, and attachments, drive-by downloads, and supply chain
protocol implementations. Look for domain generation compromises. Look for dropper behavior and multi-
algorithms and steganography. stage loading sequences.

Installation Exploitation
Establishing persistence mechanisms. Analyze registry Executing payload by exploiting vulnerabilities. Analysis
modifications, service creation, scheduled tasks, and reveals memory corruption techniques, shellcode, and
bootkit components. Often involves multiple redundant privilege escalation methods. Critical section for
persistence methods. understanding attack methodology.

Understanding this attack lifecycle enables analysts to determine where in the sequence a particular malware sample operates and what capabilities to expect. Modern
malware families often implement multiple stages of this lifecycle in separate executable components to improve stealth and resilience.

preencoded.png
Static vs. Dynamic Analysis
Static Analysis Dynamic Analysis
Examining malware without execution Executing malware in controlled environments

Benefits Benefits

• Zero risk of infection or system compromise • Reveals actual runtime behavior

• Rapid identification of indicators and patterns • Bypasses packing and obfuscation
• Complete coverage of all code paths • Captures network communications
• Detection of hidden functionality • Observes file/registry modifications

Limitations Limitations

• Ineffective against sophisticated packing • Requires secure isolation (sandbox/VM)

• Limited visibility into runtime behavior • May miss environment-aware malware
• Time-consuming for complex samples • Limited to executed code paths
• Difficult to analyze polymorphic code • Time-constrained observation window

Key Tools Key Tools

• IDA Pro/Ghidra: Disassembly and decompilation • Process Monitor: System interactions

• PEiD/PPEE: Packer identification • Wireshark: Network traffic analysis
• Strings/FLOSS: String extraction • x64dbg/OllyDbg/GDB: Runtime debugging
• YARA: Pattern matching • Cuckoo Sandbox: Automated analysis

preencoded.png
Disassembly & Debugging Techniques
Disassembly Debugging
Tools that translate raw machine code into assembly language, revealing program logic without Interactive execution of malware under debugger control to inspect behavior, memory, and
source code. execution flow.

Key Tools Key Tools

• IDA Pro: Commercial disassembler with advanced features for static analysis, pseudocode • x64dbg: Windows debugger with CISC/RISC support and extensive plugin ecosystem
generation, and scripting • OllyDbg: 32-bit assembler-level analyzing debugger optimized for binary code analysis
• Ghidra: NSA's free, open-source alternative with comparable functionality to IDA Pro • GDB: Cross-platform debugger for Linux environments with scriptable capabilities
• Radare2: Open-source framework for reverse engineering and binary analysis
Critical Debugging Capabilities
Disassembly Output Example
• Breakpoint management (hardware/software/conditional)

• Memory inspection and modification

push ebpmov ebp, espsub esp, 0x10mov eax, [ebp+8]cmp eax, 0jz .exit...
• Call stack analysis and tracing
• Anti-debugging detection and bypass

Effective malware analysis requires combining disassembly and debugging techniques. Disassembly provides the structural understanding of the code, while debugging reveals runtime behavior that may
be obscured by packing or encryption. Advanced malware often employs anti-analysis techniques specifically targeting these tools.
preencoded.png
Essential Analysis Tools

Disassemblers Debuggers
Tools that translate machine code into assembly language, revealing program logic without source Tools that provide interactive control over program execution, allowing analysts to step through
code. code and inspect memory.

• IDA Pro: Industry-standard commercial disassembler with advanced features for complex • x64dbg: Windows debugger with powerful features for both 32-bit and 64-bit malware
malware analysis • OllyDbg: 32-bit assembler-level analyzing debugger optimized for malware analysis
• Ghidra: Free, open-source tool from NSA that provides powerful decompilation capabilities • GDB: GNU Debugger for Linux-based malware analysis with extensive scripting support
• Binary Ninja: Modern disassembler with intermediate language representation for enhanced
analysis

Sandboxes Network & Behavior Monitors

Isolated environments where malware can be executed safely to observe behavior without risking Tools that capture and analyze system and network activity during malware execution.
systems.
• Wireshark: Network protocol analyzer for capturing C2 communications
• Cuckoo Sandbox: Automated malware analysis system with comprehensive behavior reporting • Process Monitor: Windows system monitor for file, registry, and process activity
• ANY.RUN: Interactive online sandbox with real-time analysis capabilities • Process Explorer: Advanced task manager for analyzing loaded modules and handles
• VMRay: Hypervisor-based detection-resistant sandbox for advanced threats

Professional malware analysis requires proficiency with multiple tool categories, often using them in concert to develop a comprehensive understanding of the sample. Each tool category provides different
insights into the malware's structure, behavior, and capabilities.

preencoded.png
Practical Malware Analysis Methodology

Initial Triage
Begin with basic static analysis to identify file type, hashes, strings, and imports. Use VirusTotal and OSINT to gather context. Estimate complexity and required analysis depth.

Controlled Execution
Run the sample in an isolated environment configured with monitoring tools. Observe file system, registry, process, and network activities. Document all behavioral indicators.

Code Analysis
Use disassemblers to examine the program's structure and logic. Identify key algorithms, decryption routines, and anti-analysis techniques. Focus on entry points and critical functions.

Interactive Debugging
Apply debuggers to step through complex or obfuscated code. Set breakpoints at suspicious API calls and examine memory for decrypted content. Bypass anti-debugging protections.

Documentation
Compile comprehensive technical documentation including capabilities, indicators of compromise (IOCs), attribution information, and defensive recommendations.

Effective malware analysis requires a methodical approach that combines multiple techniques. Start with safe static analysis, progress to controlled dynamic analysis, and then apply more advanced
techniques as needed. Always maintain proper isolation and documentation throughout the process.

Remember that modern malware often employs sophisticated anti-analysis techniques that require patience and creativity to overcome. A flexible methodology that adapts to each sample's unique
characteristics will yield the most effective results.
preencoded.png