Firmion

Firmion is a domain specific language for linking and composing of an output file. Firmion simplifies construction of complex files by managing sizes, offsets and ordering in a readable declarative style. Firmion tries to be especially useful when creating FLASH, ROM or other non-volatile memory images.

Features

Support For All Common Firmware Image Structures

• Define, compose and nest content sections to create your output
• Define platform specific memory regions to set address and size boundaries
• Write raw data, strings, and repeated values
• Write hashes and checksums
• Write offsets, addresses and sizes
• Extract and write sections from ELF and other other object file formats
• Copy external files into your output
• Align and pad content
• Output detailed map files in various formats: Rust, C header, JSON and CSV

Firmion Language Features

• Thorough reference documentation
• Comfortable curly-brace and semicolon syntax
• Declarative style so source code resembles the output file
• Include other .firm files for modularity
• Robust address and offset management with full section scope support
• Full support for arithmetic expressions
• Conditional expressions with if/else
• Compile-time interface for user-defined extensions
• Handy shorthand for numeric values, e.g. 4M is 4 x 1024 x 1024.
• Label any output location for easy reference
• Support for "-D" command-line definitions visible to your program, e.g. "-DMEM_SIZE=1M"

Debug And Diagnostic Features

• Use assert statements to provide inline validation of your program
• Use print statements for debug and or any other console messages
• Use trace statements to peek into Firmion's iterative image generation process
• Firmion provides clear error messages with full source code context
• Optional verbose debug output levels
• Firmion has hundreds of integration tests exercising features and edge cases
• Firmion is actively fuzz tested against panics

Cross-Platform Support

• Implemented in Rust with support for any Rust language host platform
• Fully open source with MIT license

Quick Start

Install Prebuilt Binaries for Linux

curl --proto '=https' --tlsv1.2 -LsSf https://github.com/steveking-gh/firmion/releases/download/7.0.2/firmion-installer.sh | sh

Install Prebuilt Binaries for Windows

Start a command prompt and execute the following:

powershell -ExecutionPolicy Bypass -c "irm https://github.com/steveking-gh/firmion/releases/download/7.0.2/firmion-installer.ps1 | iex"

Build From Source

Step 1: Install Rust

Firmion is written in rust, which works on all major operating systems. Installing rust is simple and documented in the Rust Getting Started guide.

Step 2: Clone Firmion

From a command prompt, clone Firmion and change directory to your clone. For example:

git clone https://github.com/steveking-gh/firmion.git
cd firmion

Step 3: Build and Run Self-Tests

cargo test --release --all

All tests should pass, 0 tests should fail.

Step 4: Install Firmion

The previous build step created the Firmion binary as ./target/release/firmion. You can install the Firmion binary anywhere on your system. As a convenience, cargo provides a per-user installation as $HOME/.cargo/bin/firmion.

cargo install --path ./

What Can Firmion Do?

Firmion can assemble any number of input files into a unified output.

Firmion can calculate relative or absolute offsets, allowing your output to contain pointer tables, cross-references and so on.

Firmion can add pad bytes to force parts of the file to be a certain size.

Firmion can add pad bytes to force parts of the file to start at an aligned boundary or at an absolute location.

Firmion can write your own strings and data defined within your Firmion source file.

Firmion provides full featured assert and print statement support to help with debugging complex output files.

Hello World

For a source file called hello.firm:

/*
 * A section defines part of an output.
 */
section foo {
    // Print a quoted string to the console
    print "Hello World!\n";
}

// An output statement outputs the section to a file
output foo;

Running Firmion on the file produces the expected message:

$ firmion hello.firm
Hello World!
$

Firmion also produced an empty file called output.bin. This file is the default output when you don't specify some other name on the command line with the -o option. Why is the file empty? Because nothing in our program produced output file content -- we just printed the console message.

Let's fix that. We can replace the print command with the wrs command, which is shorthand for 'write string':

/*
 * A section defines part of an output.
 */
section foo {
    // Write a quoted string to the output
    wrs "Hello World!\n";
}

// An output statement outputs the section to a file
output foo;

Now, running the command again:

$ firmion hello.firm
$

Produces output.bin containing the string Hello World!\n.

Basic Structure of a Firmion Program

A Firmion source file consists of one or more section definitions and exactly one output statement. The output statement specifies the top-level section that defines the output file. Starting from this top section, Firmion recursively evaluates each nested section and command to produce the output file. For example, we can define a section with a write-string (wrs) command:

section foo {        // Start a new section named 'foo'
    wrs "I'm foo";   // wrs writes a string into the section.
}

output foo;          // Final output

Produces a default output named output.bin.

$ cat output.bin
I'm foo

Using the wr command, sections can embed other sections:

section foo {
    wrs "I'm foo\n";
}

section bar {
    wrs "I'm bar\n";
    wr foo;           // nested section
}

output bar;

Produces output.bin:

$ cat output.bin
I'm bar
I'm foo

Users can extend Firmion with custom data processing using Firmion extensions. Users write the output of their extension call into a section with a wr.

section foo {
    wrs "I'm foo\n";
}

section bar {
    wrs "I'm bar\n";
    wr foo;           // nested section
}

section final {
    wr bar;
    wr my_stuff::crc(bar);  // Write a 4 byte CRC hash for section 'bar'.
}

assert(sizeof(final) == 20);
output final;

Assert and Print

To aid in debug, Firmion supports assert and print statements in your programs.

Assert expressions automate error checking. This example verifies our expectation that section 'bar' is 13 bytes long.

section bar {
    wrs "Hello World!\n";
    assert sizeof(bar) == 13;
}
output bar;

You can print this length information to the console during generation of your output:

section bar {
    print "Output size is ", sizeof(bar), " bytes\n";
    wrs "Hello World!\n";
    assert sizeof(bar) == 13;
}
output bar;

Prints the console message:

Output size is 13 bytes

Addresses and Offsets

Unlike the GNU linker 'ld' concept of a location counter, Firmion uses scoped addresses and scoped offsets to track locations. Addresses and offsets are 64-bit unsigned values that mark the position of the next byte of output. Firmion allows users to reference and manipulate these values, adding pad bytes as necessary.

Importantly, addresses and offsets are scoped to their enclosing section. When entering a nested (child) section, Firmion saves the outer (parent) section's inflight address and offset values. When exiting a child section, Firmion restores and updates the parent's address and offset values. From the perspective of the parent section, a child section is a wr with the parent's addresses and offsets updated per the size of the child.

For the specific case of the address and address offset, a child section inherits these values by default from the parent section. If the child section does not use set_addr, then the address and address offset simply continue growing in step with the parent.

The only global (non-scoped) offset is the file_offset. Starting from 0, this value monotonically increases to the end of the output file.

The following table provides a summary of the addresses and offsets used in Firmion.

The following diagram shows several address and offset concepts. Users specify the starting logical address of the output section D using a region. Alternatively, users can change the address within D using set_addr at the top of the output section.

Firmion Disallows Address Overwrites

By address, Firmion tracks all bytes written to the output. Firmion reports an error if a program's offset or address manipulations cause more than one write to the same address.

Firmion Disallows Negative Offset Changes

Firmion enforces that set offset commands must specify an offset change greater or equal to 0. Firmion emits pad bytes into the output for any offset change greater than 0.

Firmion Disallows Address and Offset Overflows

Firmion emits an error if an address or offset change causes 64-bit unsigned overflow. In other words, programs cannot use unsigned overflow wrapping back to 0.

Order of Execution

As a mental model, users can think of program execution as occurring in output order. In other words, an operation producing the first byte of the output will execute before an operation producing the second byte.

Within a section, output order and source code order are the same. Outside of a section, output order and source code order may differ. For example, source code may declare sections in a different order than instantiated into in the output.

For top-level statements, i.e. statements outside of a section declaration, Firmion orders execution as follows:

1. Top level statements before the output statement execute before Firmion begins producing output bytes.
2. The location of the output statement dictates the location of all output-ordered statements relative to top level statements like print and assert.
3. Top level statements after the output statement execute after Firmion produces the output file.

For example:

print "Start!";
section B {
    wr A;
    print "B1\n";
    print "B2\n";
}

section A {
    print "A1\n";
    print "A2\n";
}

print "Top1\n";
print "Top2\n";
output B;
print "Finish!";

Produces the output:

Start!
Top1
Top2
A1
A2
B1
B2
Finish!

Output Creation Phases

This section provides an overview of Firmion's internal output creation phases.

1. const Evaluation Phase: First, Firmion evaluates all const expressions. This phase includes evaluation of all if/else statements and the dependent const-time operations such as include statements in the taken path.
2. Layout Phase: Next, Firmion iteratively evaluates all expressions that affect output size and layout. For example, Firmion evaluates align expressions and extension size() calls during this phase. Firmion skips data generation, since knowing the size of operations suffices to determine the precise output structure. This phase completes when successive layout iterations produce identical results.
3. Generate Phase 1: Next, Firmion begins populating data values into the output. In this first generation phase, Firmion first evaluates wr statements that do NOT call extensions. Firmion evaluates wr calls in output order.
4. Generate Phase 2: Next, Firmion evaluates wr statement that call an extension. Like before, Firmion evaluates extension calls in output order. Firmion executes all extension calls serially on the engine thread.
5. Validation Phase: Finally, Firmion evaluates assert statements, including those that call extensions. Note that Firmion may take an early exit in any phase if an assert statement will unambiguously fail.

Command Line Options Reference

firmion [OPTIONS] <input>

The required input file contains the Firmion source code to compile and build the output file. Firmion source files typically have a .firm file extension.

When the user does not specify a path, Firmion writes map file(s) and the output to the current working directory.

Command-Line Const Defines

The -D option injects a const definition into the program from the command line. This option is modelled after the GCC -D preprocessor syntax. You can specify -D multiple times, once per each definition. For example:

firmion -DBASE=0x8000 -DCOUNT=16 -DSOME_PATH="/path/to/file" firmware.firm

The name must be a valid Firmion identifier. The value is optional and can be numeric or a quoted string. Without a value, Firmion sets the const to 1, with type Integer, following the GCC boolean-flag convention.

-D overrides any same-named const definition in the source.

Map output lists all const definitions including -D consts.

Value Type Inference

Firmion knows or infers the type from the value string using the same rules as source code for type inference.

Example

Define a base address at the command line:

firmion -DBASE=0x0800_0000 firmware.firm -o firmware.bin

The source can reference BASE as an ordinary const:

section entry { wr8 0x01; }
section top   { set_addr BASE; wr entry; }
output top;

Firmion Language Reference

Comments

Firmion supports C language line and block comments.

Whitespace

Firmion supports lenient C language style whitespace rules.

Semicolon Termination

Like C language, statements must be terminated with a trailing semicolon character.

Types

Firmion supports the following data types:

• U64: 64-bit unsigned values
• I64: 64-bit signed values
• Integer: 64-bit integers with flexible sign treatment
• String: UTF-8 string in double quotes

Firmion reports an error for under/overflow on arithmetic operations on U64, I64 and Integer types as described in Arithmetic Operators.

Identifiers

An identifier begins with a letter (A–Z, a–z) or an underscore (_), followed by zero or more letters, digits (0–9), or underscores. Identifiers are case-sensitive.

Reserved Identifiers

Firmion reserves certain identifiers and rejects their use as section names, const names, or label names at compile time.

Firmion also reserves two identifier prefixes. Any user defined identifier beginning with a reserved prefix triggers an error.

Firmion also reserves the following exact keywords:

Keyword reservation is case-sensitive. Fill and FILL are valid identifiers; fill is not.

Literals

Number Literals

Firmion supports number literals in decimal, hex (0x) and binary (0b) forms. After the first digit, you can use '_' within number literals to help with readability.

assert 42 == 42;
assert -42 == -42;
assert 0x42 == 0x42;
assert 0x42 == 66;
assert 0x4_2 == 66;
assert 0x42 == 6_6;

assert 0b0 == 0;
assert 0b01000010 == 0x42;
assert 0b0100_0010 == 0x42;
assert 0b101000010 == 0x142;
assert 0b0000_0000_0100_0010 == 0x42;

The following table summarizes how Firmion determines the type of number literals.

Firmion does not support negative hex or binary literals.

For convenience, the compiler casts the flexible Integer type to U64 or I64 as needed.

assert 42u == 42;  // U64 operates with Integer
assert 42i == 42;  // I64 operates with Integer

Otherwise the types used in an expression must match. For example:

assert 42u == 42i; // mix unsigned and signed

Produces an error message:

[ERR_137] Error: Input operand types do not match.  Left is 'U64', right is 'I64'
   ╭─[tests/integers_5.firm:2:12]
   │
 2 │     assert 42u == 42i; // mix unsigned and signed
   ·            ^^^    ^^^
───╯

Users can explicitly cast a number literal or expression to the required signedness using the built-in to_u64 to to_i64 functions. For example:

assert -42 != to_i64(42);  // comparing signed to unsigned

The to_u64 and to_i64 functions DO NOT report an error if the runtime value under/overflows the destination type.

assert 0xFFFF_FFFF_FFFF_FFFF == to_u64(-1); // OK
assert to_i64(0xFFFF_FFFF_FFFF_FFFF) == -1; // OK

Number Magnitude

Decimal number literals accept an optional K/M/G magnitude suffix (case sensitive) before the type suffix.

Magnitude and type suffixes combine: 4Ku is 4096 as a U64, -1Ki is -1024 as an I64.

True and False

Firmion considers a zero value false and all non-zero values true.

Quoted Strings

Firmion allows utf-8 quoted strings with the following escape characters:

Newlines are Linux style, so "A\n" is a two byte string on all platforms.

Arithmetic Operators

Firmion supports the following arithmetic operators with same relative precedence as the Rust language. Where applicable, Firmion checks for arithmetic under/overflow.

addr

addr( [identifier] ) -> U64

When called with an identifier, returns the address of the identifier as a U64. When called without an identifier, returns the current address. See Addresses and Offsets for more information.

The following table shows the scoping rules for addr. To summarize, Firmion tracks exactly one address value per name. An addr(<name>) command retrieves that one value regardless of the scope of the caller.

Example:

const BASE = 0x1000u;

section fiz {
    assert addr() == BASE + 6;
    wrs "fiz";
    assert addr() == BASE + 9;
    assert addr(foo) == BASE;
}

section bar {
    assert addr() == BASE + 3;
    wrs "bar";
    assert addr() == BASE + 6;
    wr fiz;
    assert addr() == BASE + 9;
}

// top level section
section foo {
    set_addr BASE;
    assert addr() == BASE;
    wrs "foo";
    assert addr() == BASE + 3;
    assert addr(fiz) == BASE + 6;
    wr bar;
    assert addr() == BASE + 9;
    assert addr(bar) == BASE + 3;
}

output foo;

addr_offset

addr_offset( [identifier] ) -> U64

Returns the offset from the output or most recent set_addr anchor as a U64. When called without an identifier, returns the current address offset. When called with an identifier, returns the address offset at the start of the named section or label.

The offset resets to zero on each set_addr call.

The following table shows the scoping rules for addr_offset. To summarize, Firmion tracks exactly one address offset value per name. An addr_offset(<name>) command retrieves that one value regardless of the scope of the caller.

Example:

const BASE = 0x1000u;

section fiz {
    assert addr_offset() == 6;
    wrs "fiz";
    assert addr_offset() == 9;
    assert addr_offset(foo) == 0;
}

section bar {
    assert addr_offset() == 3;
    wrs "bar";
    assert addr_offset() == 6;
    wr fiz;
    assert addr_offset() == 9;
}

// top level section
section foo {
    set_addr BASE;
    assert addr_offset() == 0;
    wrs "foo";
    assert addr_offset() == 3;
    assert addr_offset(fiz) == 6;
    wr bar;
    assert addr_offset() == 9;
    assert addr_offset(bar) == 3;
}

output foo;

align

align <expression> [, <pad byte value>];

The align statement writes pad bytes into the current section until the absolute location counter reaches the specified alignment. Align writes 0 as the default pad byte value, but the user may optionally specify a different value.

Example:

section foo {
    wrs "Hello";
    align 32;
    assert sizeof(foo) == 32;
    assert addr() == 32;
}

output foo;

assert

assert <expression>;

The assert statement reports an error if the specified expression does not evaluate to a true (non-zero) value. Assert expressions provide a means of error checking and do not affect the output file.

Example:

section foo {
    assert 1;   // OK, non-zero is true
    assert -1;  // OK, non-zero is true
    assert 1 + 1 == 2;
}

output foo;

const

const <identifier> = <expr>;

A const expression creates an immutable user defined identifier for a value. The value can consist of a number or string literal, or an expression composed of other constants and literals. Const identifier names have global scope and must be globally unique. Const identifiers cannot conflict with any other global identifiers such as section names.

Example:

const RAM_BASE = 0x8000_0000u;  // User defined unsigned constant.

section foo {
    set_addr RAM_BASE;
    wr64 RAM_BASE;
    print "RAM base address is ", RAM_BASE, "\n";
}

output foo;

Const expressions support the full set of arithmetic, bitwise and comparison operators. Comparison operators evaluate to 1 (true) or 0 (false) and are useful for expressing relationships between constants:

const FLASH_BASE = 0x0800_0000;
const FLASH_SIZE = 0x0008_0000;
const RAM_BASE   = 0x2000_0000;

// Verify flash and RAM regions do not overlap
const NO_OVERLAP = (FLASH_BASE + FLASH_SIZE) <= RAM_BASE;
assert NO_OVERLAP;

A const value expression cannot depend on addresses, sizes, offsets or any other dynamic aspect of the output file. Firmion resolves all const values before starting layout of the output. For example:

const RAM_BASE = 0x8000_0000;         // OK, just a 64b unsigned literal.
const RAM_SIZE = 32768;               // OK, just a 64b integer literal.
const RAM_END = RAM_BASE + RAM_SIZE;  // OK, const composed of other consts.

section foo {
    wrs "Hello\n";
}

const RAM_USED = sizeof(foo);         // ERROR!  Const cannot depend on section properties.

output foo;

Deferred Assignment

const variables support deferred assignment. This allows the user to declare a const variable, then assign a value to the variable exactly once in later code. For example:

const IO_START;
...
IO_START = 0xF000_0000_0000_0000;

Deferred assignment is primarily useful in if/else statements, which allow users to conditionally determine the value to assign.

To provide errors and warnings, Firmion tracks the defined/undefined and used/unused state of each variable.

if/else

if <expression> { ... } else { ... }

Allows conditional execution of other statements. As described in Output Creation, Firmion evaluates all if/else statements before starting layout of the output. Therefore, an if/else expression must only depend on const variables and literal values. In other words, if/else statements must not depend on dynamic addresses, sizes, offsets or any other layout dependent aspect of the output file.

Users must pre-declare const variables before conditionally assigning values to them. For example:

// Assume the user specified -DMEM_CONFIG="BIG" on the command line.

// Pre-declare variables prior to conditional assignment in an if/else.
// Firmion strictly tracks variable definitions to prevent use of
// uninitialized variables.
const FLASH_SIZE;
const RAM_SIZE;

print "Memory configuration is ", MEM_CONFIG, "\n";
if MEM_CONFIG == "BIG" {
    FLASH_SIZE = 0x8_0000;
    RAM_SIZE = 0x80_0000;
    include "big_config.firm";
} else {
    if MEM_CONFIG == "MEDIUM" {
        FLASH_SIZE = 0x4_0000;
        RAM_SIZE = 0x40_0000;
        include "medium_config.firm";
    } else {
        if MEM_CONFIG == "SMALL" {
            FLASH_SIZE = 0x2_0000;
            RAM_SIZE = 0x20_0000;
            include "small_config.firm";
        } else {
            print "Invalid configuration. MEM_CONFIG must be BIG, MEDIUM, or SMALL.\n";
            assert(0);  // Halt execution
        }
    }
}

If the taken path in an if/else statement does not assign a value to a predeclared const variable, then Firmion reports an error if any later program statement uses that variable.

For compactness, user's may omit braces around an else/if block. For example:

if MEM_CONFIG == "BIG" { include "big_config.firm"; }
else if MEM_CONFIG == "MEDIUM" { include "medium_config.firm"; }
else if MEM_CONFIG == "SMALL" { include "small_config.firm"; }
else { assert(0); }

include

include "<file>";

Includes another Firmion source file. Firmion processes the included file as if it were part of the current file. For example, the included file can define sections, labels, constants and nested include files.

An included file may contain an output statement. Firmion will enforce that the entire program after include file resolution contains only one output statement. See the output statement for more information.

The default path for an included file is the directory of the source file that contains the include statement. For example, if main.firm is in /home/user/project/ and contains include "sections.firm", then Firmion will look for /home/user/project/sections.firm.

Include files starting with a / are absolute paths. Likewise, Firmion supports relative paths such as ../.

All paths use Linux style forward slashes.

Example:

// file: main.firm
include "../constants.firm";
include "sections.firm";

output main_rom;

// file: ../constants.firm
const RAM_BASE = 0x8000_0000u;

// file: sections.firm
section main_rom {
    set_addr 0x1000;
    wrs "Hello\n";
}

Labels

<identifier>:

Labels assign an identifier to a specific location in the output file. Programs can then refer to the location of the label by name. Labels names have global scope and label names must be globally unique. Multiple different labels can refer to the same location.

Labels have the form <label identifier>: and can prefix most statement types.

For example:

section foo {
    set_addr 0x1000;
    // assign the label 'lab1' to the current location
    lab1: wrs "Wow!";
    // assign the label 'lab2' to the current location
    lab2:
    assert addr(lab1) == 0x1000;
    assert addr(lab2) == 0x1004;
    assert addr(lab3) == 0x1004;
    // yet another label, same location as 'lab2'
    lab3:
}

output foo;

obj

obj <obj name> { ... }

An obj statement assigns a name to a specific section in an external object or executable file. The section in this case is not a Firmion section, but a linker section such as ".text" or ".rodata" created by an external compiler toolchain, e.g. gcc.

By default, Firmion supports only the ELF format. However, using compile-time feature flags, users can enable support for any object file format supported by the Rust object crate and rebuild Firmion from source.

For example:

obj runtime_code {
    file = "/path/to/exe";
    section = ".text";
}

obj runtime_rodata {
    file = "/path/to/exe";
    section = ".rodata";
}

section main {
    wr runtime_code;
    wr runtime_rodata;
}

output main;

Obj Properties

Obj definitions support the following properties:

• file Path to the object or executable file (required)
• section Name of the section in the object file (required)

Obj Property file

Path to the object or executable file. Firmion uses the same path resolution as the wrf command. Namely, paths can relative to the current directory or absolute.

Obj Property section

Name of the linker section in the object file. The name must include the full string value recorded in the object or executable file, including any leading characters such as the "." in ".text".

Obj Size

The external object file sets size of an obj. Users can query the size with sizeof. For example:

obj runtime_rodata {
    file = "/path/to/exe";
    section = ".rodata";
}

section foo {
    print "The size of read-only data is ", sizeof(runtime_rodata), " bytes\n";
    wr64 sizeof(runtime_rodata);
    wr runtime_rodata;
}

Obj LMA and VMA

Some file formats, notably ELF, define a load memory address (LMA) for a section. The LMA specifies where a system stores a section at rest. The virtual memory address (VMA) specifies the runtime address of a section. These addresses differ when a system copies the section before use, e.g. copies program code from slow FLASH memory (at the LMA) to fast SRAM (at the VMA) before execution.

Users can query the ELF LMA and VMA of an obj using the obj_lma and obj_vma commands.

Firmion addr vs LMA and VMA

Users fully control their output files and can use Firmion's address support (addr, set_addr, region, etc) as they see fit. For most systems however, Firmion's addr plays the same role as the ELF LMA. The Firmion addr value is usually the section's storage address as might be used by a FLASH update utility. Any subsequent copy at runtime is typically outside the scope of Firmion.

obj_align

obj_align(<obj>) -> U64

Returns the alignment of the specified obj as a U64. The external object file defines this value as set by a compiler toolchain.

For example:

obj runtime_rodata {
    file = "/path/to/exe";
    section = ".rodata";
}

section foo {
    align obj_align(runtime_rodata);
    wr runtime_rodata;
}

obj_lma

obj_lma(<obj>) -> U64

Returns the load memory address (LMA) of the specified obj as a U64. The external object file defines this value as set by a compiler toolchain. See LMA and VMA for more information.

For example:

obj runtime_rodata {
    file = "/path/to/exe";
    section = ".rodata";
}

section foo {
    print "The load address is ", obj_lma(runtime_rodata), "\n";
    // Keep the object file load address as-is.
    set_addr(obj_lma(runtime_rodata));
    wr runtime_rodata;
}

Some non-ELF file formats do not support an LMA different from a VMA. If the user sets feature flags to enable additional object file formats and calls obj_lma on an unsupported format, then Firmion simply returns the obj's VMA value. In this way, a user can consistently use obj_lma for all formats.

obj_vma

obj_vma(<obj>) -> U64

Returns the virtual memory address (VMA) of the specified obj as a U64. The external object file defines this value as set by a compiler toolchain. See LMA and VMA for more information.

For example:

obj runtime_rodata {
    file = "/path/to/exe";
    section = ".rodata";
}

section foo {
    set_addr(obj_lma(runtime_rodata));
    // Our runtime doesn't support relocation.
    assert obj_vma(runtime_rodata) == obj_lma(runtime_rodata);
    wr runtime_rodata;
}

output

output <section identifier>;

An output statement specifies the top section to write to the output file. Use set_addr inside the section to control the absolute starting address, or place the top section in region with a start address.

A Firmion program must have exactly one output statement.

An include file may contain an output statement. Firmion will enforce that the entire program after include file resolution contains only one output statement.

print

print <expression> [, <expression>, ...];

The print statement evaluates the comma separated list of expressions and prints them to the console. For expressions, print displays unsigned values in hex and signed values in decimal. If needed, the to_u64 and to_i64 functions can control the output style.

Firmion executes a given print statement for each instance found in the output file. In other words, a print statement in a section written multiple times will execute multiple times in output order.

Example:

const BASE = 0x1000;

section bar {
    print "Section 'bar' starts at ", addr(), "\n";
    wrs "bar";
}

// top level section
section foo {
    set_addr BASE;
    print "Output spans address range ", BASE, "-", BASE + sizeof(foo),
          " (", to_i64(sizeof(foo)), " bytes)\n";
    wrs "foo";
    wr bar;
    wr bar;
    wr bar;
}

output foo;

Results in the following console output:

Output spans address range 0x1000-0x100C (12 bytes)
Section 'bar' starts at 0x1003
Section 'bar' starts at 0x1006
Section 'bar' starts at 0x1009

region

region <identifier> { ... }

A region declares the name and static properties of an address range. Regions provide a way to decouple memory placement and top-down layout control from the section content being placed. Unlike sections, regions are stateless and do not track dynamic information during layout.

Users place exactly one section in a region. We refer to this section as the bound section of the region. The bound section is a normal section with the following extra behaviors:

• The region sets the starting address of the bound section.
• The region caps the size of the bound section.

For example:

// Define the properties of the FLASH memory region
region FLASH {
    addr = 0xF000_0000;
    size = 1M;
}

// Define the properties of the EEPROM memory region
region EEPROM {
    addr = 0xFF00_0000;
    size = 64K;
}

// Flash sections
section boot { ... }
section flash_code { ... }
section flash_data { ... }

// EEPROM sections
section eeprom_data1 { ... }
section eeprom_data2 { ... }

// FLASH_TOP is the bound section in the FLASH region
section FLASH_TOP in FLASH {
    // Starts at address 0xF000_0000
    wr boot;
    wr flash_code;
    wr flash_data;
}

section EEPROM_TOP in EEPROM {
    assert addr() == 0xFF00_0000;
    wr runtime_code;
    wr runtime_data;
}

// The output file contains the image for FLASH and EEPROM regions.
// This section is not a bound section of a region and behaves
// like any other section.
section FIRMWARE_UPDATE_FILE {
    wr file_offset(FLASH_TOP);    // Offset to the new FLASH image
    wr file_offset(EEPROM_TOP);   // Offset to the EEPROM image
    wr FLASH_TOP;                 // FLASH image
    wr EEPROM_TOP;                // EEPROM image
}

output FIRMWARE_UPDATE_FILE;  // Write the output

Region Properties

Regions support the following properties:

• addr Starting address (required)
• size Size in bytes (required)

Region Property addr

The addr property defines the region's absolute starting address. The region's bound section starts at this address. Users can query the addr property of a region with addr(<region name>).

Region Property size

Specifies the size of the region in bytes. Firmion reports an error if the size of the bound section exceeds this value.

Users can query the size property of a region with sizeof(<region name>).

The size value accepts a K/M/G magnitude suffix.

Region Boundary Enforcement

Regions provide automatic size and boundary checking for all operations in the region. In practical terms this means:

• Write commands in a region cannot extend outside the region
• Address manipulation in a region cannot result in an address outside the region
• Offset manipulations in a region cannot result in a offset outside the region.

For example, the following wr32 command would extend outside the region by one byte, resulting in an error:

region LITTLE_ROM {
    addr = 0;
    size = 7;
}

section data in LITTLE_ROM {
    // occupies bytes 0-3
    wr32 0x12345678;
    // Occupies bytes 4-7, which extends 1 byte outside the region
    wr32 0x87654321;  // ERROR!
}

Of course, region enforcement occurs not just in the region's bound section, but in any reachable section. For example:

region LITTLE_ROM {
    addr = 0;
    size = 7;
}

section nested_stuff {
    pad_sec_offset 6;  // pad to last byte of region
    wr more_nested;
}

section more_nested {
    wr std::crc32c(more_nested);  // 4 bytes of output
}

section data in LITTLE_ROM {
    wr nested_stuff;  // ERROR!  Data written outside of region
}

The set_addr command and any offset manipulation commands are also constrained to fit in the region. For example:

region FLASH {
    addr = 0xF000_0000;
    size = 1M;
}

section foo in FLASH {
    assert addr() == 0xF000_0000;  // Start of FLASH region
    set_addr 0xF000_1000;          // OK, inside the region
    wrs "Inside region!";
    set_addr 0xA000_0000;          // ERROR, outside the region
    wrs "Outside region!";
}

output foo;

Nested Regions

Users can freely nest sections in different regions into each other. However, Firmion allows write operations only in the address range intersection permitted by all the parent regions. For example:

region READ_ONLY {
    addr = 0xF000_0000;
    size = 0x1000_0000;
}

region FLASH {
    addr = 0xF100_0000;
    size = 64K;
}

section flash_data in FLASH {
    assert addr() == 0xF100_0000;
    ...
}

section ro_data in READ_ONLY {
    assert addr() == 0xF000_0000;
    // OK, region FLASH is a subset of READ_ONLY.
    // Note that the FLASH region anchors the starting address
    // at 0xF100_0000.  This creates a logical (unpadded) address gap
    // in the ro_data section between 0xF000_0000 and 0xF100_0000.
    wr flash_data;
}

output ro_data;

Partially Overlapping Nested Regions

For completeness, the region of a nested section need not be a proper subset of the parent region. Firmion still enforces the constraints of all parent sections as follows:

• Any address written by the child section must lie in the intersection of all parent regions.
• The starting address of a nested section must fit the address range allowed by all parent regions.

Sections in Regions are Single Use

Placing a section in a region forces the starting address of the section to the region's addr value. Writing this section more than once results in an address address conflict with the previous instance of the section.

sec_offset

sec_offset( [identifier] ) -> U64

When called with an identifier, returns the unsigned 64-bit offset of the identifier from the start of the section that contains the identifier. When called without an identifier, returns the offset from the start of the current section.

Example:

section fiz {
    assert sec_offset() == 0;
    wrs "fiz";
    assert sec_offset() == 3;
}

section bar {
    assert sec_offset() == 0;
    wrs "bar";
    assert sec_offset() == 3;
    wr fiz;
    assert sec_offset() == 6;
    assert sec_offset(fiz) == 3;
}

// top level section
section foo {
    assert sec_offset() == 0;
    wrs "foo";
    assert sec_offset() == 3;
    wr bar;
    assert sec_offset() == 9;
}

output foo;

When a section offset specifies an identifier, the identifier must be in the scope of the current section. For example:

section fiz {
    wrs "fiz";
}

section bar {
    wr fiz;
    assert sec_offset(fiz) == 0; // OK fiz in scope in section bar
}

section foo {
    wr bar;
    assert sec_offset(bar) == 0; // OK, bar is local in this section
    assert sec_offset(fiz) == 0; // ERROR, fiz is out of scope in section foo
}

output foo;

section

section <name> [in <region>] { ... }

A section is a named, reusable block of content. Sections are the primary building block of a Firmion program. Each section defines a sequence of bytes, built up from write statements and padding operations such as align. Sections may also contain labels, assertions, print statements and so on. Sections may write other sections into themselves so long as the nesting does not create a cycle.

Section names must be valid identifiers, must be globally unique, and must not conflict with const names, label names, region name, or reserved identifiers.

Sections have their own section-relative location counter which resets to zero at the start of each section. Sections can read and advance the section location counter with sec_offset() and pad_sec_offset() statements respectively.

The root section named in the output statement is the only section Firmion writes to the output file. Other sections can be directly or indirectly included via wr statements from the output section. Unreachable sections produce a warning.

Sections In Regions

To help guide layout, users can place a exactly one section in a region with in <region name> after the section name. We call a section placed in a region as the bound section of the region.

Example:

section magic {
    wrs "FIRM";           // 4-byte magic number
    wr8 0x01;             // version
    assert sec_offset() == 5;    // Section location counter should be 5
}

section body {
    wr8 0xAA, 16;         // 16 bytes of payload
}

section image {
    wr magic;
    align 256;            // Body should start on 256 byte boundary
    wr body;
    assert sizeof(image) == 272;  // 256 + 16
}

output image;

set_addr

set_addr <expression>;

The set_addr command forces the current address to the specified value and resets the current addr_offset to zero. These changes happen within the scope of the containing section. Child sections inherit the parent section's addr and addr_offset values.

Using set_addr does not change the value of the section offset nor file offset. A set_addr command does not add pad bytes to the output.

The set_addr command may move the address forward or backwards. However, Firmion tracks every output byte by address and reports an error if a program tries to write to the same address more than once.

Example:

section foo {
    wr8 1;
    wr8 2;
    wr8 3;
    wr8 4;
    wr8 5;
    set_addr 16;
    assert addr() == 16;
    assert addr_offset() == 0;   // set_addr resets addr_offset
    assert file_offset() == 5;  // set_addr does not pad
    assert sec_offset() == 5;
    wr8 0xAA, 3;
    assert addr_offset() == 3;
    assert file_offset() == 8;
    assert sec_offset() == 8;
    pad_sec_offset 24, 0xFF;     // Adds 24 - 8 = 16 pad bytes
    assert addr() == 35;         // 19 + 16 = 35
    assert addr_offset() == 19;  // 3 + 16 = 19
    assert file_offset() == 24;  // 8 + 16 = 24
    assert sec_offset() == 24;   // 8 + 16 = 24
}

output foo;

When used in a section in a region, Firmion reports an error if the set_addr command sets the address outside of a region.

pad_addr_offset

pad_addr_offset <expression> [, <pad byte value>];

Pads the output until addr_offset reaches the specified value. Users may specify an optional pad byte value or use the default value of 0.

If the specified value is less than the current addr_offset, Firmion reports an error.

pad_addr_offset is most useful after a set_addr call, because set_addr resets addr_offset to zero. This lets users pad to a size relative to their chosen address anchor without knowing what the surrounding section's sec_offset happens to be.

Example:

const BASE = 0x1000;

section header {
    wrs "FIRM";           // 4-byte magic number
    wr8 0x01;             // version byte
}                         // addr_offset == 5 on exit

section body {
    set_addr BASE;
    wr header;
    // Relocate body to its target load address.
    // addr_offset resets to 0.
    set_addr 0xF000;
    wr8 0xAA, 3;          // 3 bytes of payload
    // Pad to 0x20 bytes from the 0xF000 anchor.
    pad_addr_offset 0x20;
    assert addr() == 0xF020;
    assert addr_offset() == 0x20;
    assert sec_offset() == 0x25;  // 5 (header) + 3 (payload) + 29 (pad) = 0x25
}

output body;

pad_file_offset

pad_file_offset <expression> [, <pad byte value>];

The pad_file_offset command pads the output file until the file offset reaches the specified value. Users may specify an optional pad byte value or use the default value of 0.

If the specified offset is less the current offset, Firmion reports an error.

pad_file_offset is most useful when a section is written inside a parent section, because sec_offset resets to zero at the start of each child section while file_offset continues from the parent's position. This lets a child section pad to an absolute file position regardless of where the parent places it.

Example:

// A firmware container: an 8-byte header at file offset 0, followed by a
// payload that must start at file offset 512 for bootloader compatibility.

section header {
    wrs "FIRM";       // 4-byte magic
    wr32 0x00000001;  // version
}

section payload {
    // firmware writes header first (8 bytes), so payload opens at
    // file_offset 8.  Pad to the protocol-required file position 512.
    pad_file_offset 512, 0xFF;
    assert file_offset() == 512; // absolute position in the output file
    assert sec_offset() == 504;  // sec_offset starts from 0 inside payload
    wrs "PAYLOAD";               // 7 bytes of payload data
    assert file_offset() == 519;
    assert sec_offset() == 511;
}

section firmware {
    wr header;
    wr payload;
}

output firmware;

pad_sec_offset

pad_sec_offset <expression> [, <pad byte value>];

The pad_sec_offset command pads the current section until the section offset reaches the specified value. Users may specify an optional pad byte value or use the default value of 0.

If the specified offset is less the current offset, Firmion reports an error.

Example:

section foo {
    wr8 1;
    wr8 2;
    wr8 3;
    wr8 4;
    wr8 5;
    pad_sec_offset 16;
    assert addr() == 16;
    assert file_offset() == 16;
    assert sec_offset() == 16;
    wr8 0xAA, 3;
    pad_sec_offset 24, 0xFF;
    assert addr() == 24;
    assert file_offset() == 24;
    assert sec_offset() == 24;
    pad_sec_offset 24, 0xEE; // should do Nothing
    wr8 0xAA, 3;
    pad_sec_offset 27, 0x33; // should do nothing
    pad_sec_offset 28, 0x77; // should pad to 28
    assert sizeof(foo) == 28;
}

output foo;

sizeof

sizeof( <identifier> ) -> U64

Returns the size in bytes of the specified identifier.

Example:

section empty_one {}
section foo {
    wrs "Wow!";
    wr empty_one;
    assert sizeof(empty_one) == 0;
    assert sizeof(foo) == 4;
}

output foo;

When called with an extension identifier, sizeof returns the size of the extension's output. For example:

print "CRC size=", sizeof(std::crc32c);  // returns "CRC size=4"

When called with a region identifier, sizeof returns the fixed size of the region regardless of whether the user's program writes any data in the region.

region FLASH { ...; size = 8K; ... }
...
print "FLASH size=", sizeof(FLASH);  // returns "FLASH size=8192"

When called with a section identifier, sizeof returns the size of the section in the file. Therefore, this size does not take into account operations that do not write data nor pad bytes. For example, address jumps, e.g. by using set_addr do not change the sizeof() result for a section.

section foo {
    set_addr 0;
    wrs "Hello\n";
    // Address jumps by 0x1000, but no data nor pads written, so
    // no effect on sizeof(foo).
    set_addr 0x1000;
    wrs "World\n";
    assert sizeof(foo) == 12;
}

to_i64

to_i64( <expression> ) -> I64

Converts the specified expression to the I64 type without regard to under/overflow.

Example:

section foo {
    assert to_i64(0xFFFF_FFFF_FFFF_FFFF) == -1;
    assert to_i64(42u) == 42;
    assert to_i64(42u) == 42i;
    assert to_i64(42) == 42i;
}

output foo;

to_u64

to_u64( <expression> ) -> U64

Converts the specified expression to the U64 type without regard to under/overflow.

Example:

section foo {
    assert 0xFFFF_FFFF_FFFF_FFFF == to_u64(-1);
    assert to_u64(42i) == 42;
    assert to_u64(42i) == 42u;
    assert to_u64(42) == 42u;
}

output foo;

trace

trace <expression> [, <expression>, ...];

The trace command provides debug output to help diagnose errors before Firmion is able to internally execute a program. Trace is especially useful for errors reported during layout phase operations before Firmion determines the size and location of everything in the output file. During this time, Firmion is actively cooking the output, so layout depend values revealed by trace may change as iteration proceeds.

The trace command has the same form and capability as the print command. However, trace executes on every internal iteration pass as Firmion tries to stabilize the output's layout.

For example:

trace "Start!\n";
section B {
    trace "Size of B is ", sizeof(B), "\n";
    wr A;
    wr8 0xBB;
    trace "B1\n";
}

section A {
    wr8 0xAA;
    trace "A1\n";
}

trace "Top1\n";
output B;
trace "Finish!\n";

The program above might produce an output like the following, the caveat being that Firmion purposely does not rigorously define trace output:

[Trace-1] Start!
[Trace-1] Top1
[Trace-1] Size of B is 0x0
[Trace-1] A1
[Trace-1] B1
[Trace-1] Finish!
[Trace-2] Start!
[Trace-2] Top1
[Trace-2] Size of B is 0x2
[Trace-2] A1
[Trace-2] B1
[Trace-2] Finish!
[Trace-2] Start!
[Trace-2] Top1
[Trace-2] Size of B is 0x2
[Trace-2] A1
[Trace-2] B1
[Trace-2] Finish!

The number in Trace-n is the internal iteration count. In this example, Firmion required two iterations to stabilize the output.

Transient Values in trace Output

Because the trace command provides a peek into Firmion's internal workings, displayed values may appear erroneous. In particular, transient values (often equal to 0) can cause arithmetic expressions to also return bogus results. During image generation, Firmion deals with bogus values by suppressing certain types of errors until the image stabilizes. When suppressing an error, Firmion substitutes 0 for the bogus value. As Firmion iterates, "good" values propagate and eventually eliminate these suppressed errors.

In summary, treat trace values as hints. As iteration continues, trace values should converge on the actual value, i.e. what print would display.

wr

The wr command has two forms. The first form writes the contents of another section into the current section. The second wr form invokes an extension and writes the output into the current section.

wr section

wr <section identifier>;

Firmion adds the specified in section to the current section at the current section offset.

wr extension

wr <namespace>::<extension_name>(<arg1>, <arg2>, ...);

Evaluates the specified extension call and writes the result to the output. The extension's .size() method specifies the size of the result. See Firmion Extensions for more information.

Example

Using wr, you can build complex outputs by composing smaller, modular sections together.

Example:

section header {
    wrs "FILE";   // Write a string.
    wr8 0x01;     // Write a byte.
}

section data {
    wrs "DATA";
    wr8 0xFF, 4;
}

// Compose the top-level section
section my_firmware {
    wr header;
    wr data;
    // Use an extension to append a CRC to a section.
    // Extensions can refer to their containing section.
    wr std::crc32c(my_firmware);
}

output my_firmware;

wr8 to wr64

Little-endian:

wr8 <expression> [, <expression>]; wr16 <expression> [, <expression>]; wr24 <expression> [, <expression>]; wr32 <expression> [, <expression>]; wr40 <expression> [, <expression>]; wr48 <expression> [, <expression>]; wr56 <expression> [, <expression>]; wr64 <expression> [, <expression>];

Big-endian:

wrbe8 <expression> [, <expression>]; wrbe16 <expression> [, <expression>]; wrbe24 <expression> [, <expression>]; wrbe32 <expression> [, <expression>]; wrbe40 <expression> [, <expression>]; wrbe48 <expression> [, <expression>]; wrbe56 <expression> [, <expression>]; wrbe64 <expression> [, <expression>];

Evaluates the first expression and writes the result as a little-endian value to the output file, or as a big-endian value for the be form. The optional second expression specifies the repetition count.

Example:

// Test expressions in wrx; addr(foo) == 10 as set by region TOP
region TOP { addr = 10; size = 64K; }
section foo in TOP {
    wr8  (1 + 2) + file_offset() + addr(foo) + sizeof(foo); // 3 + 0 + 10 + 36  = 49
    wr16 (1 + 2) + file_offset() + addr(foo) + sizeof(foo); // 3 + 1 + 10 + 36  = 50 00
    wr24 (1 + 2) + file_offset() + addr(foo) + sizeof(foo); // 3 + 3 + 10 + 36  = 52 00 00
    wr32 (1 + 2) + file_offset() + addr(foo) + sizeof(foo); // 3 + 6 + 10 + 36  = 55 00 00 00
    wr40 (1 + 2) + file_offset() + addr(foo) + sizeof(foo); // 3 + 10 + 10 + 36 = 59 00 00 00 00
    wr48 (1 + 2) + file_offset() + addr(foo) + sizeof(foo); // 3 + 15 + 10 + 36 = 64 00 00 00 00 00
    wr56 (1 + 2) + file_offset() + addr(foo) + sizeof(foo); // 3 + 21 + 10 + 36 = 70 00 00 00 00 00 00
    wr64 (1 + 2) + file_offset() + addr(foo) + sizeof(foo); // 3 + 28 + 10 + 36 = 77 00 00 00 00 00 00 00
    assert sizeof(foo) == 36;
}

output foo;

Another example using the optional repetition expression.

section foo {
    wr32 0x12345678, 10; // write 0x12345678 10 times to the output file.
    wr8 0, addr() % 4096; // write zero enough times to align to 4KB boundary.
}

wrf "<quoted file path>";

Write the file at the specified path into the output file. Firmion treats all input files as binary files. Paths can be relative to the current directory or absolute.

For example, given the file test_source_1.txt containing:

Hello!

The following program simply copies these 6 UTF-8 characters to the output file.

section foo {
    wrf "test_source_1.txt"; // Hello!
    assert sizeof(foo) == 6;
}

output foo;

wrs <expression> [, <expression>, ...];

Evaluates the comma separated list of expressions and writes the resulting string to the output file. Wrs accepts the same expressions and operates similarly to the print statement. For more information, see print.

The wrs statement does not implicitly write a terminating 0 byte after the string. Users creating null terminated (C style) strings in an output file should add an explicit \0.

wrs "my null terminated string\0";

Built-in Variables

Firmion pre-defines built-in identifiers that begin with __ (double underscore). They can appear in any expression context that accepts the corresponding type. As shown in the table below, some builtins cannot be used in const expressions because their values depend on dynamic layout values.

__OUTPUT_SIZE

Returns the total size of the output file in bytes.

Example — write a 4-byte header field containing the total output size:

section payload {
    wrs "Hello";
}

section hdr {
    wr32 __OUTPUT_SIZE;  // filled with total image size at link time
}

section image {
    wr hdr;
    wr payload;
    assert __OUTPUT_SIZE == sizeof(image);  // equivalent forms
}

output image;

__OUTPUT_ADDR

Returns the absolute starting address of the output section. Equivalent to addr(<output-section>). Without placing the section in region, __OUTPUT_ADDR is zero regardless of set_addr command internal to the output section. This occurs because set_addr is a scoped operation. A set_addr within a section affects address calculations for subsequent writes internal to the section, not the logical start of the section itself.

If user places the output section in a region, then __OUTPUT_ADDR is the starting address of the region.

Example — embed the output base address in a table:

region TOP { addr = 0x0800_0000; size = 64K; }

section vtable {
    wr32 __OUTPUT_ADDR;  // base address of the output image
}

section code {
    wrs "code";
}

section image in TOP {
    wr vtable;
    wr code;
    assert __OUTPUT_ADDR == addr(image);  // equivalent expressions
}

output image;

__FIRMION_VERSION_STRING

Returns the Firmion tool version as a string (e.g. "4.0.0"). The value is fixed at compile time and may be used in const expressions, wrs, and print.

Example — stamp the tool version into a firmware header:

section hdr {
    wrs __FIRMION_VERSION_STRING;
}

section image {
    wr hdr;
    wrs "payload";
}

output image;

__FIRMION_VERSION_MAJOR, __FIRMION_VERSION_MINOR, __FIRMION_VERSION_PATCH

Return the individual numeric components of the Firmion version as U64 values. All three are fixed at compile time and may be used in const expressions and arithmetic.

Example — pack the version into a 3-byte field and assert the tool is new enough:

const MIN_MAJOR = 4u;

section hdr {
    assert __FIRMION_VERSION_MAJOR >= MIN_MAJOR;
    wr8 __FIRMION_VERSION_MAJOR;
    wr8 __FIRMION_VERSION_MINOR;
    wr8 __FIRMION_VERSION_PATCH;
}

section image {
    wr hdr;
    wrs "payload";
}

output image;

Firmion Extensions

Firmion supports compile time extensions to simplify the addition of new functionality. This extension capability enables user defined hashing, compression, validation and other binary data processing tasks. The following sections describe how extensions work and how to create them.

The command line option --list-extensions outputs the names of all available extensions as enabled by Cargo feature flags. The following table shows the currently support standard extensions.

Extensions Are A Compile-Time Feature

Extensions build and link to Firmion at compile time as controlled by Cargo feature flags. Because Rust does not guarantee a stable ABI between versions, Firmion requires compile time construction to eliminate ABI incompatibilities and enable the use of safe Rust. The following bullets provide an overview of how extensions work:

• Extensions interact with Firmion through the FirmionExtension trait.

• Extensions can read directly from the output buffer for a specified section via zero-copy and safe-memory slices (&[u8]).

• In addition to output buffer access, extensions can have their own input parameters like a normal function call.

• Extensions are identified by a name in a namespace. Firmion reserves the namespaces std and firmion.

• Extensions report their fixed length binary footprint by implementing the .size() trait method. Firmion calls each extension's .size() method exactly once during output layout calculations and caches the result. Firmion always passes a mutable output slice (&mut [u8]) of the reported size to the extension's .generate() method.

• Extensions register themselves at compile time in Firmion's internal extension registry.

• The FirmionExtension trait interface allows extensions to return logging and error diagnostics integrated with Firmion's own diagnostic output. See []

Invoking Extensions

Users invoke extensions using function-style syntax. Users creating their own extension can take any number of parameters of any Firmion support type:

turbo::boost("Big", 1, -42, 0x12345678);

Fixed-size write commands like wr32 are invalid for extensions. If the designer needs to pad the extension's output to a specific size, they must follow the wr command with a pad_sec_offset or align statement.

Passing Section Data to Extensions

Users can pass the data in a section to an extension by passing the section name as a parameter. Extensions take section data as an immutable zero-copy slice parameter of Rust type &[u8]. Section data passed to the extension at the time of the call includes all data generated by non-extension write commands. Furthermore, the data includes the output of extensions executed before the current extension.

As an example, consider the std::crc32c extension. This extension generates a CRC hash over the data provided by the specified section. The extension produces a 4-byte output.

section foo_binary {
    wrf "foo.bin"; // Write the file foo.bin in this section.
}

section bar {
    wr foo_binary;
    // Write the CRC hash of everything in foo_binary
    wr std::crc32c(foo_binary);
}

Users can also pass the section containing the extensions own output to the extension. The extension receives a slice of the full size of the section, including the size of the extension's own output. On input, the slice contains zero bytes at the location of the extension's future output. For example:

section foo_binary {
    wrf "foo.bin";
    // Warning, the CRC input data is the full length of the foo_binary
    // section and includes 4 trailing zero bytes in place of the
    // extension's output.
    wr std::crc32c(foo_binary);
}

section bar {
    wr foo_binary;
}

Named Parameters

To help eliminate bugs, Firmion extensions support named parameters. Extensions define their parameter names when registered. In the example below, we call the extension custom::my_extension passing it the required parameters data_section and code_section. The compiler passes the values by name in the order expected by the extension, regardless of the order given at the call site.

//
// extension example
//
section my_data {
    wrf "cool_data.bin";
};

section my_code {
    wrf "cool_code.bin";
};

section stuff {
    wr my_data;
    wr my_code;
    // Use named arguments to avoid positional and semantic bugs!
    custom::my_extension(data_section=my_data, code_section=my_code);
};

Size of Extension Output

Users can query the size of an extension's output using the sizeof command. For example:

assert sizeof(std::crc32c) == 4;

Creating and Registering a New Extension

Extensions register through the extensions crate (extensions/src/lib.rs). process.rs calls extensions::register_all once at startup; adding an extension requires no changes outside extensions/.

Step 1 — Create the extension crate

Place new extensions under std/ for proposed standard library extensions, or under a workspace path matching your namespace for third-party extensions. Implement the FirmionExtension trait from the firmion_extension crate.

// my_extension/src/lib.rs
use firmion_extension::FirmionExtension;
use extension_registry::ExtensionRegistry;

pub struct MyExtension;

impl FirmionExtension for MyExtension {
    fn name(&self) -> &str { "my_ns::my_ext" }
    fn size(&self) -> usize { 4 }
    fn execute(&self, _args: &[u64], img: &[u8], out: &mut [u8]) -> Result<(), String> {
        // write 4 bytes into out
        Ok(())
    }
}

pub fn register(registry: &mut ExtensionRegistry) {
    registry.register_ranged(Box::new(MyExtension));
}

Step 2 — Add the crate to the workspace

In the root Cargo.toml, add the crate path to [workspace] members.

Step 3 — Wire into extensions/

In extensions/Cargo.toml, add the new crate as a dependency:

my_extension = { path = "../my_extension" }

In extensions/src/lib.rs, call its register function inside register_all:

pub fn register_all(registry: &mut ExtensionRegistry) {
    std_crc32c::register(registry);
    my_extension::register(registry);  // add this line
}

Step 4 — Add tests

Create a tests/ directory in your extension crate with .firm scripts and an integration.rs test file. Use CARGO_MANIFEST_DIR to locate .firm files relative to the workspace root — see std/crc32c/tests/integration.rs for a complete example.

Run the extension's tests with:

cargo test -p my_extension

Firmion Development

This section provides notes for developers interested in contributing to Firmion.

Unit Testing

Firmion relies on 100's of unit tests to catch bugs. You can run these with:

cargo test --all

Fuzz Testing

Firmion supports fuzz tests for several of its internal libraries. Fuzz testing starts from a corpus of random inputs and then further randomizes those inputs to try to cause crashes and hangs. At the time of writing, fuzz testing requires the nightly build. See fuzz_help.md in the source repo for more information.

Checking Test Code Coverage

If you're using Windows as a development platform, then this worked for me to install the llvm-cov tool. I have the free version of Microsoft Visual Studio installed.

rustup component add llvm-tools
cargo install cargo-llvm-cov --locked

To generate an ASCII table of coverage stats to the terminal:

cargo llvm-cov --all-features --workspace

To update the coverage table in this README from Windows, run .\update_coverage.ps1.

Filename                                     Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
argvaldb/argvaldb.rs                               3                 0   100.00%           1                 0   100.00%           3                 0   100.00%           0                 0         -
ast/ast.rs                                      2426               510    78.98%          68                 9    86.76%        1319               222    83.17%           0                 0         -
ast/lexer.rs                                     421                13    96.91%          16                 0   100.00%         259                10    96.14%           0                 0         -
astdb/astdb.rs                                   781               117    85.02%          12                 0   100.00%         373                38    89.81%           0                 0         -
firmion_extension/lib.rs                             3                 0   100.00%           1                 0   100.00%           3                 0   100.00%           0                 0         -
const_eval/const_eval.rs                        1259               207    83.56%          32                 5    84.38%         800               174    78.25%           0                 0         -
depth_guard/depth_guard.rs                       146                 0   100.00%          17                 0   100.00%          77                 0   100.00%           0                 0         -
diags/diags.rs                                   256                26    89.84%          12                 1    91.67%         134                20    85.07%           0                 0         -
exec_phase/exec_phase.rs                         823               240    70.84%          20                 5    75.00%         535               137    74.39%           0                 0         -
extension_registry/extension_registry.rs         258                 9    96.51%          18                 3    83.33%         126                 9    92.86%           0                 0         -
extension_registry/test_mocks.rs                 259                34    86.87%          38                 6    84.21%         204                33    83.82%           0                 0         -
extensions/src/lib.rs                              8                 0   100.00%           1                 0   100.00%           5                 0   100.00%           0                 0         -
ir/ir.rs                                         309                30    90.29%          30                 1    96.67%         230                21    90.87%           0                 0         -
irdb/irdb.rs                                    1067               111    89.60%          25                 1    96.00%         610                68    88.85%           0                 0         -
layout_phase/layout_phase.rs                    1640               319    80.55%          48                 2    95.83%        1014               170    83.23%           0                 0         -
layoutdb/layoutdb.rs                             795               164    79.37%          20                 0   100.00%         423                65    84.63%           0                 0         -
linearizer/linearizer.rs                         603                29    95.19%          21                 1    95.24%         337                19    94.36%           0                 0         -
locationdb/locationdb.rs                          39                 4    89.74%           3                 1    66.67%          28                 4    85.71%           0                 0         -
map_phase/map_phase.rs                           892                13    98.54%          57                 0   100.00%         613                 9    98.53%           0                 0         -
process/process.rs                               425                25    94.12%          26                 5    80.77%         240                 9    96.25%           0                 0         -
prune/prune.rs                                   150                 9    94.00%          12                 2    83.33%          95                 7    92.63%           0                 0         -
regiondb/regiondb.rs                             127                 5    96.06%           3                 0   100.00%         107                 5    95.33%           0                 0         -
src/main.rs                                      164                14    91.46%          11                 3    72.73%         108                10    90.74%           0                 0         -
std/crc32c/src/crc32c.rs                          31                 2    93.55%           5                 0   100.00%          26                 3    88.46%           0                 0         -
std/md5/src/md5.rs                                31                 2    93.55%           5                 0   100.00%          26                 3    88.46%           0                 0         -
std/sha256/src/sha256.rs                          31                 2    93.55%           5                 0   100.00%          26                 3    88.46%           0                 0         -
symtable/symtable.rs                             107                 5    95.33%          14                 2    85.71%          78                 5    93.59%           0                 0         -
validation_phase/validation_phase.rs              98                 3    96.94%           4                 0   100.00%          68                 2    97.06%           0                 0         -
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                          13152              1893    85.61%         525                47    91.05%        7867              1046    86.70%           0                 0         -

Firmion Source Code Overview

Rebuilding the vscode Syntax Highlighting Extension

Rebuilding the extension require Node.js. After you install Node.js, you may need to restart your command prompt.

Building the extension requires vsce. One time, you'll need to use npm to install vsce

npm install -g @vscode/vsce

Now you're ready to rebuild the extension.

cd vscode-firmion
vsce package

To install the extension into vscode locally:

code --install-extension vscode-firmion-0.1.0.vsix