Skip to content

MisterVVP/FauxzureQ

Repository files navigation

FauxzureQ

FauxzureQ is a lightweight Azure Storage emulator for Queue, Table, Blob, and File endpoints. The emulator keeps a fast in-memory working set while asynchronously persisting queue, table, blob, and file state to the local WAL-backed storage path so it can recover data after restarts. It powers internal integration tests as well as external consumers.

Project status

FauxzureQ remains an experimental, nights-and-weekends pet project. We welcome thoughtful contributions, but please expect occasional rough edges while the emulator matures.

Supported features

Queue service

  • Create, list, and delete queues, including metadata requests and delete-hold semantics that mirror Azure responses (queue routes, in-memory store).
  • Enqueue messages with configurable initial visibility delays and TTLs, peek without dequeuing, dequeue with visibility timeouts, update message contents or visibility, clear a queue, and delete individual messages with pop-receipt validation (queue routes, in-memory store).
  • Automatically redirect poison messages that exceed the configurable dequeue threshold to a -poison queue, preserving their payloads for inspection (in-memory store: poison handling, queue cleanup).

Table service

  • List, create, and delete tables via the standard OData endpoints, including support for the Prefer: return-no-content hint (table routes).
  • Insert, merge, replace, and delete entities with PartitionKey/RowKey addressing, ETag conditional headers, and Prefer negotiation for response bodies (table routes).
  • Query entities with continuation tokens, $top limits, and filters covering PartitionKey equality, RowKey range predicates, and property equality comparisons (table routes: query handling, filter parsing).
  • Serialize entity properties using the Azure Table type system (e.g., Edm.Int32, Edm.Guid, Edm.DateTime, binary) and emit the correct metadata annotations (entity JSON serialization).

Blob service

  • Create, list, and delete containers, then upload, download, list, and delete blobs over REST-style endpoints (blob routes, blob store).

File service

  • Create, list, and delete shares, then upload, download, list, and delete files using share-relative paths (file routes, file store).

Shared capabilities

  • Queue, table, blob, and file state persistence through the local asynchronous WAL repository, plus per-service health probes and optional HTTP request logging.

Current limitations

  • Simplified authentication. Configuration exposes only the storage account name and does not validate Shared Key or SAS tokens; use the emulator on trusted networks (storage model).
  • Basic OData filter support. Table queries support equality for PartitionKey, range predicates for RowKey, and simple property equality, but they do not yet implement the full OData grammar (no or, arithmetic, or function calls) (filter parsing).

Prerequisites

FauxzureQ targets C++20. Ensure your toolchain and standard library fully support C++20 before configuring the build (we regularly exercise Clang 17+ on Linux and MSVC 19.44+ on Windows).

Linux build

Ensure the Ninja build system is available (sudo apt-get install ninja-build on Debian/Ubuntu). FauxzureQ serves requests from its in-memory working set and asynchronously records WAL entries under --storage-path so state can be replayed on startup after a restart.

cmake -S . -B build -G "Ninja" -DCMAKE_BUILD_TYPE=Debug
cmake --build build
./build/fauxzureq --listen=:10001 --table-listen=:10002 --blob-listen=:10003 --file-listen=:10004 --storage-path=.fauxzureq-data --account=devstoreaccount1

Windows Build

Install Ninja and ensure it is on your PATH.

cmake -S . -B build -G "Ninja" -DCMAKE_BUILD_TYPE=Debug
cmake --build build
.\build\fauxzureq.exe --listen=:10001 --table-listen=:10002 --blob-listen=:10003 --file-listen=:10004 --account=devstoreaccount1

Running unit tests

cmake -S . -B build -G "Ninja" -DCMAKE_BUILD_TYPE=Debug
cmake --build build
ctest --test-dir build --output-on-failure

The unit test suite lives under tests/ and covers queue, table, blob, and file behavior along with shared utilities.

Performance testing

The repository ships with k6 workloads for queue enqueue, table write, table read, blob write/read, and file write/read paths. You can run the queue, table, and blob workloads against FauxzureQ or Azurite. File workloads currently run only against FauxzureQ because Azurite does not implement Azure Files.

Table read performance testing

The repository ships with a k6 workload that stresses the table GET path. To run it locally:

# build the emulator first
cmake -S . -B build -G "Ninja" -DCMAKE_BUILD_TYPE=Release
cmake --build build

# install the latest k6 release if it is not already available
tmp_k6=$(mktemp -d)
curl -sS -H "Accept: application/vnd.github+json" -H "User-Agent: fauxzureq-docs" \
  https://api.github.com/repos/grafana/k6/releases/latest | \
  python3 -c "import json,sys; print(json.load(sys.stdin)['tag_name'], end='')" > "$tmp_k6/tag"
k6_tag=$(cat "$tmp_k6/tag")
curl -sSfL "https://github.com/grafana/k6/releases/download/${k6_tag}/k6-${k6_tag}-linux-amd64.tar.gz" -o "$tmp_k6/k6.tgz"
tar -xzf "$tmp_k6/k6.tgz" -C "$tmp_k6"
sudo mv "$tmp_k6"/k6-*/k6 /usr/local/bin/k6
rm -rf "$tmp_k6"

# start the emulator in one terminal window
./build/fauxzureq --listen=:10001 --table-listen=:10002 --account=devstoreaccount1 --http-log=false

# in a second terminal, execute the performance scenario
TABLE_URL=http://127.0.0.1:10002 \
TABLE_READ_DURATION=30s \
TABLE_READ_VUS=24 \
k6 run --summary-export table-read-summary.json tests/perf/table_read.js

# inspect the exported summary for request latency and failure information
cat table-read-summary.json

Adjust TABLE_READ_DURATION and TABLE_READ_VUS to model heavier or lighter table workloads as needed.

Additional knobs exposed by the scenario:

  • TABLE_ENTITY_COUNT – total entities provisioned before the run (default 512).
  • TABLE_PARTITION_COUNT – number of partitions to spread inserts across (default 32).
  • TABLE_READ_SEED – deterministic RNG seed so you can reproduce read distributions.
  • TABLE_ACCOUNT / TABLE_NAME – override the storage account and table names if required.

The --summary-export artifact contains useful latency and failure rates. For a quick latency snapshot you can extract the p(95) duration with:

jq -r '.metrics["http_req_duration{scenario:tableRead}"].values["p(95)"]' table-read-summary.json

The scenario also exports two custom trends:

  • table_read_waiting_ms – the client-observed time spent waiting for the emulator to respond. A rising tail here usually means the runner is CPU saturated or socket accepts are queuing.
  • table_read_server_ms – the duration the emulator reports spending on the request (emitted via the X-Fauxzureq-Server-Duration-Ms response header).

During straggler analysis look at both. If table_read_waiting_ms spikes into seconds while table_read_server_ms remains sub-millisecond, the long latency was caused by the host not scheduling the client promptly (common on shared GitHub runners). The k6 script now logs a warning with a full timing breakdown whenever a read waits longer than one second so you can correlate stragglers with the exported summary.

Blob write and read performance testing

Blob workloads follow the same pattern on the blob endpoint:

# start FauxzureQ with the blob endpoint enabled
./build/fauxzureq --listen=:10001 --table-listen=:10002 --blob-listen=:10003 --file-listen=:10004 --account=devstoreaccount1 --http-log=false

# in another terminal, measure write throughput and latency
BLOB_URL=http://127.0.0.1:10003 \
BLOB_WRITE_DURATION=30s \
BLOB_WRITE_VUS=64 \
k6 run --summary-export blob-write-summary.json tests/perf/blob_write.js

# then measure blob reads against a seeded container
BLOB_URL=http://127.0.0.1:10003 \
BLOB_READ_DURATION=30s \
BLOB_READ_VUS=64 \
k6 run --summary-export blob-read-summary.json tests/perf/blob_read.js

Useful blob knobs:

  • BLOB_WRITE_PAYLOAD_SIZE – blob payload size in bytes for write tests (default 4096).
  • BLOB_READ_BLOB_COUNT – number of blobs seeded before the read test (default 256).
  • BLOB_READ_BLOB_SIZE – blob size in bytes for the read test corpus (default 4096).
  • BLOB_ACCOUNT / BLOB_KEY / BLOB_URL – override the target storage account, shared key, or base URL when benchmarking Azurite or another compatible emulator.

Like the existing table and queue workloads, the blob scenarios export latency distributions and custom waiting/server timing trends so you can distinguish emulator-side work from client-side runner contention.

File write and read performance testing

File workloads exercise the simplified share-relative file endpoints exposed by FauxzureQ:

# start FauxzureQ with the file endpoint enabled
./build/fauxzureq --listen=:10001 --table-listen=:10002 --blob-listen=:10003 --file-listen=:10004 --account=devstoreaccount1 --http-log=false

# in another terminal, measure file uploads
FILE_URL=http://127.0.0.1:10004 \
FILE_WRITE_DURATION=30s \
FILE_WRITE_VUS=64 \
k6 run --summary-export file-write-summary.json tests/perf/file_write.js

# then measure file reads against a seeded share
FILE_URL=http://127.0.0.1:10004 \
FILE_READ_DURATION=30s \
FILE_READ_VUS=64 \
k6 run --summary-export file-read-summary.json tests/perf/file_read.js

Useful file knobs:

  • FILE_WRITE_PAYLOAD_SIZE – file payload size in bytes for write tests (default 4096).
  • FILE_READ_FILE_COUNT – number of files seeded before the read test (default 256).
  • FILE_READ_FILE_SIZE – file size in bytes for the read corpus (default 4096).
  • FILE_ACCOUNT / FILE_KEY / FILE_URL – override the target storage account, shared key, or base URL for the benchmark target.

About

High throughput azure queue and table storage emulator.

Topics

Resources

License

Contributing

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors