Skip to content
boxthetaPublic

About

No description, website, or topics provided.

Resources

Code of conduct

Contributing

Security policy

Stars

0 stars

Watchers

0 watching

Forks

asnkit

Rust Python package PyPI crates.io npm

Note

trunk is at 2.0.0-rc8, a release candidate, and this README describes it. The registries still serve 1.2.0 by default, and 1.2.0 cannot read the version 2 .asni files that the 2.0 compiler writes. Until 2.0.0 is released, ask for the release candidate explicitly: pip install --pre asnkitrt, cargo install asnkitcc --version 2.0.0-rc8 (and the same --version for each plugin), cargo add asnkitrt@2.0.0-rc8 and npm install asnkitrt@next.

Release candidates of a new major version make no promise of compatibility with each other: the APIs, the command line and the .asni format may change between them, as the header of .asni files did between rc4 and rc5. Use the same release candidate for the compiler, the plugins and the runtimes, and compile your .asni files again when you move to another. See CHANGELOG.md.

An ASN.1 toolkit that parses a specification once, so that encoding and decoding can happen anywhere. A compiler turns ASN.1 into a compiled specification IR, an .asni file. A runtime loads that file to encode and decode. It never parses ASN.1 itself, which keeps it small enough for embedded devices and makes it behave the same in every language. Plugins generate code, such as data classes or Rust codecs, from the same file.

foo.asn --> compiler --> foo.asni --> runtime: encode, decode
                            |
                            +-------> plugin: Python or Rust code

Try it in the browser in the playground, which runs the compiler and the runtime as WebAssembly; the documentation is on the same site.

asnkit supports a large subset of ASN.1; what it does not support yet is listed under Known limitations, and is rejected with an error rather than encoded wrongly.

What is in this repository

Package What it does Location Registry License
Runtimes Encode and decode with an .asni file.
asnkitrt Runtime for Python. packages/asnkitrt PyPI MIT
asnkitrt Runtime for Rust. Needs alloc but not std, so it builds for embedded targets. packages/asnkitrt-rs crates.io MIT
asnkitrt Runtime for browsers and Node.js: the Rust runtime as WebAssembly. packages/asnkitrt-wasm npm MIT
asnkitrt-c Runtime for C and C++: the Rust runtime as a shared and a static library, with a header for each. packages/asnkitrt-c build from source MIT
org.boxtheta:asnkitrt Runtime for the JVM, in Kotlin, for Kotlin, Java and the other languages of the JVM. packages/asnkitrt-jvm GitHub Packages MIT
Compiler Parses ASN.1 and writes .asni files.
asnkit-compiler The compiler as a Rust library, which also runs the plugins, for an application to compile ASN.1 itself. packages/asnkit-compiler crates.io Apache-2.0
asnkitcc The command, made with the library. packages/asnkitcc crates.io Apache-2.0
asnkit-compiler The compiler for browsers and Node.js, as WebAssembly. packages/asnkit-compiler-wasm npm Apache-2.0
Plugins Generate code from a compiled specification.
asnkit-gen-python Python data classes. packages/asnkit-gen-python PyPI Apache-2.0
asnkit-gen-rust Rust structs and enums with serde. packages/asnkit-gen-rust crates.io Apache-2.0
asnkit-gen-rust-codec Rust codecs for one selected encoding: BER, DER or UPER, with borrowed values and caller-provided buffers. packages/asnkit-gen-rust-codec crates.io Apache-2.0

All the packages are released together, with one version; see CHANGELOG.md. Within a major version, the .asni and .asnst formats, the runtime APIs and the command line keep working.

Also in the repository:

  • docs/file_formats.md: the .asni and .asnst file formats.
  • docs/third_party_runtimes.md: guidelines for writing a runtime in another language.
  • docs/plugins.md: how plugins are run, and how to write one.
  • docs/any_defined_by.md: how to configure the types an ANY DEFINED BY can hold.
  • docs/versioning.md: what each kind of release may change.
  • skills/asnkit: an agent skill for using the runtimes and the compiler.
  • tests/vectors: encodings and decodings recorded from the Rust runtime, the one of reference, for testing a runtime in any language.
  • examples/corpus: .asni files for testing a reader of the format.

Codecs

Codec Python Rust WebAssembly and C JVM
Basic Encoding Rules (BER) yes yes yes yes
Distinguished Encoding Rules (DER) yes yes yes yes
Octet Encoding Rules (OER) yes yes yes not yet
Aligned Packed Encoding Rules (PER) yes yes yes not yet
Unaligned Packed Encoding Rules (UPER) yes yes yes not yet
JSON Encoding Rules (JER) yes yes, feature jer yes yes
XML Encoding Rules (XER) yes yes, feature xer yes yes
Generic String Encoding Rules (GSER) encoding only yes, feature gser yes not yet

The Rust runtime is the one of reference: the more than 5,700 vectors in tests/vectors are what it does, including constraint checks, and the Python runtime gives the same results for all of them. The WebAssembly and C runtimes are the Rust one; the WebAssembly runtime is tested on the vectors through its JavaScript API. The JVM runtime is held to all the vectors of the codecs it has. In Rust, the text codecs are crate features that are enabled by default. GSER decoding takes the generic forms of RFC 3641, as bare values or as the value assignments the encoder writes; the Python runtime only encodes GSER. xer uses quick-xml and bigint uses num-bigint, and both need std, so embedded builds use default-features = false. On the JVM, XER reads XML with Woodstox.

The DER decoder rejects anything X.690 does not allow, such as a BOOLEAN other than 0x00 or 0xff, an INTEGER that is not minimally encoded, an unsorted SET OF, a REAL not in its canonical form, a UTCTime or GeneralizedTime other than the one form DER has, or data after the members of a SEQUENCE or a SET that are not additions after its extension marker. Pass lenient=True to asnkitrt.load(), set lenient in the options of the other runtimes, or give --lenient to asnkitcc convert, to accept anything BER allows.

The BER decoder does not take data after the members of a SEQUENCE or a SET either, other than additions after an extension marker, nor after the value inside an explicit tag. lenient has both decoders leave out whatever is there. Neither takes a tag number in more bytes than it needs, or a value that ends after the contents it is in, lenient or not.

Every decoder refuses data that its codec does not write, with a decoding error and nothing else, in time and memory proportional to the data. What the nodes mean tells what each codec reads.

Known limitations

  • EMBEDDED PDV is not supported yet.

  • ANY and ANY DEFINED BY are raw encoded bytes, which BER and DER write as they are and PER, UPER and OER as an open type. The bytes are in the codec's own encoding, so a value decoded with BER can't be encoded again with PER as is. JER and XER need the actual type, and so reject them. Use asnkitcc compile --any-defined-by choices.toml (also supported by generate) to select types from defining member values; see docs/any_defined_by.md. Explicit mappings let the codecs encode and decode the selected values. Fallback types and runtime load-time overrides are not implemented.

  • The Rust runtime does not support CHARACTER STRING. Bounds of constraints of more than 128 bits need its feature bigint, which is on by default and needs std. Anything unsupported returns an error; it never produces a wrong value.

  • A union inside one constraint that mixes kinds, such as WITH COMPONENT (SIZE (1) | FROM ("a")), is checked as no constraint: the sizes and the alphabets are combined separately, as PER does. This accepts more values than the standard allows, but never rejects a valid one.

  • The Python runtime does not check a WITH COMPONENTS on a type that refers to itself. The Rust runtime does.

Installation

The Python packages need Python 3.13 or later, the crates Rust 1.88 or later, and the WebAssembly runtime Node.js 18 or later (or a browser with WebAssembly).

pip install asnkitrt                 # Encode and decode.
pip install asnkit-gen-python        # Generate Python code from ASN.1 with asnkitcc.

The compiler, the Rust runtime and the plugins for Rust are on crates.io, and the WebAssembly runtime and compiler are on npm. The runtime for C and C++ is built from source; see packages/asnkitrt-c. The JVM runtime needs Java 17 or later; see packages/asnkitrt-jvm.

cargo install asnkitcc               # The compiler.
cargo install asnkit-gen-rust        # Generate Rust code from ASN.1 with asnkitcc.
cargo install asnkit-gen-rust-codec  # Generate Rust codecs from ASN.1 with asnkitcc.
cargo add asnkitrt                   # Encode and decode in Rust.
npm install asnkitrt                 # Encode and decode in browsers and Node.js.
npm install asnkit-compiler          # Compile ASN.1 in browsers and Node.js.

While trunk is a release candidate, add the pre-release flags from the note at the top of this README.

Example

This ASN.1 specification defines the messages of a fictitious Foo protocol (based on the FooProtocol example on Wikipedia):

Foo DEFINITIONS ::= BEGIN

    Question ::= SEQUENCE {
        id        INTEGER,
        question  IA5String
    }

    Answer ::= SEQUENCE {
        id        INTEGER,
        answer    BOOLEAN
    }

END

Compile it to foo.asni in the current directory (the file is tests/files/foo.asn in this repository):

$ asnkitcc compile tests/files/foo.asn

Python

Load the file and encode and decode with it. Only asnkitrt is needed:

>>> import asnkitrt
>>> foo = asnkitrt.load_asni_file('foo.asni')
>>> encoded = foo.encode('Question', {'id': 1, 'question': 'Is 1+1=3?'})
>>> encoded
b'0\x0e\x02\x01\x01\x16\tIs 1+1=3?'
>>> foo.decode('Question', encoded)
{'id': 1, 'question': 'Is 1+1=3?'}

BER is the default codec. The same with PER:

>>> foo = asnkitrt.load_asni_file('foo.asni', 'per')
>>> foo.encode('Question', {'id': 1, 'question': 'Is 1+1=3?'})
b'\x01\x01\tIs 1+1=3?'

Rust

The Rust runtime loads the same file. Values are Values, with a Map for a SEQUENCE and so on, just like the dictionaries in Python:

use asnkitrt::container::Checks;
use asnkitrt::runtime::{Options, Specification};
use asnkitrt::types::Codec;
use asnkitrt::value::Value;

let data = std::fs::read("foo.asni")?;
let foo = Specification::from_asni(&data, &Checks::default(), Options::new(Codec::Uper))?;

let question = Value::Map(vec![
    ("id".into(), Value::Integer(1)),
    ("question".into(), Value::text("Is 1+1=3?")),
]);
let encoded = foo.encode("Question", &question)?;

assert_eq!(foo.decode("Question", &encoded)?, question);

JavaScript

The WebAssembly runtime loads the same file, in a browser or in Node.js, with the same values as in Python, in JavaScript types:

import { loadFile } from 'asnkitrt';

const foo = await loadFile('foo.asni', 'uper');
const encoded = foo.encode('Question', { id: 1, question: 'Is 1+1=3?' });

foo.decode('Question', encoded); // { id: 1, question: 'Is 1+1=3?' }

Generated code

A plugin generates native types for a language from the same specification. asnkitcc generate compiles the specification and runs the plugin on it:

$ asnkitcc generate --plugin python -o out tests/files/foo.asn   # Writes out/foo.py.
$ asnkitcc generate --plugin rust -o src tests/files/foo.asn     # Writes src/foo.rs.
>>> from foo import Question
>>> encoded = Question(id=1, question='Is 1+1=3?').encode('uper')
>>> Question.decode(encoded, 'uper')
Question(id=1, question='Is 1+1=3?')
mod foo;

use asnkitrt::types::Codec;
use foo::{Asn1Type, Question};

let specification = foo::load(Codec::Uper)?;
let encoded = Question { id: 1, question: "Is 1+1=3?".into() }.encode(&specification)?;

assert_eq!(Question::decode(&specification, &encoded)?.id, 1);

See the examples folder for more, docs/plugins.md for how plugins work and how to write one, and docs/file_formats.md for what an .asni file contains.

Generated Rust codecs

asnkit-gen-rust-codec generates Rust types with encoding and decoding methods for one selected codec: BER (the default), DER or UPER. The generated code needs no .asni file at runtime. It borrows decoded data from the input and encodes into a caller-provided buffer, avoiding heap allocations except for boxed recursive values. Allocating to_ber(), to_der() or to_uper() helpers are also available.

The plugin is published on crates.io. Install it and the matching compiler, replacing <version> with the same published release for both (including the -rcN suffix for a release candidate), then generate UPER code for the example above:

cargo install asnkitcc --version "<version>" --locked
cargo install asnkit-gen-rust-codec --version "<version>" --locked
asnkitcc generate --plugin rust-codec --plugin-opt codec=uper -o src tests/files/foo.asn
mod foo;

use foo::Question;

let question = Question { id: 1, question: "Is 1+1=3?".into() };
let mut buffer = [0u8; 64];
let length = question.encode_uper(&mut buffer)?;
let decoded = Question::decode_uper(&buffer[..length])?;

assert_eq!(decoded.question, "Is 1+1=3?");

Only the selected codec's methods are emitted. The consuming crate needs asnkitrt of the same version as the plugin, as asnkitrt::wire may change between minor versions; default-features = false supports builds without std. Use the plugin option no_std=true when generating for those builds.

Across 37 X.509 certificates on an Apple M5, generated BER/DER decoding took about 0.74 µs per certificate and buffer encoding about 1.52 µs; UPER took about 0.76 µs and 3.64 µs respectively. All three used zero heap allocations for decoding, length calculation and buffer encoding. The decoded value occupied 768 bytes on this 64-bit target, borrowing its input; this is not peak stack usage. See the benchmark results and reproduction command for buffer sizes and measurement details, and the plugin documentation for options and supported types. Generated codecs support fewer types and constraint checks than the interpreter; their limitations are listed separately.

Command line

asnkitcc compile foo.asn    # Writes foo.asni, for a runtime to load.
asnkitcc parse foo.asn      # Writes foo.asnst, a syntax tree for plugins.
asnkitcc generate --plugin NAME [--plugin-opt KEY=VALUE ...] foo.asn
asnkitcc convert -t Question -i q.json -o q.der foo.asni   # JER to DER.
asnkitcc schema -o foo.schema.json foo.asn   # A JSON Schema of JER, or foo.xsd or foo.dtd for XER.
asnkitcc --help             # Every option.

python -m asnkitrt.syntax foo.asni    # Prints a file as JSON.

convert decodes a value with one codec and encodes it with another, with an .asni file or ASN.1 files it compiles first. schema writes a JSON Schema of JER, or an XML Schema or a DTD of XER, of the types, and refuses what a schema can not tell, as an ANY; see packages/asnkitcc. asnkitcc prints nothing on success but warnings. Its output is checked against the cases in packages/asnkit-compiler/tests/golden, recorded from the Python compiler it replaced.

The compiler warns, on standard error, of what is valid ASN.1 but likely to cause trouble, and still writes the file. For now that is RecursiveType: types that refer to themselves, as List ::= SEQUENCE { tail List OPTIONAL }. Their values nest as deep as the data has them; the decoders stop at 128 levels, and other tools may not support them.

It refuses, as errors, what no runtime could use:

  • A type defined by itself alone, through references, as A ::= B and B ::= [0] A, or through COMPONENTS OF, which no runtime loads.
  • A type of which every value would hold another, as Never ::= SEQUENCE { next Never }, so none can be encoded or decoded.
  • Constraints that permit no value, as INTEGER (5..1), or (10..20) on a type that is (0..5), which the runtimes would take for no constraint.

.asni uses format 2.0, with a 10-byte header that ends with the CRC-32 of the rest of the file. Every runtime checks it before it parses anything, so a corrupt file is rejected and never read. Recompile version 1 files, and those of 2.0.0-rc4, from their ASN.1 sources. .asnst is at 1.1, which a reader of 1.0 reads. Signing, encryption, nonces and binding have been removed, along with their options and dependencies; the CRC-32 detects corruption, not tampering. Provide those protections in the firmware update or patching system, or another layer that distributes the files. See File formats for the layout.

Contributing

Contributions are welcome. See CONTRIBUTING.md for how to build and test a change, and for the sign-off (DCO) that each commit needs. Everyone who takes part follows the Code of Conduct. To report a vulnerability, see SECURITY.md.

Standards

The ASN.1 standards published by the ITU-T and the IETF.

Notation

Encoding rules

Acknowledgements

asnkit began as a fork of asn1tools by Erik Moqvist, and has since become a different project, with its own compiler, file formats and runtimes. What is left of asn1tools is in the codecs of the Python runtime, which keep its copyright and its MIT license (see packages/asnkitrt/NOTICE).

License

The runtimes, such as asnkitrt, are licensed under the MIT License (LICENSE-RUNTIMES). The compiler, the plugins, the documentation and everything else are licensed under the Apache License 2.0 (LICENSE-COMPILER). The test vectors are made from ASN.1 modules written by others, and keep their authors' licenses. We make no licensing claims on what the compiler and the first-party plugins produce: you may use their output under your own terms.

THIS PROJECT IS PROVIDED AS IS WITH NO WARRANTY AS PER THE LICENSE.

About

No description, website, or topics provided.

Resources

Code of conduct

Contributing

Security policy

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages