Minor bug fixes

This release contains a variety of small features:

• Origami expressions can now appear in YAML documents
• New Origami.htmlEscape builtin for escaping HTML entities
• Added support for the JavaScript in and instanceof operators
• Improved interop with JavaScript global methods
• Dev.audit and Dev.crawl can now accept a string argument with the URL of the site to examine
• Origami.rss now translates the JSON Feed external_url field to the RSS <source> field, often used in link blogs. Thanks @jimniels!

Change to reduceFn parameter of Tree.mapReduce builtin

This release includes a breaking change in mapReduce. Previously its reduceFn parameter had a signature of (values, keys, source). The new signature is now (node, source). Instead of passing separate mapped values and the original keys, the new node is a Map of the keys to the new, mapped values.

This release also includes a new builtin, Tree.reduce, which does the same reduction work without the need to specify a value mapping function.

Map-based tree API members

Origami makes extensive use of Map and AsyncMap objects as nodes in a tree. To facilitate various tree operations, Origami also supports optional tree members.

trailingSlashKeys property

One optional tree member is the trailingSlashKeys property. Origami classes that create map-based trees generally set this property to true to indicate that the tree's keys will include trailing slashes for child nodes.

The Origami tree methods that traverse trees can inspect the trailingSlashKeys property to save work: if a tree supports trailing slashes, but a key doesn't have a trailing slash, then the traversal operation assumes the key is for a leaf node. The operation therefore won't get the key's value. In traversals of large sites this can save a significant amount of work.

Two tree operations that now rely on trailingSlashKeys are Tree.paths and Tree.sitemap. Previously those operations had supported an assumeSlashes option, but that option is no longer necessary; passing it will have no effect.

Google Drive extension now supports writes

The @weborigami/gdrive extension has been updated to support writes as well as reads. This allows, for example, the use of ori copy to copy a tree to a Google Drive folder.

This is a significant Origami release with hopefully few user-visible effects beyond those documented here.

Existing Origami projects should work the same as before, but given the number of changes made to Origami's foundation, this release has a minor version bump.

Complete rewrite of fundamental tree interface to use standard Map class

The higher levels of the Web Origami project rest on a foundational notion of a tree of nodes that may require asynchronous calls to traverse. For the past few years, these trees have been defined by a small interface called AsyncTree. That interface can be compared to a stripped-down version of the standard JavaScript Map class. Origami didn't use the Map class itself because it has some quirks.

However, recent experiments suggested ways to work around those quirks. That has now been done, encapsulating the workarounds in a new base class, SyncMap, that inherits from Map. With that, most of the trees you work with in Origami, including those based on in-memory objects, the file system, and local data like JSON/YAML files, are now represented with Map instances.

Being able to build everything on the interface of the Map class should make it easier to explain the lower levels of Origami and make using the async-tree library from JavaScript easier to understand.

Because the Map class doesn't support async calls, it was necessary to introduce an async variation of that class called AsyncMap. That class is used to represent network resources.

Rewritten pattern walkthrough

The Pattern walkthrough has been completely rewritten using the Map class as the basic interface.

The remaining documentation has similarly been overhauled to use Map terminology. In general, the word "map" is used to refer to something that is a Map or behaves like one. The word "tree" is used when such map objects are organized into tree structures.

Easier to view tree content in JavaScript debugger

When debugging a JavaScript function that's been passed a Map, you can directly inspect the contents of the map in the debug console. Additionally, Map objects created for sync data sources like objects and files will expose a preview property accessor: inspecting the value of that property will show you the current contents of the map.

More reliable ori output

One side effect of using Map-based trees is that the ori CLI can use them to more accurately render command output as YAML.

Previously ori would render a tree to a plain JavaScript object before rendering that as YAML. This conversion would cause any integer keys to appear sorted before any string keys. It turns out the YAML rendering library natively understands the Map class. The CLI can now pass a Map-based tree directly to the YAML library and correctly preserves the order of the keys.

Before:

$ ori { a: 1, 0: 2 }
"0": 2
a: 1

After:

$ ori { a: 1, 0: 2 }
a: 1
"0": 2

This YAML improvement is, in fact, a good example of the advantages of using a standard class.

Deprecations

Builtin functions that used "tree" terminology have been renamed to use Map terminology. The old functions still work but display a deprecation warning in the console.

• Tree.length() → Tree.size() for consistency with Map.prototype.size
• Tree.isAsyncTree() → Tree.isMap(). That method returns true for anything with the abstract Map interface, including regular Map instances as well as AsyncMap instances.
• Tree.isTreelike() → Tree.isMaplike()

Underlying driver classes have also been renamed.

• ExplorableSiteTree → ExplorableSiteMap
• FileTree → FileMap
• FunctionTree → FunctionMap
• ObjectTree → ObjectMap
• SetTree → SetMap
• SiteTree → `SiteMap

Extensions

If you use any of the following extensions, update them to 0.0.15 to get the version that works with this Origami 0.6.0 release.

• @weborigami/dropbox
• @weborigami/epub
• @weborigami/gdrive
• @weborigami/gist
• @weborigami/handlebars
• @weborigami/json-schema
• @weborigami/pagefind
• @weborigami/ts
• @weborigami/zip

Adds support for .tsv and .sh files

• .tsv files are treated as tab-separated values with a header row. A new builtin, Origami.tsv, can output a tree in TSV format.
• .sh files are treated as shell script files. Unpacking them returns a function that can be called to run the script. You can pass an argument to that function; it will be treated as stdin.

Bug fixes

• Fixed Tree.mapExtension so that, when invoked with parameters (tree, extensions, value) it properly accepts a .ori or .js file name as the value parameter
• Fixed the Explorer page available via !explore so that it once again shows project folders and files outside the specific site file being debugged

This release contains many changes, although only a few of them might affect sites written in Origami.

Tree.groupBy builtin replaces Tree.group

For consistency with the JavaScript Object.prototype.groupBy method, the Tree.group builtin has been renamed to Tree.groupBy. Thanks to @jimniels for drawing attention to the inconsistency with JavaScript.

To aid migration, Tree.group continues to work in this release and displays a deprecation warning in the console.

Tree.mapExtension

A new builtin, Tree.mapExtension, now handles the extension-mapping feature of Tree.map.

If you have code like:

posts/ = Tree.map(markdown, { extension: ".md->.html", value: fn })

replace that with one of the following:

posts/ = Tree.mapExtension(markdown, ".md->.html", fn)
posts/ = Tree.mapExtension(markdown, ".md->.html", { value: fn })
posts/ = Tree.mapExtension(markdown, { extension: ".md->.html", value: fn })

Refactoring

This release includes substantial refactoring to the organization of the Origami monorepo which have some slight effects on calling Origami from JavaScript.

• The built-in set of file extension handlers and URL protocol handlers have been moved from the @weborigami/origami repo to the @weborigami/language repo. This reflects the fact that those features are core parts of the Origami language. The handlers are grouped into a Handlers export. E.g., the markdown handler is available as Handlers.md_handler. (Also see handlers below.)
• The toString() utility has been moved from @weborigami/origami to @weborigami/async-tree. Since the async-tree library doesn't know anything about document objects, the ability to implicitly extract a document's _body property has been dropped; you'll need to extract that explicitly when working with documents.
• The builtins exported by @weborigami/origami are now grouped into Dev and Origami objects just like the builtins are exposed to Origami code.

File extension handler names now use an underscore

Origami file extension handlers are identified by names. Previously the names had to end in .handler (with a dot). The names now end in _handler (with an underscore). This allows extension handler names to be valid JS identifiers, making it easier to work with them in JS.

If you have a config.ori file that specifies an extension handler for something like foo.handler, update it to foo_handler.

Better support for calling standard JavaScript methods

This release includes substantial changes to the Origami runtime. Among other things, these changes enable more support for calling standard JavaScript methods.

Examples: the following code fragments now work:

Promise.all(['fruit', 'computer', 'park'].map(item => `Apple ${item}`)).join(', ')
Math.max.apply(null, [1, 3, 2])

Removal of deprecated features

This release drops support for a number of deprecated features. All of these were flagged with console warnings in earlier releases.

• .jse and .jse.html file extensions are no longer supported. Use .ori and .ori.html instead.
• Shorthand function syntax with = is no longer supported in Origami programs (but still supported in the shell). Use standard JS () => arrow syntax instead.
• A Unicode ellipsis … is no longer supported for object spreads. Instead, use three periods ... as in JavaScript.
• Tree.concat is no longer supported. Use Tree.deepText instead.
• Tree.defineds is no longer supported.
• Tree.fromFn is no longer supported. Use Tree.withKeys instead.
• Tree.remove is no longer supported. Use Tree.delete instead.
• Tree.setDeep is no longer supported. Use Tree.assign instead.

Update extensions

Most of the Origami extensions have now been updated to version 0.0.14:

• @weborigami/dropbox
• @weborigami/epub
• @weborigami/esbuild
• @weborigami/gdrive
• @weborigami/gist
• @weborigami/handlebars
• @weborigami/json-schema
• @weborigami/ts
• @weborigami/zip

This is a minor release for Origami language users, but includes breaking changes for JavaScript users directly calling functions in the async-tree library.

The focus of this release in internal refactoring to align the functions in the async-tree library so that they exactly match what's visible to Origami programs as the Tree namespace. Previously, there were two implementations of many operations like Tree.map: a lightweight one in the async-tree library, and a more feature-rich one in the main origami package. This release converges those implementations and moves all tree operations to the async-tree library.

Tree builtins

All builtins should generally work the same as before. Reviewing and updating this code revealed that a number of seldom-used functions may no longer be necessary. These have been deprecated:

• Tree.concat is deprecated. For a while it's been essentially the same as Tree.deepText but due to an oversight was never removed. Use Tree.deepText instead.
• Tree.defineds is deprecated because I'm not aware of a use for it. If you're using it, let me know.
• Tree.fromFn has been generalized and renamed to Tree.withKeys.
• Tree.setDeep has been deprecated because it turns out to have the same behavior as Tree.assign; use that instead.
• Tree.remove has been renamed to Tree.delete for consistency with JavaScript methods like Map.prototype.delete.

async-tree API changes

This release also includes some non-trivial changes to the JavaScript async-tree library, although few people are using that library directly. None of these changes should have visible effects at the Origami language level.

• All tree functions (map, reverse, sort, etc.) are now async.
• The async-tree package now exports all operations as a single Tree. collection, rather than as individually named exports.
• The map operation now calls the key function option with an initial value. The key function's parameters were previously (sourceKey, tree) and are now (sourceValue, sourceKey, tree). The signature now matches what has always been exposed to Origami programs.
• The origami package no longer exports any tree operations; use the async-tree package directly instead.

Experimental Tree.mapExtension method

The ability of Tree.map to map file extensions is extremely useful, but the API design is bad. The extension option conflicts with use of the key or inverseKey options, which are used to provide finer-grained control over key mapping. In fact, under the hood, setting extension actually generates key and inverseKey functions for you.

Whenever a method has multiple parameters that can't work together, that's a signal that the method is trying to do too many things.

The proposed solution to this problem is to separate the extension-mapping behavior into a new builtin, Tree.mapExtension.

// Existing syntax
Tree.map(posts, { extension: ".md→.html", value: page.ori })

// Proposed
Tree.mapExtension(posts, ".md→.html", page.ori) // last param is the value function
Tree.mapExtension(posts, ".md→.html", { deep: true, value: page.ori }) // if you have more options

This new Tree.mapExtension builtin is available in this release, but for the time being is not yet included in the documentation.

Give this a try and share your thoughts in the Origami chat. If this design feels good, this would become the official way of mapping extensions, and the extension option in Tree.map would be retired.

Update to extensions

The changes to async-tree required updating the following Origami extensions:

• esbuild
• json-schema
• handlebars

If you're using those extensions, update your package.json to pick up the latest version.

The CLI and the underlying Tree.plain function have been adjusted to produce more useful results when manipulating an array. Follow that link for more details.

Previously, applying Tree.reverse or Tree.sort to an array in the CLI would return a plain JS object. Because JavaScript always sorts the numeric keys of an object, this would have the effect of undoing the work of Tree.reverse or Tree.sort. Those functions only change the order of keys, they don't change a key's numeric value. So previously you would see

$ ori Tree.reverse [5, 2, 8]
- 5
- 2
- 8

which makes it look like Tree.reverse had no effect.

Tree.plain now expands its definition of an "array-like" tree to any tree whose keys are integers completely covering the range 0..length-1. Given an array-like tree, Tree.plain will return a JavaScript array with the tree's values in the order they're given but ignoring the numeric value of the keys.

$ ori Tree.reverse [5, 2, 8]
- 8
- 2
- 5

Minor improvements to JavaScript compatibility

• Support for the JavaScript typeof operator
• Support for the (extremely obscure, rarely encountered) JavaScript void operator
• The async and await keywords are now valid in Origami although they have no effect and aren't necessary; all functions are always async, and the runtime will implicitly await the value of any expression