Skip to content
This repository has been archived by the owner on Oct 15, 2024. It is now read-only.
/ backend.ai-media Public archive

Auxiliary Javascript libraries for Backend.AI front-end developers

Notifications You must be signed in to change notification settings

lablup/backend.ai-media

Repository files navigation

Sorna Media Library

This contains a set of media support stubs for the Sorna code execution service.

Library Overview

  • sorna.drawing -- This is a simple drawing library that maps server-side running Python codes into a frontend-side running Javascript/Fabric canvas drawings.
  • sorna.matplotlib -- A plotting backend to encode the generated plot images into dataURI strings when the user code calls plt.show().
  • assets -- The front-end side Javascript assets to utilize JSON-encoded media outputs generated by above server-side Python packages as well as web terminals.

Server-side Python packages records the media outputs in the builtins module and Dockerized Sorna REPL scripts pass them to the Sorna Agent along with the stdout/stderr strings via JSON over ZeroMQ sockets. Upon receiving those results, the front-end side Javascripts should call Sorna.Media.handle_all(<media-output-array>, <result-identifier>, <result-ctonainer>) where result-identifier should be a unique string for each code block and result-container should be a reference to HTML element such as <div> blocks used for rendering the execution results.

For Developers

Setting up

We use yarn and webpack for bundling Javascript files and CSS resources so that we keep the main script small (less than 100KB) while the main script loads all the necessary stuffs dynamically.

Check out the installation instruction of yarn package manager first.

# Use package.json to install all dependencies locally:
$ yarn install
# To run a local development server serving auto-rebuilt in-memory bundles:
$ yarn run devserver
# To run the production build:
$ yarn run build

Integration with a front-end

You need to specify Sorna.assetRoot in Javascript to let our scripts know which location to fetch additoinal scripts from. The main.min.js is designed to be small for faster page loads and most functionality (e.g., drawing support) are loaded on demand.

For production:

<script>
window.Sorna = window.Sorna || {};
window.Sorna.assetRoot = '//<sorna-serving-host>/<hash>';
</script>
<script src="//<sorna-serving-host>/<hash>/js/main.min.js"></script>

<sorna-serving-host> would be placed by a template variable from application server settings.

For development:

<script>
window.Sorna = window.Sorna || {};
window.Sorna.assetRoot = 'http://localhost:8002/latest';
</script>
<script src="http://localhost:8002/latest/js/main.min.js"></script>

When receiving Sorna execution results:

var response = ...;
var result_id = ...;
var result_container = document.getElementById(...);
// media is a list of (type, data) tuples produced by server-side Pyhon packages
Sorna.Media.handle_all(response.media, result_id, result_container);

Developing with local neumann frontend instances

We use webpack-dev-server to automatically recompile the sources on the memory whenever they change (aka "watch-mode"). However, you need to manually refresh the page to get the latest bundles as we do not use "hot module refresh" (HMR) due to conflicts with script tags without src attributes (e.g., ZenDesk-injected scripts).

$ yarn run devserver
# Bundled scripts are served at http://127.0.0.1:8002/latest/js/...

Note that the port number in the configuration is fixed for Lablup's internal development configuration. You may change it if you have different frontends.

Deploying for production service

We use the standard aws-cli tool. You should configure your AWS access key and the secret key to make it working.

Before uploading, we first need to compile the resources for production.

$ yarn run update
# This includes "yarn run build" process.

This script will write the compiled resources into assets/<hash> directory, where the hash value depends on the content of all resource files. It also deletes all other assets/<old-hash> directories automatically to avoid duplicate transfers below. To debug the webpack build process, simply run webpack and see what it says.

Then, run the following to upload all assets:

$ aws s3 cp assets s3://sorna-assets/ --recursive

Afterwards, you must update the production configuration (e.g., SORNA_ASSET_ROOT in Django/Flask settings) for your front-end using the latest hash value. (e.g., https://s3.ap-northeast-2.amazonaws.com/sorna-assets/1234567890abcdef1234 )

About

Auxiliary Javascript libraries for Backend.AI front-end developers

Topics

Resources

Stars

Watchers

Forks

Packages

No packages published

Languages