- Apply image processing algorithms using WebGL.
- Supports pictures, videos, webcam streams and user defined arrays.
- Everything is sent to the fragment shader as 2D textures.
- Let fragment shaders handle the hard work.
- Blog post.
- The store_* functions return a storageId, which is passed into the processing functions.
window.onload = function() {
// Send the picture to WebGL as texture. To do this with video/webcam...
// ... change store_picture to store_video / store_webcam.
var storageId = numgl.store_picture("imgElementID");
numgl.show_texture(storageId);
}numgl.grey(storageId);Fiddle with it:
// [-1,-1,-1,0,0,0,1,1,1] is a 3x3 kernel, where:
// - 1st row is [-1,-1,-1]; 2nd row is [0,0,0]; 3rd row is [1,1,1].
numgl.convolution(storageId, [-1,-1,-1,0,0,0,1,1,1]);
// Other kernels are supported.JS Fiddle with it:
// 80 is the thresholding value.
numgl.threshold(storageId, 80);JS Fiddle:
- The image processing functions return the GLSL variable that will hold the final pixel value.
- This return variable can be used to chain call other image processing functions.
window.onload = function() {
var imageId = numgl.store_picture("image");
numgl.show_canvas(imageId);
// Convolution followed by threshold
var convResult = numgl.convolution(imageId,[-1,-1,-1,0,0,0,1,1,1]);
numgl.threshold(convResult,10);
numgl.do_it();
}JS Fiddle:
- Enable / disable fps for video and webcam
// If this is not set, the fps are not shown.
numgl.set_fps_element(fpsElementId);- Store JS arrays as RGBA textures
// Draws the following 2x2 RGBA texture:
// [white, black,
// black, white]
// numgl.store_array()'s last 2 args are width and height.
var arrayId = numgl.store_array([255,255,255,255,
0,0,0,255,
0,0,0,255,
255,255,255,255],
2, 2);
// show_texture() calls do_it() internally.
numgl.show_texture(arrayId);- Read pixels from canvas
// If no width and height are specified, read_canvas() will read the whole canvas.
console.log(numgl.read_canvas().toString());
// Using the above example the result would be: "255,255,255,255,0,0,0,255,0,0,0,255,255,255,255,255"- Flip texture - arrays aren't flipped, but images and videos are by default (texture and clipspace coordinates are different). JS Fiddle example
// Called before numgl.do_it()
numg.textures[storageId].flipTexture = true/false- Manually define fl_FragCoord. JS Fiddle example
// Paint the screen blue.
numgl.fragColor = "vec4(0, 0, 1, 1);"- See the generated GLSL code.
// Fragment shader code. Vertex shader code function is vs_code().
console.log(numgl.fs_code("pretty"));-
Video and webcam use requestAnimationFrame so the video/webcam frames are drawn multiple times. Pictures are only drawn once.
-
Javascript functions generate GLSL code, and when the user calls numgl.do_it() that GLSL code is compiled and executed. eg:
window.onload = function() {
// "image" is the <img> tag ID.
var imageId = numgl.store_picture("image");
numgl.grey(imageId);
// Console log the GLSL code generated by the numgl.grey() call:
console.log(numgl.fs_code("pretty"));
}The resulting GLSL code: (uTexture0 is the stored texture - it can be an array, picture or video frame)
precision highp float;
uniform vec2 uResolution;
uniform sampler2D uTexture0;
uniform vec2 uTextureSize0;
varying vec2 vTextCoords;
void main(void) {
float float_0 = 0.2126 * texture2D(uTexture0, vTextCoords).r;
float float_1 = 0.7152 * texture2D(uTexture0, vTextCoords).g;
float float_2 = 0.0722 * texture2D(uTexture0, vTextCoords).b;
float float_3 = float_0 + float_2 + float_1;
vec4 vec4_0 = vec4(float_3,float_3,float_3,1);
gl_FragColor = vec4_0;
}- The final fragment color (gl_FragColor) is set by these GLSL code creating functions, and can be checked or set manually using numgl.fragColor. JS Fiddle example.
-
For local images / video, you'll need to set up a local server. It's easy, just open up your console, go to your project's folder and type in
python3 -m http.server 8000. -
Images and video from other websites require the inline HTML attribute
crossorigin="anonymous"- see the img tag here, for example.
- Webcams only work on https sites if you're using Chrome. On Firefox, webcams should work fine (for now).