Creating WebGL Effects with CurtainsJS

This article focuses adding WebGL effects to <image> and <video> elements of an already “completed” web page. While there are a few helpful resources out there on this subject (like these two), I hope to help simplify this subject by distilling the process into a few steps: 

• Create a web page as you normally would.
• Render pieces that you want to add WebGL effects to with WebGL.
• Create (or find) the WebGL effects to use.
• Add event listeners to connect your page with the WebGL effects.

Specifically, we’ll focus on the connection between regular web pages and WebGL. What are we going to make? How about a draggle image slider with an interactive mouse hover!

We won’t cover the core functionality of slider or go very far into the technical details of WebGL or GLSL shaders. However, there are plenty of comments in the demo code and links to outside resources if you’d like to learn more. 

We’re using the latest version of WebGL (WebGL2) and GLSL (GLSL 300) which currently do not work in Safari or in Internet Explorer. So, use Firefox or Chrome to view the demos. If you’re planning to use any of what we’re covering in production, you should load both the GLSL 100 and 300 versions of the shaders and use the GLSL 300 version only if curtains.renderer._isWebGL2 is true. I cover this in the demo above.

First, create a web page as you normally would

You know, HTML and CSS and whatnot. In this case, we’re making an image slider but that’s just for demonstration. We’re not going to go full-depth on how to make a slider (Robin has a nice post on that). But here’s what I put together:

1. Each slide is equal to the full width of the page.
2. After a slide has been dragged, the slider continues to slide in the direction of the drag and gradually slow down with momentum.
3. The momentum snaps the slider to the nearest slide at the end point. 
4. Each slide has an exit animation that’s fired when the drag starts and an enter animation that’s fired when the dragging stops.
5. When hovering the slider, a hover effect is applied similar to this video.

I’m a huge fan of the GreenSock Animation Platform (GSAP). It’s especially useful for us here because it provides a plugin for dragging, one that enables momentum on drag, and one for splitting text by line . If you’re uncomfortable creating sliders with GSAP, I recommend spending some time getting familiar with the code in the demo above.

Again, this is just for demonstration, but I wanted to at least describe the component a bit. These are the DOM elements that we will keep our WebGL synced with. 

Next, use WebGL to render the pieces that will contain WebGL effects

Now we need to render our images in WebGL. To do that we need to:

1. Load the image as a texture into a GLSL shader.
2. Create a WebGL plane for the image and correctly apply the image texture to the plane.
3. Position the plane where the DOM version of the image is and scale it correctly.

The third step is particularly non-trivial using pure WebGL because we need to track the position of the DOM elements we want to port into the WebGL world while keeping the DOM and WebGL parts in sync during scroll and user interactions.

There’s actually a library that helps us do all of this with ease: CurtainsJS! It’s the only library I’ve found that easily creates WebGL versions of DOM images and videos and syncs them without too many other features (but I’d love to be proven wrong on that point, so please leave a comment if you know of others that do this well).

With Curtains, this is all the JavaScript we need to add:

// Create a new curtains instance
const curtains = new Curtains({ container: "canvas", autoRender: false });
// Use a single rAF for both GSAP and Curtains
function renderScene() {
  curtains.render();
}
gsap.ticker.add(renderScene);
// Params passed to the curtains instance
const params = {
  vertexShaderID: "slider-planes-vs", // The vertex shader we want to use
  fragmentShaderID: "slider-planes-fs", // The fragment shader we want to use
  
 // Include any variables to update the WebGL state here
  uniforms: {
    // ...
  }
};
// Create a curtains plane for each slide
const planeElements = document.querySelectorAll(".slide");
planeElements.forEach((planeEl, i) => {
  const plane = curtains.addPlane(planeEl, params);
  // const plane = new Plane(curtains, planeEl, params); // v7 version
  // If our plane has been successfully created
  if(plane) {
    // onReady is called once our plane is ready and all its texture have been created
    plane.onReady(function() {
      // Add a "loaded" class to display the image container
      plane.htmlElement.closest(".slide").classList.add("loaded");
    });
  }
});

We also need to update our updateProgress function so that it updates our WebGL planes.

function updateProgress() {
  // Update the actual slider
  animation.progress(wrapVal(this.x) / wrapWidth);
  
  // Update the WebGL slider planes
  planes.forEach(plane => plane.updatePosition());
}

We also need to add a very basic vertex and fragment shader to display the texture that we’re loading. We can do that by loading them via <script> tags, like I do in the demo, or by using backticks as I show in the final demo.  

Again, this article will not go into a lot of detail on the technical aspects of these GLSL shaders. I recommend reading The Book of Shaders and the WebGL topic on Codrops as starting points.

If you don’t know much about shaders, it’s sufficient to say that the vertex shader positions the planes and the fragment shader processes the texture’s pixels. There are also three variable prefixes that I want to point out:

• ins are passed in from a data buffer. In vertex shaders, they come from the CPU (our program). In fragment shaders, they come from the vertex shader.
• uniforms are passed in from the CPU (our program).
• outs are outputs from our shaders. In vertex shaders, they are passed into our fragment shader. In fragment shaders, they are passed to the frame buffer (what is drawn to the screen).

Once we’ve added all of that to our project, we have the same exact thing before but our slider is now being displayed via WebGL! Neat.

CurtainsJS easily converts images and videos to WebGL. As far as adding WebGL effects to text, there are several different methods but perhaps the most common is to draw the text to a <canvas> and then use it as a texture in the shader (e.g. 1, 2). It’s possible to do most other HTML using html2canvas (or similar) and use that canvas as a texture in the shader; however, this is not very performant.

Create (or find) the WebGL effects to use

Now we can add WebGL effects since we have our slider rendering with WebGL. Let’s break down the effects seen in our inspiration video:

1. The image colors are inverted.
2. There is a radius around the mouse position that shows the normal color and creates a fisheye effect.
3. The radius around the mouse animates from 0 when the slider is hovered and animates back to 0 when it is no longer hovered.
4. The radius doesn’t jump to the mouse’s position but animates there over time.
5. The entire image translates based on the mouse’s position in reference to the center of the image.

When creating WebGL effects, it’s important to remember that shaders don’t have a memory state that exists between frames. It can do something based on where the mouse is at a given time, but it can’t do something based on where the mouse has been all by itself. That’s why for certain effects, like animating the radius once the mouse has entered the slider or animating the position of the radius over time, we should use a JavaScript variable and pass that value to each frame of the slider. We’ll talk more about that process in the next section.

Once we modify our shaders to invert the color outside of the radius and create the fisheye effect inside of the radius, we’ll get something like the demo below. Again, the point of this article is to focus on the connection between DOM elements and WebGL so I won’t go into detail about the shaders, but I did add comments to them.

But that’s not too exciting yet because the radius is not reacting to our mouse. That’s what we’ll cover in the next section.

I haven’t found a repository with a lot of pre-made WebGL shaders to use for regular websites. There’s ShaderToy and VertexShaderArt (which have some truly amazing shaders!), but neither is aimed at the type of effects that fit on most websites. I’d really like to see someone create a repository of WebGL shaders as a resource for people working on everyday sites. If you know of one, please let me know.

Add event listeners to connect your page with the WebGL effects

Now we can add interactivity to the WebGL portion! We need to pass in some variables (uniforms) to our shaders and affect those variables when the user interacts with our elements. This is the section where I’ll go into the most detail because it’s the core for how we connect JavaScript to our shaders.

First, we need to declare some uniforms in our shaders. We only need the mouse position in our vertex shader:

// The un-transformed mouse position
uniform vec2 uMouse;

We need to declare the radius and resolution in our fragment shader:

uniform float uRadius; // Radius of pixels to warp/invert
uniform vec2 uResolution; // Used in anti-aliasing

Then let’s add some values for these inside of the parameters we pass into our Curtains instance. We were already doing this for uResolution! We need to specify the name of the variable in the shader, it’s type, and then the starting value:

const params = {
  vertexShaderID: "slider-planes-vs", // The vertex shader we want to use
  fragmentShaderID: "slider-planes-fs", // The fragment shader we want to use
  
  // The variables that we're going to be animating to update our WebGL state
  uniforms: {
    // For the cursor effects
    mouse: { 
      name: "uMouse", // The shader variable name
      type: "2f",     // The type for the variable - https://webglfundamentals.org/webgl/lessons/webgl-shaders-and-glsl.html
      value: mouse    // The initial value to use
    },
    radius: { 
      name: "uRadius",
      type: "1f",
      value: radius.val
    },
    
    // For the antialiasing
    resolution: { 
      name: "uResolution",
      type: "2f", 
      value: [innerWidth, innerHeight] 
    }
  },
};

Now the shader uniforms are connected to our JavaScript! At this point, we need to create some event listeners and animations to affect the values that we’re passing into the shaders. First, let’s set up the animation for the radius and the function to update the value we pass into our shader:

const radius = { val: 0.1 };
const radiusAnim = gsap.from(radius, { 
  val: 0, 
  duration: 0.3, 
  paused: true,
  onUpdate: updateRadius
});
function updateRadius() {
  planes.forEach((plane, i) => {
    plane.uniforms.radius.value = radius.val;
  });
}

If we play the radius animation, then our shader will use the new value each tick.

We also need to update the mouse position when it’s over our slider for both mouse devices and touch screens. There’s a lot of code here, but you can walk through it pretty linearly. Take your time and process what’s happening.

const mouse = new Vec2(0, 0);
function addMouseListeners() {
  if ("ontouchstart" in window) {
    wrapper.addEventListener("touchstart", updateMouse, false);
    wrapper.addEventListener("touchmove", updateMouse, false);
    wrapper.addEventListener("blur", mouseOut, false);
  } else {
    wrapper.addEventListener("mousemove", updateMouse, false);
    wrapper.addEventListener("mouseleave", mouseOut, false);
  }
}


// Update the stored mouse position along with WebGL "mouse"
function updateMouse(e) {
  radiusAnim.play();
  
  if (e.changedTouches && e.changedTouches.length) {
    e.x = e.changedTouches[0].pageX;
    e.y = e.changedTouches[0].pageY;
  }
  if (e.x === undefined) {
    e.x = e.pageX;
    e.y = e.pageY;
  }
  
  mouse.x = e.x;
  mouse.y = e.y;
  
  updateWebGLMouse();
}


// Updates the mouse position for all planes
function updateWebGLMouse(dur) {
  // update the planes mouse position uniforms
  planes.forEach((plane, i) => {
    const webglMousePos = plane.mouseToPlaneCoords(mouse);
    updatePlaneMouse(plane, webglMousePos, dur);
  });
}


// Updates the mouse position for the given plane
function updatePlaneMouse(plane, endPos = new Vec2(0, 0), dur = 0.1) {
  gsap.to(plane.uniforms.mouse.value, {
    x: endPos.x,
    y: endPos.y,
    duration: dur,
    overwrite: true,
  });
}


// When the mouse leaves the slider, animate the WebGL "mouse" to the center of slider
function mouseOut(e) {
  planes.forEach((plane, i) => updatePlaneMouse(plane, new Vec2(0, 0), 1) );
  
  radiusAnim.reverse();
}

We should also modify our existing updateProgress function to keep our WebGL mouse synced.

// Update the slider along with the necessary WebGL variables
function updateProgress() {
  // Update the actual slider
  animation.progress(wrapVal(this.x) / wrapWidth);
  
  // Update the WebGL slider planes
  planes.forEach(plane => plane.updatePosition());
  
  // Update the WebGL "mouse"
  updateWebGLMouse(0);
}

Now we’re cooking with fire! Our slider now mets all of our requirements.

Two additional benefits of using GSAP for your animations is that it provides access to callbacks, like onComplete, and GSAP keeps everything perfectly synced no matter the refresh rate (e.g. this situation).

You take it from here!

This is, of course, just the tip of the iceberg when it comes to what we can do with the slider now that it is in WebGL. For example,  common effects like turbulence and displacement can be added to the images in WebGL. The core concept of a displacement effect is to move pixels around based on a gradient lightmap that we use as an input source. We can use this texture (that I pulled from this displacement demo by Jesper Landberg — you should give him a follow) as our source and then plug it into our shader. 

To learn more about creating textures like these, see this article, this tweet, and this tool. I am not aware of any existing repositories of images like these, but if you know of one please, let me know.

If we hook up the texture above and animate the displacement power and intensity so that they vary over time and based on our drag velocity, then it will create a nice semi-random, but natural-looking displacement effect:

It’s also worth noting that Curtains has its own React version if that’s how you like to roll.

That’s all I’ve got for now. If you create something using what you’ve learned from this article, I’d love to see it! Connect with me via Twitter.

---

The post Creating WebGL Effects with CurtainsJS appeared first on CSS-Tricks.

You can support CSS-Tricks by being an MVP Supporter.

source https://css-tricks.com/creating-webgl-effects-with-curtainsjs/