Clean selective outlines from pixelated objects ~ Journey Beyond Devlog #1
The start of something
I've been working on Journey Beyond the Edge of the World for quite a while now, with occasional updates on the development process on my Twitter and Mastodon profiles. As I'm getting further into development, I thought it might be good to keep an actual Devlog on here, out of the constraints of social media posts. I also want to get the game out there more and hopefully start a small community for those interested in the project.
And this topic is the perfect place to start as I feel very good about having solved this (to me at least) pretty complicated issue!
Less pixels please!
Journey Beyond's visual style is influenced by quite a few, actually pretty different games. The amazing Return of the Obra Dinn certainly pushed my interest in setting a game on a boat, and while I love the look of that game, I pretty early realized I wanted a more "grounded" style. In a way, I actually want the visuals to take a step back since the sound experience, the music, and the spatial audio features are what should make the game stand out the most.
But, I do love me some crunchy pixels and I've been toying around with that kind of look for years now. I think what finally pushed me to try it on an actual project was A Short Hike and watching a post mortem by Adam Robinson-Yu (highly recommended watch!) where he talked about initially going for that pixelated look out of his limited skills in modelling (and of course because it looks great). Which I totally resonated with! I for one am personally really drawn to that look, it just has something textural about it that I like. But, even though my 3D modelling skills have improved quite a bit over the years, I didn't feel comfortable enough going for a truly realistic look in regards to my current skill level. Other styles like a toon or cel shading approach might have worked, but again I wanted a little less focus on the visual style and apart from the fact that I feel that look is highly oversaturated right now, I just didn't feel like it would fit well with the game's themes.
So, I went for it and here we are: A first person game with pixelated visuals. I've already gotten some feedback from players who felt a bit dizzy with all those moving pixels, so it's definetely a work in progress. Similar to how A Short Hike is handling it, I will most likely give the player a certain amount of control over the pixelation, also for accessibility reasons. But right now I'm personally quite happy with the look.
Challenges
While I love the pixelation on the actual objects in the game, I don't want that look to compromise on readability and interacting with the game. So it was clear to me that e.g. the UI would be except from the look and just be drawned in a clear manner on top. Easy!
But where things got tricky was when I experimented with ways to make interactable objects, specifically items that can be picked up, more visible. The obvious approach here where outlines of course, as many games do. Just draw an outline around the object when the player looks at them so they know they can be interacted with. While I do think this can take the player out of the - watch out, overused word coming up - immersion, I feel its usually a good compromise.
I've done my fair bit of work on outlines, edge detection and all that stuff on my previous project Relevating, so I already have a pretty solid system in place for that. But there I outlined every object on the screen using a full screen pass, something that wouldn't work for individual objects or layers. I theoretically knew how to overcome this by using other techniques (well of knowledge Alexander Ameye has a couple of great blog post about outline generation here and here). But here I ran into some hurdles ...
Clean outlines ... from pixelated objects?
Due to everything on the screen getting pixelated, I had to find a way to generate and display the outlines on top of that pixelation: I wanted to follow the style I've set for myself in the UI discussion and keep those outlines clean. I had no idea where to even start, but thankfully Alexander pointed me in the right direction (thank you, Alexander!) and started me on a journey beyond (sorry) Unity's Scriptable Render Features and RenderTextures.
RenderTextures to the rescue
I'm not super sure this is the best way to do all this, but since I don't see any performance implications this is what I'm going to stick with for now. The method uses two distinct render passes using Unity's Scriptable Render Features. The overall approach follows these steps:
1) Highlighted objects get shifted to the _Outline layer
2) In a first pass, all objects on the _Outline layer get drawn as white silhouettes to a RenderTexture (SilhouetteRenderTexture)
3) In a second pass, I perform edge detection using the Sobel operator on that RenderTexture and create outlines from that; saving the result into another RenderTexture (OutlineRenderTexture)
4) I draw the RenderTexture with the outlines to the screen using a RawImage, so it gets overlayed ontop of the main image
Things to watch out for
I feel like even regardless of my pretty specific situation this might be a neat solution to render selective outlines through a custom pass. Some gotchas in all of these where the fact that I had to assign the SilhouetteRenderTexture to the material manually once, as the ScriptableRendererFeature was for some reason not able to assign the texture itself. Not sure what is going on here. Also the Render Pass Events I've set in the Features might seem a bit odd, but since I have another Overlay camera in my camera stack for rendering objects that are being held by the player on top of everything else, I had to shift around some stuff here so things would get rendered in the proper order. Honestly I don't fully understand why the silhouette generation works right now that it is happening after post processing is rendered. I would think that would result in a pixelated silhouette! I have to wrap my head around this at some point, it feels weird not knowing what one's actual code is doing, haha. Right now I'm going for a "if it works, don't touch it" mentality on this.
Todos
What I do want to add at some point is, as Alexander suggested to me, having each objects drawn with a distinct color to the SilhouetteRenderTexture. This would help in avoiding issues where two objects overlap, resulting in a merged outline. But I'm shelving this for a later date. I also need to clean up the code a bit, I could theoretically handle the entire thing in one pass since I don't need access to the Silhouette RenderTexture - it could just be a temporary rendertexture. For debugging purposes I will keep it as it is for now though.
Pass 1: RendererFeature for drawing a layer to a RenderTexture
using UnityEngine;
using UnityEngine.Rendering;
using UnityEngine.Rendering.Universal;
public class CustomLayerToBufferFeature : ScriptableRendererFeature
{
[System.Serializable]
public class CustomLayerSettings
{
public LayerMask layerMask;
public Material customMaterial;
public RenderPassEvent renderPassEvent = RenderPassEvent.AfterRenderingOpaques;
public RenderTexture renderTexture; // RenderTexture to save the image
}
public CustomLayerSettings settings = new CustomLayerSettings();
CustomLayerToBufferPass customLayerToBufferPass;
public override void Create()
{
customLayerToBufferPass = new CustomLayerToBufferPass(settings.layerMask, settings.customMaterial, settings.renderTexture);
customLayerToBufferPass.renderPassEvent = settings.renderPassEvent;
}
public override void AddRenderPasses(ScriptableRenderer renderer, ref RenderingData renderingData)
{
renderer.EnqueuePass(customLayerToBufferPass);
}
}
public class CustomLayerToBufferPass : ScriptableRenderPass
{
private LayerMask layerMask;
private Material customMaterial;
private RenderTargetIdentifier renderTarget;
public CustomLayerToBufferPass(LayerMask layerMask, Material customMaterial, RenderTexture renderTexture)
{
this.layerMask = layerMask;
this.customMaterial = customMaterial;
renderTarget = new RenderTargetIdentifier(renderTexture);
}
public override void Configure(CommandBuffer cmd, RenderTextureDescriptor cameraTextureDescriptor)
{
// Configure the RenderTexture as the target for rendering
cmd.SetRenderTarget(renderTarget);
cmd.ClearRenderTarget(true, true, Color.clear);
}
public override void Execute(ScriptableRenderContext context, ref RenderingData renderingData)
{
CommandBuffer cmd = CommandBufferPool.Get();
using (new ProfilingScope(cmd, new ProfilingSampler("CustomLayerToBufferPass")))
{
var filterSettings = new FilteringSettings(RenderQueueRange.opaque, layerMask);
var drawSettings = CreateDrawingSettings(new ShaderTagId("UniversalForward"), ref renderingData, SortingCriteria.CommonOpaque);
drawSettings.overrideMaterial = customMaterial;
context.DrawRenderers(renderingData.cullResults, ref drawSettings, ref filterSettings);
// No need to blit to the RenderTexture since it's the render target
}
context.ExecuteCommandBuffer(cmd);
CommandBufferPool.Release(cmd);
}
}Pass 2: RendererFeature for generating Outlines from Silhouette and drawing to a RenderTexture
using UnityEngine;
using UnityEngine.Rendering;
using UnityEngine.Rendering.Universal;
public class CustomRenderTextureFeature : ScriptableRendererFeature
{
[System.Serializable]
public class CustomRenderTextureSettings
{
public LayerMask layerMask;
public Material customMaterial;
public RenderPassEvent renderPassEvent = RenderPassEvent.AfterRenderingOpaques;
public RenderTexture targetRenderTexture; // RenderTexture to save the image
public RenderTexture sourceRenderTexture; // Assignable source RenderTexture
}
public CustomRenderTextureSettings settings = new CustomRenderTextureSettings();
CustomRenderTexturePass customRenderTexturePass;
public override void Create()
{
customRenderTexturePass = new CustomRenderTexturePass(settings.layerMask, settings.customMaterial, settings.targetRenderTexture, settings.sourceRenderTexture);
customRenderTexturePass.renderPassEvent = settings.renderPassEvent;
}
public override void AddRenderPasses(ScriptableRenderer renderer, ref RenderingData renderingData)
{
renderer.EnqueuePass(customRenderTexturePass);
}
}
public class CustomRenderTexturePass : ScriptableRenderPass
{
private LayerMask layerMask;
private Material customMaterial;
private RenderTexture targetRenderTexture;
private RenderTexture sourceRenderTexture;
public CustomRenderTexturePass(LayerMask layerMask, Material customMaterial, RenderTexture targetRenderTexture, RenderTexture sourceRenderTexture)
{
this.layerMask = layerMask;
this.customMaterial = customMaterial;
this.targetRenderTexture = targetRenderTexture;
this.sourceRenderTexture = sourceRenderTexture;
}
public override void Configure(CommandBuffer cmd, RenderTextureDescriptor cameraTextureDescriptor)
{
// No need to configure the RenderTexture as the target for rendering
// It will be set as the target in the Execute function
}
public override void Execute(ScriptableRenderContext context, ref RenderingData renderingData)
{
CommandBuffer cmd = CommandBufferPool.Get();
using (new ProfilingScope(cmd, new ProfilingSampler("CustomRenderTexturePass")))
{
var filterSettings = new FilteringSettings(RenderQueueRange.opaque, layerMask);
var drawSettings = CreateDrawingSettings(new ShaderTagId("UniversalForward"), ref renderingData, SortingCriteria.CommonOpaque);
drawSettings.overrideMaterial = customMaterial;
// Set the source RenderTexture as the active RenderTexture
RenderTexture.active = sourceRenderTexture;
// Copy the source RenderTexture to the target RenderTexture
cmd.Blit(sourceRenderTexture, targetRenderTexture, customMaterial);
// Reset the active RenderTexture
RenderTexture.active = null;
}
context.ExecuteCommandBuffer(cmd);
CommandBufferPool.Release(cmd);
}
}Silhouette Shader
Shader "Custom/SilhouetteShader"
{
Properties
{
_Color("Silhouette Color", Color) = (1, 1, 1, 1)
}
SubShader
{
Tags { "RenderType" = "Opaque" }
Pass
{
ZWrite On
Cull Front
CGPROGRAM
#pragma vertex vert
#pragma fragment frag
struct appdata
{
float4 vertex : POSITION;
};
struct v2f
{
float4 pos : SV_POSITION;
};
v2f vert(appdata v)
{
v2f o;
o.pos = UnityObjectToClipPos(v.vertex);
return o;
}
fixed4 _Color;
fixed4 frag(v2f i) : SV_Target
{
return _Color;
}
ENDCG
}
}
}Silhouette to Outline Shader
Shader "Custom/SilhouetteOutlineShader"
{
Properties
{
_MainRenderTexture ("Source RenderTexture", 2D) = "white" {}
_OutlineColor ("Outline Color", Color) = (1, 0, 0, 1)
_OutlineThickness ("Outline Thickness", Range(0, 0.1)) = 0.01
}
SubShader
{
Tags { "RenderType"="Opaque" }
LOD 100
Pass
{
CGPROGRAM
#pragma vertex vert
#pragma fragment frag
#include "UnityCG.cginc"
struct appdata
{
float4 vertex : POSITION;
float2 uv : TEXCOORD0;
};
struct v2f
{
float2 uv : TEXCOORD0;
float4 vertex : SV_POSITION;
};
sampler2D _MainRenderTexture;
float4 _OutlineColor;
float _OutlineThickness;
v2f vert(appdata v)
{
v2f o;
o.vertex = UnityObjectToClipPos(v.vertex);
o.uv = v.uv;
return o;
}
fixed4 frag(v2f i) : SV_Target
{
fixed4 color = tex2D(_MainRenderTexture, i.uv);
// Apply edge detection using Sobel operator
float4 topLeft = tex2D(_MainRenderTexture, i.uv + float2(-1, 1) * _OutlineThickness);
float4 topRight = tex2D(_MainRenderTexture, i.uv + float2(1, 1) * _OutlineThickness);
float4 bottomLeft = tex2D(_MainRenderTexture, i.uv + float2(-1, -1) * _OutlineThickness);
float4 bottomRight = tex2D(_MainRenderTexture, i.uv + float2(1, -1) * _OutlineThickness);
float4 horizontal = topRight - topLeft;
float4 vertical = bottomLeft - topLeft;
float edgeIntensity = length(horizontal) + length(vertical);
// Draw the outline using the outline color
color = step(1, edgeIntensity) * _OutlineColor;
return color;
}
ENDCG
}
}
}