In this article, we explore post-processing by writing a pixelate filter using a compute shader.<\/p>\n\n\n\n
At its core, Post-processing, aka image processing or image filters, are a process where we take an image and modify the pixels. When it comes to games, generally, we take the image rendered by the camera and process it before outputting it to the screen. As a result, we can consider the image holistically, as opposed to inside the rendering pipeline, where we only have the information we calculated in the previous stages of the pipeline. On the other hand, we no longer have a 3d scene to work with; we only have a 2d image. As such, sometimes we’ll generate extra buffers to assist in the post-processing stage, such as a texture that contains the normal of every pixel in the final rendered image. The bottom line is that some effects are better suited to post-processing, and others fit into the rendering pipeline.<\/p>\n\n\n\n
As far as I know, using compute shaders for post-processing is out of the ordinary. However, I find it very effective and surprisingly natural. In particular, the ability to control the number of threads is handy. For example, dispatching a thread per pixel is easy to visualize. In our case, we’ll take the average of multiple pixels and set the value back to the texture. That means we can dispatch a thread per group of pixels instead. I believe that having that control is advantageous over standard post-processing.<\/p>\n\n\n\n
So, how do we write a pixelate filter? We take the average colour of a block of pixels and set those pixels to that average colour. It’s surprisingly straightforward.<\/p>\n\n\n\n
We’ll dispatch one thread for each square block of pixels. So, first, we’ll convert the thread id into a pixel position. Do that by multiplying the thread id by the block size.<\/p>\n\n\n\n
const float2 startPos = id.xy * _BlockSize;<\/code><\/pre>\n\n\n\nThis gives us the first pixel in the block, the corner. To make this code work, we need to know the block size. We’ll pass that in from C# so that we can change it on the fly. Then, we iterate through all the pixels in the block, sum all the colours and divide to get the average.<\/p>\n\n\n\n
float numPixels = blockWidth * blockHeight;\nfloat4 colour = float4(0, 0, 0, 0);\nfor (int i = 0; i < blockWidth; ++i)\n{\n for (int j = 0; j < blockHeight; ++j)\n {\n const uint2 pixelPos = uint2(startPos.x + i, startPos.y + j);\n colour += _Result[pixelPos];\n }\n}\ncolour \/= numPixels;<\/code><\/pre>\n\n\n\nHowever, before we do this, we need to calculate the block height and width. Usually, this is as easy as taking the _BlockSize<\/code> and assigning that. However, there’s a chance our block of pixels might extend past the end of the _Result<\/code> texture. That’s because our _BlockSize<\/code> may not fit perfectly into our _Result<\/code> texture resolution. So, we’ll do a little calculation to check. By the way, the _Result<\/code> texture is the final image output to the screen.<\/p>\n\n\n\nconst int blockWidth = min(_BlockSize, _ResultWidth - startPos.x);\nconst int blockHeight = min(_BlockSize, _ResultHeight - startPos.y);<\/code><\/pre>\n\n\n\nTo make this work, we’ll need to know the width and height of our _Result<\/code> texture, so we’ll pass that from C# as well. The equation will take the _BlockSize<\/code> as the width and height unless the distance from our starting position to the end of the texture is smaller.<\/p>\n\n\n\nThe final part of the shader is taking the average colour and assigning it to every pixel in our block.<\/p>\n\n\n\n
for (int i = 0; i < blockWidth; ++i)\n{\n for (int j = 0; j < blockHeight; ++j)\n {\n const uint2 pixelPos = uint2(startPos.x + i, startPos.y + j);\n _Result[pixelPos] = colour;\n }\n}<\/code><\/pre>\n\n\n\nPutting it all together, here’s what we get.<\/p>\n\n\n\n
#pragma kernel Pixelate\n\nRWTexture2D<float4> _Result;\n\nint _BlockSize;\nint _ResultWidth;\nint _ResultHeight;\n\n[numthreads(8,8,1)]\nvoid Pixelate (uint3 id : SV_DispatchThreadID)\n{\n if (id.x >= _ResultWidth || id.y >= _ResultHeight)\n return;\n\n const float2 startPos = id.xy * _BlockSize;\n \n if (startPos.x >= _ResultWidth || startPos.y >= _ResultHeight)\n return;\n \n const int blockWidth = min(_BlockSize, _ResultWidth - startPos.x);\n const int blockHeight = min(_BlockSize, _ResultHeight - startPos.y);\n const int numPixels = blockHeight * blockWidth;\n \n float4 colour = float4(0, 0, 0, 0);\n for (int i = 0; i < blockWidth; ++i)\n {\n for (int j = 0; j < blockHeight; ++j)\n {\n const uint2 pixelPos = uint2(startPos.x + i, startPos.y + j);\n colour += _Result[pixelPos];\n }\n }\n colour \/= numPixels;\n\n for (int i = 0; i < blockWidth; ++i)\n {\n for (int j = 0; j < blockHeight; ++j)\n {\n const uint2 pixelPos = uint2(startPos.x + i, startPos.y + j);\n _Result[pixelPos] = colour;\n }\n }\n}<\/code><\/pre>\n\n\n\nC# Compute Shader Runner<\/h2>\n\n\n\n
Over on the C# side, we’re mostly throwing values over to the GPU. However, there’s one unique feature since we’re doing post-processing. We’ll take advantage of the OnRenderImage<\/code> callback to get the camera’s frame buffer after rendering. We have to copy the frame buffer to a render texture with enableRandomWrite<\/code> enabled. Then, we perform the image processing on that render texture before copying it to the screen. By the way, OnRenderImage<\/code> only works with the built-in renderer, so this doesn’t work in URP or HDRP. To do this using the SRPs, I suspect you would write a Renderer Feature to run the compute shader after all the pipeline stages.<\/p>\n\n\n\nvoid OnRenderImage(RenderTexture src, RenderTexture dest)\n{\n Graphics.Blit(src, _renderTexture);\n\n var mainKernel = PixelateComputeShader.FindKernel(\"Pixelate\");\n PixelateComputeShader.SetInt(\"_BlockSize\", BlockSize);\n PixelateComputeShader.SetInt(\"_ResultWidth\", _renderTexture.width);\n PixelateComputeShader.SetInt(\"_ResultHeight\", _renderTexture.height);\n PixelateComputeShader.SetTexture(mainKernel, \"_Result\", _renderTexture);\n PixelateComputeShader.GetKernelThreadGroupSizes(mainKernel, out uint xGroupSize, out uint yGroupSize, out _);\n PixelateComputeShader.Dispatch(mainKernel,\n Mathf.CeilToInt(_renderTexture.width \/ (float)BlockSize \/ xGroupSize),\n Mathf.CeilToInt(_renderTexture.height \/ (float)BlockSize \/ yGroupSize),\n 1);\n\n Graphics.Blit(_renderTexture, dest);\n}<\/code><\/pre>\n\n\n\nThe rest of the script creates the render texture and updates it if the screen size changes. Here’s the entire script.<\/p>\n\n\n\n
using UnityEngine;\n\npublic class PixelateRunner : MonoBehaviour\n{\n public ComputeShader PixelateComputeShader;\n [Range(2, 40)] public int BlockSize = 3;\n\n int _screenWidth;\n int _screenHeight;\n RenderTexture _renderTexture;\n\n void Start()\n {\n CreateRenderTexture();\n }\n\n void CreateRenderTexture()\n {\n _screenWidth = Screen.width;\n _screenHeight = Screen.height;\n \n _renderTexture = new RenderTexture(_screenWidth, _screenHeight, 24);\n _renderTexture.filterMode = FilterMode.Point;\n _renderTexture.enableRandomWrite = true;\n _renderTexture.Create();\n }\n\n void Update()\n {\n if (Screen.width != _screenWidth || Screen.height != _screenHeight)\n CreateRenderTexture();\n }\n\n void OnRenderImage(RenderTexture src, RenderTexture dest)\n {\n Graphics.Blit(src, _renderTexture);\n\n var mainKernel = PixelateComputeShader.FindKernel(\"Pixelate\");\n PixelateComputeShader.SetInt(\"_BlockSize\", BlockSize);\n PixelateComputeShader.SetInt(\"_ResultWidth\", _renderTexture.width);\n PixelateComputeShader.SetInt(\"_ResultHeight\", _renderTexture.height);\n PixelateComputeShader.SetTexture(mainKernel, \"_Result\", _renderTexture);\n PixelateComputeShader.GetKernelThreadGroupSizes(mainKernel, out uint xGroupSize, out uint yGroupSize, out _);\n PixelateComputeShader.Dispatch(mainKernel,\n Mathf.CeilToInt(_renderTexture.width \/ (float)BlockSize \/ xGroupSize),\n Mathf.CeilToInt(_renderTexture.height \/ (float)BlockSize \/ yGroupSize),\n 1);\n\n Graphics.Blit(_renderTexture, dest);\n }\n}<\/code><\/pre>\n\n\n\nThat’s how you set up post-processing using a compute shader. Despite the name PixelateRunner<\/code>, this template could work for any post-processing-via-compute-shader situation. This example is relatively simple. We could extend this technique to create new effects, though. For example, by clamping the colours to a limited palette, we emulate different retro hardware. We could also experiment with operations other than taking the average colour, such as taking the most prominent or repeated colour in a block of pixels.<\/p>\n\n\n\nExperiment with the project <\/strong>here<\/strong><\/a> on GitHub. I’d love to hear about any variations on the effect that you create. If you find my work useful, <\/strong>join my mailing list,<\/strong><\/a> and I’ll email you whenever a new post is released.<\/strong><\/p>\n","protected":false},"excerpt":{"rendered":"In this article, we explore post-processing by writing a pixelate filter using a compute shader. How does post-processing work? At its core, Post-processing, aka image processing or image filters, are a process where we take an image and modify the pixels. When it comes to games, generally, we take the image rendered by the camera […]<\/p>\n","protected":false},"author":1,"featured_media":468,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[25,17,1],"tags":[26,8,41,40],"class_list":["post-461","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-compute-shaders","category-graphics","category-unity-programming","tag-compute-shaders","tag-game-development","tag-image-filter","tag-post-processing"],"_links":{"self":[{"href":"https:\/\/bronsonzgeb.com\/index.php\/wp-json\/wp\/v2\/posts\/461","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/bronsonzgeb.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/bronsonzgeb.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/bronsonzgeb.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/bronsonzgeb.com\/index.php\/wp-json\/wp\/v2\/comments?post=461"}],"version-history":[{"count":7,"href":"https:\/\/bronsonzgeb.com\/index.php\/wp-json\/wp\/v2\/posts\/461\/revisions"}],"predecessor-version":[{"id":469,"href":"https:\/\/bronsonzgeb.com\/index.php\/wp-json\/wp\/v2\/posts\/461\/revisions\/469"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/bronsonzgeb.com\/index.php\/wp-json\/wp\/v2\/media\/468"}],"wp:attachment":[{"href":"https:\/\/bronsonzgeb.com\/index.php\/wp-json\/wp\/v2\/media?parent=461"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/bronsonzgeb.com\/index.php\/wp-json\/wp\/v2\/categories?post=461"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/bronsonzgeb.com\/index.php\/wp-json\/wp\/v2\/tags?post=461"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}