Android Question Fast blur for OpenGL ?

[Informatix]

Hello,

I know that probably no one will answer to such a question, but why not?
I'm looking for a solution to blur quickly an image with OpenGL. I know that Renderscript can do it easily but I have not a clear idea of how you can combine OpenGL and Renderscript. A good solution would be a shader of course, but until now all shaders that I found are either slow (because not designed for OpenGL ES on mobile devices) or very bad, or there's something special to do with the result and it's not mentioned (so the result is unusable).
I read of course the excellent tutorial of Matt DesLauriers but the suggested solutions do not cover my case (blurring bitmap fonts generated in real time).

 

[thedesolatesoul]

I know nothing about OpenGLES, but I am still intrigued.
I have used fastBlur and the RenderScript blur, see here: http://www.b4x.com/android/forum/threads/blur-effect-demo.40920/
There is always trickey involved in blurring, the main thing is to down sample the image before you blur it (I would suggest atleast 4 times).
From what I read, blurring is a two pass operation for the GPU, for some reason the GPU cannot do it so fast, this surprises me because:
- iOS uses a frosted glass effect which is effectively blur plus some alpha/color
- Android used to have BLUR_BEHIND_FLAG pre 2.3 I think (removed due to performance reasons)
- Some ROMs like MIUI are still using blur
Maybe they all are doing it on the CPU I guess.

 

[Informatix]

I know nothing about OpenGLES, but I am still intrigued.
I have used fastBlur and the RenderScript blur, see here: http://www.b4x.com/android/forum/threads/blur-effect-demo.40920/
There is always trickey involved in blurring, the main thing is to down sample the image before you blur it (I would suggest atleast 4 times).
From what I read, blurring is a two pass operation for the GPU, for some reason the GPU cannot do it so fast, this surprises me because:
- iOS uses a frosted glass effect which is effectively blur plus some alpha/color
- Android used to have BLUR_BEHIND_FLAG pre 2.3 I think (removed due to performance reasons)
- Some ROMs like MIUI are still using blur
Maybe they all are doing it on the CPU I guess.

I saw your library but it's very slow. My current shader for OpenGL is much faster but it is still too slow for a real-time use (imagine that you want to blur all the background to create an illusion of depth; that has to be done in less than 20ms) and I lose a bit of luminance in the process (the more you blur the more you darken the image). I don't have the mathematical skill to improve my current work.
The trick with resampling the image is in fact very fast in OpenGL (you select a level of your mip map chain) but it produces a lot of artifacts on some devices and with very blurry images, it's very ugly. You can achieve a perfect result, with the same speed, by creating yourself each element of the mip map chain (i.e. by using pre-blurred images), but for fonts generated during the game, you cannot create the levels in advance; that would be too long and cumbersome.
There's nothing hardcoded in the GPU to ease blurring. But it's definitely in the GPU that you have to run your code. Any Java-based solution will never be fast enough. I'm really interested by the Renderscript built-in functions because they are run in the GPU and benefit from multithreading, but I have no idea on how to pass data from Renderscript to OpenGL. What I read on the subject is not clear at all.

 

[thedesolatesoul]

Yes, my library was pretty slow that is why I added downsampling, maybe I should try the renderscript version.
Most of the code available for renderscript blurring is taking and returning a bitmap, is that not what you need in OpenGL?

I dont think blurring has anything to do with losing luminance. Maybe your source is just too dark, so you blur shader is just adding up luminance without normalizing it?

I'll see if I can find more information.

EDIT:
Apparently we need to use FBOs and Fragment Shader.
Seems like fragment shader works pixel by pixel, we just need to get the neighbours, apply a fancy gaussian blur or something and off we go.

 

[Informatix]

Yes, my library was pretty slow that is why I added downsampling, maybe I should try the renderscript version.
Most of the code available for renderscript blurring is taking and returning a bitmap, is that not what you need in OpenGL?

Bitmaps do not exist in OpenGL (we could describe a bitmap like a quad with a texture in OpenGL terms). There's a similar concept in the libGDX lib called Pixmap but these Pixmaps have to be converted to textures before being displayed and the process is quite slow.

I dont think blurring has anything to do with losing luminance. Maybe your source is just too dark, so you blur shader is just adding up luminance without normalizing it?

It's a typical issue with the gaussian blur algorithm but I found on internet an excel sheet that does the computation for me and gives me the ideal sigma and gaussian weights to avoid losing light. This problem is now fixed. Internet is amazing !
I optimized a little bit my code but it's still not usable in real time. Look at this APK. It blurs the whole background and a label over it. The number in the top left corner is the FPS + blur level (the level is increased every 3 seconds). The result is very clean (no artefact, no boxing). Exactly what you should get with your favourite paint program. The bottleneck in my fragment shader is the number of texcoord reads.
https://drive.google.com/file/d/0B-kneWWcCy7PbHVnaGZ3ZWJDU00/view?usp=sharing

 

[thedesolatesoul]

Nice work!
I can see as your blur radius increases the fps drops. But even on the very first blur it dropped to 30 fps on my GS3.
I guess it now depends on your algorithm, gaussian is pretty heavy, maybe try reading every alternate texcoord instead of all of them.

 

[thedesolatesoul]

Looks like there are many optimizations to be done, like multi pass on a box blur.
blog.ivank.net/fastest-gaussian-blur.html

 

[Informatix]

I found an algo for iOS that could be a solution. It's the first time that I try to understand a code written in Objective-C and it's just horrible to read (who's the crazy guy who invented this C offspring?).
EDIT: after reading Wikipedia, I see it's a language from the 80's. When computers were mainly for serious guys to do serious things with cryptic languages...

 

[Informatix]

Looks like there are many optimizations to be done, like multi pass on a box blur.
blog.ivank.net/fastest-gaussian-blur.html

Thanks for the link. Unfortunately, these algorithms are not ideal for shaders.

 

[thedesolatesoul]

Yes i didnt realize we shouldnt be doing multiple passes on the shader.
I dont have anymore ideas, I will probably have to learn more open gl es.

 

[Informatix]

I improved my code a lot since the last time, and I agree with your first post, it's cheaper to downsample in a FBO before blurring but now, I have another problem is that I create distinct blur levels. You jump from one to the other. The transition is a bit complicated to do (except by blending two textures blurred at different levels, but your speed is halved). Anyway, I made some significant progress and my gaussian blur is more or less usable in real-time. I will post my findings when I am really satisfied.

 

[thedesolatesoul]

I did read somewhere that people use 'blur keyframes' by pre-blurring them, and then blending between them. I didnt think that is very usable for games though?
I would have hoped there is some real-time blur support in HW somehow using OpenGL.
I would be very interesting to see your results if when you get them.

 

[Informatix]

Here are the current results with the Kasawe blur algorithm, which does not give a result as perfect as with the gaussian blur algorithm, but is faster (and you don't really do the difference in a game context). I downsample a bit (/2) before computing.
Here are the best references on the subject for Android (no boring theory, and source code available):
https://software.intel.com/en-us/bl...ast-real-time-gpu-based-image-blur-algorithms
https://github.com/mattdesl/lwjgl-basics/wiki/OpenGL-ES-Blurs
In the second page, you will see the technique of pre-blurred images. It's very usable but you have to create the mip map chain in advance, so it's not for graphics generated in real-time like a text. For images, it's probably one of the best solutions (with the drawback that the mip map chain takes extra space in memory).
I will write a tutorial for libGDX (including bloom/glow effect) this week if I have some time on my hands.
EDIT: in the demo, the second number is the blur kernel size.

 

[thedesolatesoul]

Thats a lot of stuff to read. (will takeme a while)
Looks great to me though, only on a kernel size of 127 does the framerate drop from 60 to ~45
Impressive

 

[Informatix]

Finally, I made a library (SpecialFX) and, in the examples, I show how to blur progressively (with the Kasawe algorithm) a bunch of sprites in real-time. I added also a few optimized shaders for common post-processing FX.