B4A Library [Class] - OpenGL ES - 2D Image library

V1.5
Added: SetSkew() - To skew to image
Added: Reset() - Resets general image parameters (skew, position, rotation, handle, animation, flipped state)


I have put together a library designed for the purpose of displaying 2D images using OpenGL ES. The library needs Andrew Graham's libraries:

OpenGL ES library.
Reflection Library

Also, if you want to employ error checking you will also need Andrew's Threading library.

VIDEO: Jelly Beans


Key Features for the 2D Images
=======================================
* Pixel-Perfect image drawing (with unfiltered or filtered render options)
* Resize, rotate, flip, skew, change colour, alpha, blending modes, cropped areas - All faster than stock bitmap/canvas drawing
* Collision checks (only basic box-box checks)
* Anchor points - Each image has its own anchor hotspot (called handle in the library) so that the image can be rotated/flipped around a particular point
* An image can share the texture from an existing image (to save space and improve memory performance)
* Images don't need to be a power of 2 since the library will re-size them if required.
* Animation - Images can be animated easily by setting 'frames'. Image strips as well as image grids are supported
* Set specific areas of the image to draw

Key Features for the Display
=======================================
* Virtual display setup makes it easy to create a common display which scales to the devices physical screen size
* Global positioning (anchor/handle), scaling, and rotation which impacts all images.
* GL surface view can be made transparent so that GL images appear on top of regular views.

Have a play with the examples to see how the basic implementation works.
Hopefully the commands make sense.

OpenGL ES 2D Image Library v1.5 - Download link -- Select File then Download
NOTE: Demos should be run in Release Mode

V1.1
Added: Transparency for GL surface view. New subs = SetClsAlpha(), SetClsColor()
Changed: ClearScreen() to Cls()
V1.2
Added: Virtual display commands/functions
Added: SetRect() for images
V1.3
Added: GetLeft(), GetRight(), GetTop(), GetBottom(), AtPoint()
Added: More demos
V1.4a Minor change: Removed glBlendEquation
V1.4
Added: SetBlend - To change the rendering appearance of images (Alpha, Light,Dark,Mask)
Added: Jelly Beans demo
Changed: The way images are loaded. Instead of checking for surface changes every time Draw() is called, a dedicated (one-shot) LoadImages() sub is called instead.

 

[Informatix]

Although I really appreciate that free library, still, I agree with Informatix - the performance of an OpenGL surface should be better. As an example, I can point out Erel's Game View - it's much smoother though it's canvas based, I think. Too bad I cannot use it for Android below version 3.0, so I have to create 2 versions for each app where I need it - one OpenGL ES for Android below 3.0, and the other one using the Game View lib. Then I switch them depending on OS. That almost doubles the amount of work.

The GameView class does not really uses a "simple" canvas. In fact, it uses the hardware accelerated canvas available since Honeycomb. In the Android API, this canvas is a wrapper around OpenGL functions. I use the same canvas in my AcceleratedSurface lib (but I'm compatible with Gingerbread )

 

[bloxa69]

I use the same canvas in my AcceleratedSurface lib (but I'm compatible with Gingerbread )

Searched the forums for "AcceleratedSurface" but couldn't find anything. Is it your private lib?

 

[Informatix]

Searched the forums for "AcceleratedSurface" but couldn't find anything. Is it your private lib?

Yes, I don't know if I will release it publicly. It's not decided yet. I just mentioned it here to compare results.

 

[agraham]

As long as your target is running API 11 (Honeycomb) or later you can enable hardware acceleration by a manifest entry.

Hardware Acceleration | Android Developers

You can check if a View is hardware accelerated by Reflection.

Dim r As Reflector
Dim hw As Boolean
r.Target = myview
hw = r.RunMethod("isHardwareAccelerated")
Msgbox(hw, "HardwareAccelerated")

I got the reflection code above reporting hardware acceleration on my Xoom just by by putting

SetApplicationAttribute(android:hardwareAccelerated, "true")

in the Manifest Editor. I was already compiling with "...\android-13\android.jar" in my android.jar path.

The minSdkVersion and targetSdkVersion don't seem to matter. I've set both to 4 and I still get hardware acceleration reported.

 

[Informatix]

As long as your target is running API 11 (Honeycomb) or later you can enable hardware acceleration by a manifest entry.

That won't change anything for the B4A Canvas (it can't be hardware accelerated).
And I'm not even sure you have to enable the hardware acceleration for OpenGL, since OpenGL is always hardware accelerated.

 

[agraham]

You probably know better than me as I tend to stay away from GUI stuff, but the documentation seems to imply that it affects all Views though not all operations on those Views. I'm probably misunderstanding it or there are further qualifications that I am not aware of.

The easiest way to enable hardware acceleration is to turn it on globally for your entire application. If your application uses only standard views and Drawables, turning it on globally should not cause any adverse drawing effects.

If a B4A Canvas cannot be hardware accelerated I would be interested if you know why as I can see only one Canvas class in Android and the B4A Canvas is an instance of it. The documentation seems to imply that Canvas automatically gets accelerated.

When hardware accelerated, the 2D rendering pipeline supports the most commonly used Canvas drawing operations as well as many less-used operation

The manifest entry certainly affects whether a glSurfaceView reports that it is being hardware accelerated or not but I'm getting out of my depth here

 

[bloxa69]

As long as your target is running API 11 (Honeycomb) or later you can enable hardware acceleration by a manifest entry.

The problem is I must cover all SDKs starting with 7, as there is a lot of those devices still, so enabling hardware acceleration alone doesn't cut it, it has to be some kind of OpenGL canvas or surface. As I mentioned, I have to actually write 2 independent apps and wrap them in one .apk to cover both older and newer SDKs, as Erel's GameView is much smoother but it doesn't work properly below SDK 11.

 

[Informatix]

You probably know better than me as I tend to stay away from GUI stuff, but the documentation seems to imply that it affects all Views though not all operations on those Views. I'm probably misunderstanding it or there are further qualifications that I am not aware of.


If a B4A Canvas cannot be hardware accelerated I would be interested if you know why as I can see only one Canvas class in Android and the B4A Canvas is an instance of it. The documentation seems to imply that Canvas automatically gets accelerated.


The manifest entry certainly affects whether a glSurfaceView reports that it is being hardware accelerated or not but I'm getting out of my depth here

All functions and classes are not in the API. You won't find any mention in the documentation of the GLES20Canvas

 

[agraham]

Yes but the documentation explicitly says

When hardware accelerated, the 2D rendering pipeline supports the most commonly used Canvas drawing operations as well as many less-used operations

Where "Canvas" is a link to the android.graphics.Canvas documentation so implying that a normal canvas will be hardware accelerated

Also

Starting from Android 3.0 (API level 11), a hardware-accelerated OpenGL renderer is available to applications, to improve performance for many common 2D graphics operations. When the hardware-accelerated renderer is enabled, most operations in Canvas, Paint, Xfermode, ColorFilter, Shader, and Camera are accelerated. This results in smoother animations, smoother scrolling, and improved responsiveness overall, even for applications that do not explicitly make use the framework's OpenGL libraries.

Again mentioning Canvas.

 

[Informatix]

Yes but the documentation explicitly says
Where "Canvas" is a link to the android.graphics.Canvas documentation so implying that a normal canvas will be hardware accelerated

Also
Again mentioning Canvas.

I read it, yes, but when I get the type of the B4A Canvas on my Nexus 7 with the hardware acceleration enabled, I get:
android.graphics.Canvas

Dim r As Reflector
r.Target = C
r.Target = r.GetField("canvas")
Log(GetType(r.Target))

When I get the type of my accelerated Canvas, I get:
android.view.GLES20RecordingCanvas
It's a subclass of android.view.GLES20Canvas.

The canvas returned by a view in the onDraw event is obviously different from a canvas instance of the Canvas class.

EDIT: GameView uses the GLES20Canvas.

EDIT2: http://stackoverflow.com/questions/9588776/canvas-in-surfaceview-hardware-acceleration

 

[agraham]

That's interesting, though I'm not sure it's hard evidence that a normal Canvas isn't accelerated. The documentation I've read, and some posts by Romain Guy of the Android team, all seem to imply that a Canvas uses the OpenGL pipeline for 2D operations when acceleration is enabled and I wouldn't have thought the documentation would be that wrong.

Maybe the "hardware-accelerated renderer" can paint onto a normal Canvas. However I can't get excercised enough to start digging into the Android code base to find out, as like I said I normally avoid GUI stuff if I can.

 

[Informatix]

That's interesting, though I'm not sure it's hard evidence that a normal Canvas isn't accelerated. The documentation I've read, and some posts by Romain Guy of the Android team, all seem to imply that a Canvas uses the OpenGL pipeline for 2D operations when acceleration is enabled and I wouldn't have thought the documentation would be that wrong.

Maybe the "hardware-accelerated renderer" can paint onto a normal Canvas. However I can't get excercised enough to start digging into the Android code base to find out, as like I said I normally avoid GUI stuff if I can.

One thing is sure: the B4A CanvasWrapper class is not hardware accelerated. I can easily compare its performance with my AcceleratedSurface, and it's inferior.

 

[Erel]

Canvas is not hardware accelerated. GameView is however hardware accelerated and if there is a need for it I can add a Draw event that will expose an accelerated Canvas.

 

[Jim Brown]

Yes please Erel. That will be a welcome move.

 

[Informatix]

Yes please Erel. That will be a welcome move.

I released my AcceleratedSurface lib.

 

[Informatix]

Yes please Erel. That will be a welcome move.

Any ideas about the poor performance that I noticed?

 

[Jim Brown]

I would say this is because of the software-driven canvas as discussed. For performance I took many steps to ensure things remain efficient. For example, using the same image so that OpenGL surface usage is a minimum, and culling the backface, which would never normally be seen. Although doing that stopped the Image flip from working. The actual OpenGL drawing operations are pretty minimal I believe.

You talked about a FPS timing issue in my smileys demo a few posts back but when I checked my drawing loop I am doing the same as you. That is, checking how much time has passed once the complete draw cycle is finished.

 

[Informatix]

I would say this is because of the software-driven canvas as discussed.

I don't see any Canvas in your code.

You talked about a FPS timing issue in my smileys demo a few posts back but when I checked my drawing loop I am doing the same as you. That is, checking how much time has passed once the complete draw cycle is finished.

In your Performance example, you count the number of times you enter into the Timer1_Tick sub. That could be fine if you entered the same number of times in the glsv_Draw sub, but it's not the case despite your RequestRender call, so your count is completely wrong. Move your code in the glsv_Draw sub and you'll see. I have not this problem in my AcceleratedSurface example.

Here's a log showing the problem (I traced the calls with 200 smileys):
TICK
TICK
DRAW
TICK
TICK
TICK
TICK
DRAW
TICK
TICK
TICK
TICK
TICK
TICK
TICK
TICK
TICK
TICK
DRAW
TICK
TICK
TICK
TICK
DRAW
TICK
TICK
TICK
TICK
TICK
TICK
DRAW
TICK
TICK
TICK
TICK
TICK
DRAW
TICK
DRAW
TICK
TICK
TICK
TICK
TICK
TICK

 

[Jim Brown]

Regarding surfaces, I can only assume Andrew's GLSurfaceView is software at this time. Hence the slower performance
OpenGL - SurfaceView

That could be fine if you entered the same number of times in the glsv_Draw sub, but it's not the case despite your RequestRender call

That's interesting. I assumed the rendering happens every time glsv.RequestRender is called, since I initialize the GL surface view via RENDERMODE_WHEN_DIRTY. The document info says:

RENDERMODE_WHEN_DIRTY As Int
The renderer only renders when the surface is created, or when requestRender() is called

 

[agraham]

I can only assume Andrew's GLSurfaceView is software at this time.

No, it's a VERY thin wrapper over the standard API and has no determination within it whether hardware acceleration is used or not. However Informatix knows far more about this than I do and is best placed to comment.