B4A Library External Displays

Android has had official support for external displays since version 4.2.
Prior to version 4.2 some devices had hardware that was capable of outputting a display signal to an external display but control of this output was implemented by the device manufacturer.
Generally the external display would mirror the device's primary display and that was it.

There was no standard API to control external displays until android version 4.2.
Android version 4.2 introduced the Presentation class.
A Presentation is a container for displaying a user interface on an external display.

An external display could now be configured to display any custom user interface - as well as still being able to simply mirror the device's primary display.

An external display is a display that is temporarily connected to a device.
Examples are:

• TV or projector connected to device using an HDMI cable or MHL adapter.
• TV or projector connected to device using the Miracast wireless protocol.
  This includes a Samsung device using Samsung's AllShare Cast technology - AllShare Cast is Samsung's proprietary implementation of the Miracast protocol.
• TV or projector connected to device using a SlimPort cable.

Whereas a primary display is a display that the device normally uses for output.
A TV or projector connected to an android TV stick or OUYA device is still a primary display and not an external display as it is the display that the device normally uses for output.

A Chromecast device is not presently classified as an external display.
A Chromecast device is a media streaming device that can be told to playback a media resource - it will retrieve and display that media resource.
The Chromecast can be told to display a media resource by various methods: an android or iOS application or the desktop Chrome browser can tell a Chromecast device what media resource to display.
But the Chromecast is not currently an external display - i say currently as there is the possibility that Google will update the Chromecast firmware so that it behaves as an external display - but that is just a possibility and may or (more likely) may not happen.

But in this thread i'll publish my attempts to wrap the parts of the android API that allow control over both external displays and Chromecast devices.

If you do not have a device which you can connect to an external display then you are not out of luck - as long as your device is running android 4.2 or later.
In your device's Developer Options under the Drawing category you should find an option to Simulate secondary displays.
Give that option a click and you'll be presented with a list of available emulated secondary displays.
Select an option from that list and a small window will be overlaid on the device display - that small window is a software emulated external display and can be used as an alternative to a real external display.
You can tap and drag the window around the screen so as to not obscure a part of the device's main screen that you need to interact with.

I'll keep this thread tidy by attaching all library files to this first post.

Martin.

 

[warwound]

It might be a good idea for us to report what android version(s) we're testing this with...

• I originally tested on my Moto G which has KitKat 4.4.2 - everything works as expected.
  Only my Chromecast shows up in the MediaRouteButton select a device dialog.
• Just now tested on my S3 4.3, the MediaRouteButton dialog shows both my Chromecast and my Measy miracast dongle in it's device selection dialog.
  Selecting my Chromecast connects but the RouteSelected event is not raised.
  Another click on the MediaRouteButton gives me the volume and disconnect options - so i'm sure that my S3 has connected.
  I can disconnect and reconnect but the RouteSelected event is still never raised.
  Next i ran the official Play Store Chromecast application, it detected my Chromecast.
  I've already run this application many times in the past week or so and i've used the Chromecast on my S3 many times - this wasn't the first time i'd tried to use the S3 and Chromecast.
  Anyway i started the demo project again and now, after connecting, the RouteSelected event is raised and i can play, pause, resume and stop the demo video.
  All these tests were done using Release mode.
  Next i tested the 2 Debug modes (rapid and legacy) - everything continues to work as expected.
  Finally i tried Obfuscated mode and again the RouteSelected event is not raised.
  Reverted to Release and each Debug mode - now RouteSelected is not raised.
  Ran the Chromecast app, still no RouteSelected event raised.
  Rebooted S3 and ran the demo (compiled in Release mode) - no RouteSelected event raised.
  Ran the Chromecast app and then the demo again - now the RouteSelected event is raised.
  Disconnected and re-compiled using Obfuscated mode - no RouteSelected event raised.

So there's some research for me to do.
Presumably sometimes the demo app is not completely releasing the connection to the Chromecast when exited and running the demo a second time might then fail.
And Obfuscated mode for me on the S3 is failing - maybe someone else can try Obfuscated mode and see if it works.

As for the NullPointerException, not sure what's happened here but could be the library is failing to find a resource required for the MediaRouteButton - a resource not found will often result in a NullPointerException BUT also raise a more detailed entry in the log.

Martin.

 

[warwound]

@DSD Meant to ask what device are you running the demo on?

 

[warwound]

And to confuse things a little...

Obfuscated mode works fine on my Moto G!

Martin.

 

[DSD]

@DSD Meant to ask what device are you running the demo on?

I'm running it on:

Sony Xperia Z
Sony Xperia Tablet Z
Both running Android 4.3

Sony Xperia Tablet S running Android 4.1.1

I get force close on all 3.
But I'm starting to think that I've missed something...

 

[thedesolatesoul]

I am running on a Galaxy S3 with 4.1
I have the EXACT same problem as @warwound. I havent figured out what causes it to stop working and start working again, but it is definitely flakey and the Route_Selected event is not raised. I have always tested in Release mode.

Also @DSD in Release mode you should get the full stack track in the logs. Sometimes in debug mode it doesnt show the full stack trace. I actually saw this same exception in the first or second time I ran the app, but I cannot reproduce it anymore.

 

[PaulR]

4.2.1
Star N9599T
Measy A2W Miracast dongle - EZCast working

Thanks for the work here, I am running the original CWAC Presentation library/example on 4.2.1 and getting the following 2 errors....

LogCat connected to: B4A-Bridge: alps 01v21_v89_gq3008s_89t-356876010527086
--------- beginning of /dev/log/main
Copying updated assets files (3)
** Activity (main) Create, isFirst = true **
** Activity (main) Resume **
GC_CONCURRENT freed 1360K (14384), 32% free 3161K/4648K, paused 2ms+1ms, total 13ms
Unexpected event (missing RaiseSynchronousEvents): presentationhelper1_clearpresentation
Check the unfiltered logs for the full stack trace.
PresentationHelper1_ClearPresentation
create interp thread : stack size=32KB
create new thread
new thread created
update thread list
threadid=15: interp stack at 0x5deed000
threadid=15: created from interp
start new thread
threadid=15: notify debugger
threadid=15 (Thread-355): calling run()
create interp thread : stack size=32KB
create new thread
new thread created
update thread list
threadid=16: interp stack at 0x5dff5000
threadid=16: created from interp
start new thread
threadid=16: notify debugger
threadid=16 (java.lang.ProcessManager): calling run()
** Activity (main) Pause, UserClosed = false **
PresentationHelper1_ClearPresentation
Error occurred on line: 60 (main)
java.lang.NullPointerException
    at uk.co.martinpearman.b4a.externaldisplaysdemo.main._presentationhelper1_clearpresentation(main.java:413)
    at java.lang.reflect.Method.invokeNative(Native Method)
    at java.lang.reflect.Method.invoke(Method.java:511)
    at anywheresoftware.b4a.shell.Shell.runMethod(Shell.java:636)
    at anywheresoftware.b4a.shell.Shell.raiseEventImpl(Shell.java:302)
    at anywheresoftware.b4a.shell.Shell.raiseEvent(Shell.java:238)
    at anywheresoftware.b4a.shell.Shell$1.run(Shell.java:276)
    at android.os.Handler.handleCallback(Handler.java:725)
    at android.os.Handler.dispatchMessage(Handler.java:92)
    at android.os.Looper.loop(Looper.java:153)
    at android.app.ActivityThread.main(ActivityThread.java:5297)
    at java.lang.reflect.Method.invokeNative(Native Method)
    at java.lang.reflect.Method.invoke(Method.java:511)
    at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:833)
    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:600)
    at dalvik.system.NativeStart.main(Native Method)
threadid=15: exiting
threadid=15: bye!
create interp thread : stack size=32KB
create new thread
new thread created
update thread list
threadid=15: interp stack at 0x5deed000
threadid=15: created from interp
start new thread
threadid=15: notify debugger
threadid=15 (Thread-357): calling run()
--onServiceStateChanged,state=SIM2 0 home O2 -UK 23410 23410  HSPA CSS not supported -1 -1 RoamInd=-1 DefRoamInd=-1 EmergOnly=false Regist state: 1

... I'll give the new app a whirl and get back, but I thought the null pointer exception in that case was worth posting.

 

[PaulR]

I'm getting a compilation error with the ExternalDisplaysDemo. I've changed the Project Attributes section to point at the correct places on my system and the sdk\extras\android\support\v7\mediarouter\res directory exists, but I'm getting the following error from line 54...

MediaRouter1.AddCallback(MediaRouteSelector1, MediaRouterCallback1, MediaRouter1.CALLBACK_FLAG_REQUEST_DISCOVERY)

...

B4A line: 54
MediaRouter1.AddCallback(MediaRouteSelector1, MediaRouterCallback1, 
MediaRouter1.CALLBACK_FLAG_REQUEST_DISCOVERY)
javac 1.8.0_05
src\uk\co\martinpearman\b4a\externaldisplaysdemo\main.java:340: error: package
 android.support.v7.media does not exist
_mediarouter1.AddCallback((android.support.v7.media.MediaRouteSelector)
(_mediarouteselector1.getObject()),(android.support.v7.media.MediaRouter.Callback)
(_mediaroutercallback1.getObject()),_mediarouter1.CALLBACK_FLAG_REQUEST_DISCOVERY);

A point towards the correct dependency or origin of the error would be appreciated.

 

[warwound]

I see a report HERE that complains about the MediaRouter.Callback callbacks not being executed.
(The b4a RouteSelected and RouteUnselected events are raised when the MediaRoute.Callbacks callbacks are executed).

A reply on that page, dated March 10 2014 states:

There is a bug in the latest version of Android SDK/Play services that causes the behavior you are seeing. This will be fixed in the next release of SDK and Google Play services.

It doesn't specifically say whether the bug is in the android SDK or the Play Services library so it's impossible to say whether the bug has been fixed or is outstanding.

This is all related to my tests with my S3 reported earlier where the events were not raised, more tests on my Tab2 running android 4.2.2 suffer the same problem - the RouteSelected and RouteUnselected events are not raised.

My Moto G running android 4.4.2 everything seems fine so far.

@DSD How are you connected to your device? Via USB or B4ABridge?
Could be that the logs not showing the complete exception?

@PaulR
Can you try the attached CWAC-Presentation library updates.
It might be that the library is not thread safe and the b4a events are being raised from a non UI thread, if that's the problem then this update should fix it.

Martin.

 

[warwound]

@PaulR
First i'll assume that you've run the android SDK manager and ensured that your SDK is up to date.

Next look in your b4a additional libraries folder.
Can you see these three files:

• android-support-v4.jar
• android-support-v7-appcompat.jar
• android-support-v7-mediarouter.jar (this seems to be missing based on the exception)

If so are they copies of the latest versions from the android SDK folder locations?
(Updating the android SDK doesn't update anything you've copied to your b4a additional libraries folder so make sure this folder contains copies of the latest versions).

Martin.

 

[thedesolatesoul]

Also, I am wondering once you have casted media to the CC, shouldnt running the app again connect to the same route and the same session? I believe on each update (like RPC status changed) the sessionid is updated.
How to we make sure opening the app again will resume from the same point i.e. the routeinfo is saved somewhere?

 

[warwound]

Also, I am wondering once you have casted media to the CC, shouldnt running the app again connect to the same route and the same session? I believe on each update (like RPC status changed) the sessionid is updated.
How to we make sure opening the app again will resume from the same point i.e. the routeinfo is saved somewhere?

The RemotePlaybackClient has methods GetSessionId and SetSessionId.
I'd guess when you start a new session you keep a reference to the session id - you'd get the new session id in the SessionActionResult event if StartSession has succeeded - and to resume a session you'd call SetSessionId before calling StartSession.

Martin.

 

[bluedude]

Started SDK Manager and looked for these files but only thing I have is the support-v4.jar.

Does anyone know what I need to check in SDK manager to get these files?

 

[warwound]

Had to check myself there - i've long forgotten - just installing the package 'Android Support Library' should do it.
More info can be found here: http://developer.android.com/tools/support-library/features.html#v7-appcompat.

Martin.

 

[warwound]

I've just removed the attachment from post #23 (that was ExternalDisplays-20140420.zip).
And attached is an updated b4a project and library files.

• MediaRouterCallback can no longer be initialized.
  Instead it is returned by the (updated) MediaRouter methods:
  • AddCallback (EventName As String, MediaRouteSelector1 As MediaRouteSelector, Flags As Int) As MediaRouterCallback
    Registers a MediaRouterCallback to discover routes that match the selector and to receive events (defined by Flags) when they change. Returns a reference to the MediaRouterCallback which can be passed to RemoveCallback to remove the listener.
  • AddCallback2 (EventName As String, MediaRouteSelector1 As MediaRouteSelector) As MediaRouterCallback
    Registers a MediaRouterCallback to discover routes that match the selector and to receive events when they change. Returns a reference to the MediaRouterCallback which can be passed to RemoveCallback to remove the listener.
• MediaRouterCallback1 is now added as a callback in Activity_Resume and removed in Activity_Resume.
  This seems to have fixed the problem with my Samsung devices where no MediaRouterCallback events were raised.
  Both my S3 and Tab2 now connect, raise an event and allow playback.
  @tds said he has a an S3, try this update tds and see if it now works.

Currently i'm updating the b4a project to try and resume a previously started session - this isn't working yet - just wanted to get this update online so you can all try it.

Still not sure what to suggest DSD's NullPointerException...

Martin.

 

[DSD]

I still get the java.lang.NullPointerException with the new test project.
It now occurs on line 52: MediaRouteButton1.Initialize

Under Configure Paths and Additional libraries I've added the following jar-files:
Maybe this could be the reason for my NullPointerException...

android-support-v4.jar
android-support-v7-appcompat.jar
android-support-v7-mediarouter.jar
google-play-services.jar

Should the android-support-v4 be higher e.g. v7?

 

[warwound]

Should the android-support-v4 be higher e.g. v7?

Nope!

 

[PaulR]

@PaulR
First i'll assume that you've run the android SDK manager and ensured that your SDK is up to date.

Next look in your b4a additional libraries folder.
Can you see these three files:

• android-support-v4.jar
• android-support-v7-appcompat.jar
• android-support-v7-mediarouter.jar (this seems to be missing based on the exception)

If so are they copies of the latest versions from the android SDK folder locations?
(Updating the android SDK doesn't update anything you've copied to your b4a additional libraries folder so make sure this folder contains copies of the latest versions).

Martin.

Cheers, I didn't realise I had to put the jar files in the library folder.

The null pointer error is now gone with the updated libraries for the CWAC-Presentation example but I am not getting anything on the external display (nor with External Displays, but that's Chromecast only, no?). Here's the log from the CWAC example....

** Activity (main) Create, isFirst = true **
** Activity (main) Resume **
PresentationHelper1_ClearPresentation
** Activity (main) Pause, UserClosed = false **
Ignoring event: presentationhelper1_clearpresentation
** Activity (geographviewer) Create, isFirst = true **
** Activity (geographviewer) Resume **
Ignoring event ClearPresentation: no callback sub found

Hope that helps a bit.

 

[warwound]

The log line "Ignoring event ClearPresentation: no callback sub found" simply means the library has not found a callback sub in your activity for the "ClearPresentation" event.
As long as you have a callback sub for the "ShowPresentation" event i'd expect the library to connect.
If you have no callback subs for either of these events then the PresentationHelper will remain uninitialized even after calling it's Initialize method.

ExternalDisplays library is currently only for Chromecast devices but i hope to update it so it can be used with 'proper' external displays at some point - 'proper' external displays being displays connected via HDMI and MHL etc.

So assuming you're trying to connect to a 'proper' external display...
If you plug in your cable BUT do not run the b4a demo project do you see anything on your display?
That is: does your external display work with the device when plugged in but the CWAC-Presentation library is failing to raise the 'ShowPresentation' event?

I'll have to leave development of these libraries for a few days and get some work done but resume development in a few days.

Martin.

 

[bluedude]

Compiled it and works on my Galaxy Nexus. I can play, stop and resume. Running Android 4.3.

 

[PaulR]

So assuming you're trying to connect to a 'proper' external display...
If you plug in your cable BUT do not run the b4a demo project do you see anything on your display?
That is: does your external display work with the device when plugged in but the CWAC-Presentation library is failing to raise the 'ShowPresentation' event?

Nope, it's the same Measy EZCast dongle you have so it's all wireless. Screen mirroring is off and when the library is failing to raise the event I can go into EZCast and cast all the options from there without a problem. I don't know if they are using any proprietary magic in their app though. Hmmm.