B4A Library External Displays

Android has had official support for external displays since version 4.2.
Prior to version 4.2 some devices had hardware that was capable of outputting a display signal to an external display but control of this output was implemented by the device manufacturer.
Generally the external display would mirror the device's primary display and that was it.

There was no standard API to control external displays until android version 4.2.
Android version 4.2 introduced the Presentation class.
A Presentation is a container for displaying a user interface on an external display.

An external display could now be configured to display any custom user interface - as well as still being able to simply mirror the device's primary display.

An external display is a display that is temporarily connected to a device.
Examples are:

• TV or projector connected to device using an HDMI cable or MHL adapter.
• TV or projector connected to device using the Miracast wireless protocol.
  This includes a Samsung device using Samsung's AllShare Cast technology - AllShare Cast is Samsung's proprietary implementation of the Miracast protocol.
• TV or projector connected to device using a SlimPort cable.

Whereas a primary display is a display that the device normally uses for output.
A TV or projector connected to an android TV stick or OUYA device is still a primary display and not an external display as it is the display that the device normally uses for output.

A Chromecast device is not presently classified as an external display.
A Chromecast device is a media streaming device that can be told to playback a media resource - it will retrieve and display that media resource.
The Chromecast can be told to display a media resource by various methods: an android or iOS application or the desktop Chrome browser can tell a Chromecast device what media resource to display.
But the Chromecast is not currently an external display - i say currently as there is the possibility that Google will update the Chromecast firmware so that it behaves as an external display - but that is just a possibility and may or (more likely) may not happen.

But in this thread i'll publish my attempts to wrap the parts of the android API that allow control over both external displays and Chromecast devices.

If you do not have a device which you can connect to an external display then you are not out of luck - as long as your device is running android 4.2 or later.
In your device's Developer Options under the Drawing category you should find an option to Simulate secondary displays.
Give that option a click and you'll be presented with a list of available emulated secondary displays.
Select an option from that list and a small window will be overlaid on the device display - that small window is a software emulated external display and can be used as an alternative to a real external display.
You can tap and drag the window around the screen so as to not obscure a part of the device's main screen that you need to interact with.

I'll keep this thread tidy by attaching all library files to this first post.

Martin.

 

[warwound]

Ahh!

On my S3 i have to use the EZCastapp to connect to the Measy: enable mirroring and go thru the Settings to Screen Mirroring and connect.
Now if i run the demo b4a project the b4a app finds the Measy external display and eveything works.

It could be that the Measy doesn't support automatically connecting (programmatically) or Samsung's implementation of Miracast doesn't or both.

Martin.

 

[PaulR]

++Ahh!

When I do that, the desktop is mirrored so everything is on the TV anyway. I'll need to play around with it more, but to mirror without using EZCast I need to boot (read connect the power) to the dongle for it to be discoverable (although it is listed as paired in Android), with the boot mode option set to mirror. I'll have a further mooch around with it... maybe with the boot option set to cast there is a period of discovery on boot for casting too. I'll experiment some more tonight and report back.

Cheers

Paul

 

[warwound]

++Ahh!

When I do that, the desktop is mirrored so everything is on the TV anyway.

But with the CWAC-Presentation b4a demo project you do see the web content being displayed on your TV instead of within an Activity in the demo app?
(The TV is not simply mirroring the android screen but displaying custom content within the Presentation object).

Martin.

 

[warwound]

Attachment ExternalDisplays-20140421.zip now removed from post #54.
The attached ExternalDisplays-20140422.zip replaces it.

The b4a demo and cwac-mediarouter-res folder have not changed.
Only the ExternalDisplays library files have been updated.

Open ExternalDisplays.html in your browser and you'll see that the MediaRouterRouteInfo object has a couple of new methods: SupportsControlAction and SupportsControlCategory.
And the ControlIntent object (which is just a container for constants that don't belong anywhere else) has a bunch of new ACTION_???? and ERROR_???? constants defined.
The ACTION_???? and CATEGORY_???? constants can be passed as parameters in the new SupportsControlAction and SupportsControlCategory methods.
And the ERROR_???? constants are presumbly used to identify errors passed in events such as ItemActionError and SessionActionError.

Hopefully we can between us start to work out what is what and how to do what we want to do...

Martin.

The latest version of the ExternalDisplays library is now to be found attached to post #1 in this thread.

 

[PaulR]

But with the CWAC-Presentation b4a demo project you do see the web content being displayed on your TV instead of within an Activity in the demo app?
(The TV is not simply mirroring the android screen but displaying custom content within the Presentation object).

Martin.

That depends.... in the Wireless Display section, the EZCast is in a 'Paired displays' group and when it is also Connected it mirrors the screen. The EZCast software works when it is just Paired but the CWAC library does not, this is the state the feedback I posted were from. When I choose the mirroring option, the state changes to Connected and everything on the phone display is mirrored.

The result in the app in this case is that pressing an option on the list view results in no change of view on either the device or the external display. The phone display is always the same as the external display.

 

[thedesolatesoul]

Hi warwound,
Thanks a lot, it is now working great on my GS3. So far I have seen no issues whatsoever.
So far so good in the RemotePlaybackClient. I think only the ability to find the seek position in the media is missing, now sure if this has to be polled with GetStatus or GetSessionStatus.

Also, I am looking into the Google Cast API as well, do you know which class exposes the CastDevice and how to get it? Apparently one of the MediaRouterAdapater's raise it in an onConnected event, but I cant find it.

EDIT:
Apparently the CastDevice is in the 'extras' of the MediaRouteInfo. To extract it we need to use:
CastDevice device = CastDevice.getFromBundle(theRoute.getExtras());
This allows us to create a GoogleCastApiClient from:

    Cast.CastOptions.Builder apiOptionsBuilder = getCastOptionBuilder(mSelectedCastDevice);
            mApiClient = new GoogleApiClient.Builder(mContext)
                    .addApi(Cast.API, apiOptionsBuilder.build())
                    .addConnectionCallbacks(this)
                    .addOnConnectionFailedListener(this)
                    .build();

Then we can later use the onConnect event to get more information like Cast.CastAPI.getApplicationMetadata.

If you can return the extras for me, I will try to wrap the remaining code into another helper library (can be merged later).

 

[warwound]

The result in the app in this case is that pressing an option on the list view results in no change of view on either the device or the external display. The phone display is always the same as the external display.

Hmm that's not what should happen.

Take a look at the attached photo of my S3 connected via my Measy to a TV.
The library demo app has detected the connected TV, created a Presentation object and set the TV to display the Presentation object.
The Presentation object displays a WebView and the WebView loads webpages based on the ListView click.

@tds I'm sure that using the Google Cast SDK also requires that you register your app with Google and pay the $5 fee, have a read of this page: https://developers.google.com/cast/docs/developers.
Scroll down to Getting Started and look at item #4 'Register your app'.
You need an API key to use Google Cast SDK and only get an API key by registering your app.
That might or might not be a problem - what's $5 lol.

I'd estimate that there's at least 4 different techniques that can be used to connect an application to an external display or Chromecast.
There's relatively straightforward techniques as used in the existing CWAC-Presentation and ExternalDisplays libraries - these require no API key.
And the more complex techniques which offer much more control but require both an API key (and fee!) and more importantly much more time to research the API docs and learn what code to write.

It'd be a shame if this thread becomes fragmented with many libraries that can do one task instead of a single library that can do all tasks...

Martin.

 

[bluedude]

I'm already confused right now Can this library be used already to show a webview on Chromecast? Post 67 confuses me.

Although I think it would be good to have one library it makes sense to have one for Chromecast too because I bet this is the best sold device.

Can we use the Presentation object for the Chromecast?

 

[warwound]

I'm already confused right now Can this library be used already to show a webview on Chromecast? Post 67 confuses me.

Although I think it would be good to have one library it makes sense to have one for Chromecast too because I bet this is the best sold device.

Can we use the Presentation object for the Chromecast?

No my post was a reply to post #65 where PaulR reported that running the CWAC-Presentation demo app and using his Measy dongle didn't result in a WebView being displayed on the TV while a ListView was displayed on the device screen.

The Chromecast is just a media streaming device - any content you want it to display must be served by a server which the Chromecast can connect to and stream from.
Can you create a server in your b4a application and stream a live screen capture of some or part of the android device screen to the Chromecast?
That's about the only way i can see a Chromecast being able to display some or all of the content from your b4a application activity.

Martin.

 

[thedesolatesoul]

@tds I'm sure that using the Google Cast SDK also requires that you register your app with Google and pay the $5 fee, have a read of this page: https://developers.google.com/cast/docs/developers.
Scroll down to Getting Started and look at item #4 'Register your app'.
You need an API key to use Google Cast SDK and only get an API key by registering your app.
That might or might not be a problem - what's $5 lol.

I'd estimate that there's at least 4 different techniques that can be used to connect an application to an external display or Chromecast.
There's relatively straightforward techniques as used in the existing CWAC-Presentation and ExternalDisplays libraries - these require no API key.
And the more complex techniques which offer much more control but require both an API key (and fee!) and more importantly much more time to research the API docs and learn what code to write.

It'd be a shame if this thread becomes fragmented with many libraries that can do one task instead of a single library that can do all tasks...

Martin.

Hi Martin,
I have been reading and reading and reading on these APIs and now looking at the Google Cast API. To me, still nothing is really clear so still experimenting and since there are so many ways to do things I am ping ponging around reading code.
You are correct, you need an API Key for Google Cast SDK but only to register a custom/styled receiver, and to enable developer debugging on the chromecast. I dont think you should need it if your receiver is not custom? (While I am trying to use the CastAPI but not creating a receiver I shouldnt need to pay?, but ill see if i have to)
From what you have wrapped so far, using the RemotePlaybackClient, I am unable to connect to an existing session (from a different app).
That I believe can only be done using the CastAPI using either joinApplication or launchApplication.
However, that is a ridiculously large amount of code to wrap and requires a lot of error handling and fluff logic around. And also I do not know if it will work with what I want to do either!
This is why I dont want to waste your effort wrapping into it just yet, till I actually manage to get it to work.

The Chromecast is just a media streaming device - any content you want it to display must be served by a server which the Chromecast can connect to and stream from.
Can you create a server in your b4a application and stream a live screen capture of some or part of the android device screen to the Chromecast?
That's about the only way i can see a Chromecast being able to display some or all of the content from your b4a application activity.

Martin.

This is correct. From what I have heard, you need to create a small webserver on your app to stream even local files. But I guess your receiver needs to be custom to be able to do this.

 

[PaulR]

Hmm that's not what should happen.

Take a look at the attached photo of my S3 connected via my Measy to a TV.
The library demo app has detected the connected TV, created a Presentation object and set the TV to display the Presentation object.
The Presentation object displays a WebView and the WebView loads webpages based on the ListView click.

Thanks for the info... can I just ask whether your screen is mirroring to the external display before running the example app or is the only thing cast the web view when the Presentation object is active?

I can see from searching the wireless display subject that other devices have a wireless display on/off setting and an additional screen mirroring option. My device only has the former, so maybe the implementation on my phone skips the in between mode where casting to the screen is available and the EZCast software is enabling it? Since the EZCast software connects the phone to the dongle via WiFi (SSID+Passcode) initially I thought they were doing something completely different, but switching off wireless display also prevents/stops screen casting through that software so it at least smells as though they are using similar parts of the SDK.

 

[warwound]

@PaulR

The Measy was mirroring the S3 screen to the TV before i started the demo app.
I started the CWAC-Presentation app and it immediately recognised the TV and used it for the display of the Presentation.
That is the PresentationHelper ShowPresentation event was raised as soon as the demo app started.

I'd previously turned on the TV and Measy, gone to S3 Settings > Connect & Share > Screen Mirroring.
The S3 tried to connect to the Measy as soon as i selected Screen Mirroring - no need to select it.
Finally my Measy is set to boot in mirror mode not cast mode.

That process of connecting usually fails and i have to open the EZCast app, connect in cast mode then switch to mirror mode.
Sometimes it fails even then and i have to run TriangleAway and reboot otherwise the connection fails when the device realises it's been rooted and disallows the (secure) connection.

Just a thought - did you have another external display connected (HDMI or MHL etc) when you also had the Measy connected?
That might explain why the PresentationHelper didn't raise the ShowPresentation event...
The PresentationHelper is hardcoded to assume only one external display is connected at any one time, it handles only the first external display that the android API says it is connected to.
(Bit of a long shot!)

Martin.

 

[thedesolatesoul]

You will like this link: http://sermoa.wordpress.com/2014/04...f-chromecast-using-cucumber-jruby-and-rukuli/

 

[moster67]

Hi,

just started trying to get the demo-project to compile but I get errors as follows:

Parsing code.                          0.00
Compiling code.                        0.01
Compiling layouts code.                0.00
Generating R file.                      Error
C:\Android\android-sdk\extras\android\support\v7\mediarouter\res\layout-v17\mr_media_route_list_item.xml:22: error: No resource identifier found for attribute 'paddingStart' in package 'android'
C:\Android\android-sdk\extras\android\support\v7\mediarouter\res\layout-v17\mr_media_route_list_item.xml:22: error: No resource identifier found for attribute 'paddingEnd' in package 'android'
C:\Android\android-sdk\extras\android\support\v7\mediarouter\res\layout-v17\mr_media_route_list_item.xml:22: error: Error: No resource found that matches the given name (at 'paddingStart' with value '?android:attr/listPreferredItemPaddingStart').
C:\Android\android-sdk\extras\android\support\v7\mediarouter\res\layout-v17\mr_media_route_list_item.xml:22: error: Error: No resource found that matches the given name (at 'paddingEnd' with value '?android:attr/listPreferredItemPaddingEnd').

In the activity-code I replaced the original AdditionalRes with my paths:

#AdditionalRes: C:\Android\android-sdk\extras\android\support\v7\appcompat\res, android.support.v7.appcompat
    #AdditionalRes: C:\Android\android-sdk\extras\android\support\v7\mediarouter\res, android.support.v7.mediarouter
    '    #AdditionalRes: C:\Users\martin\Programming\adt-bundle-windows-x86_64-20131030\sdk\extras\google\google_play_services\libproject\google-play-services_lib\res, com.google.android.gms
    #AdditionalRes: C:\Users\mikael.NTB\Downloads\ExternalDisplays-20140422\cwac-mediarouter-res, com.commonsware.cwac.mediarouter

I have updated the SDK with latest data available. In addition, I copiedthe B4A-wrapper libraries to my extra-library folder.

Any ideas what I am doing wrong?

 

[warwound]

I've now updated the CWAC-Presentation library.

Two new objects: DisplayManager and DisplayManagerDisplayListener have been added.
These two objects are used (internally) by the PresentationHelper object to detect connection and disconnection of external displays.
So that covers HDMI and MHL cable connected displays and Miracast connected displays too.

First thing i'll say is i'm disappointed(lols), my dreams of a triple monitor S3 are over..
The S3 supports only one external display at a time!
With my MHL cable attached i tried to also connect to my Miracast Measy dongle.
A dialog appeared stating:

Unable to start Screen Mirroring.
Disconnect the HDMI cable and try again.

I've created another b4a demo project to demonstrate usage of these new objects and attached it to this post.

Once the demo is started a Service runs listening for DisplayAdded, DisplayChanged and DisplayRemoved events.
An Activity displays a ListView of all currently available displays - and that includes the android device's built in display.
So i've been connecting and disconnecting my MHL cable, and connecting to and disconnecting from my Measy dongle to establish how reliably a change in displays can be detected.
This is easy with the Measy connection but when the MHL cable is connected i am no longer connected via USB so cannot monitor the log output of the Service.
So the Service displays Toast messages when either DisplayAdded or DisplayRemoved events are raised.

On my S3 the events all seem to be raised as expected - though trying to connect both MHL and Measy caused the S3 to reboot one time and another time caused the Measy to display nothing but a green screen.
The raising of events was unpredictable until i again rebooted my S3 and this time didn't try to connect to more than one external display at a time.

The new demo is a way to debug whether your device is correctly detecting changes in connected displays - it should be useful for @PaulR unless he has fixed his connection problem already.
@PaulR if you run the demo then run the EZCast app and use the 'cast' mode you'll not see any events raised when EZCast connects in cast mode to the Measy.
Switch to mirror mode and you should see the DisplayAdded event raised.
This might indicate that EZCast isn't using the standard android API methods to display content on the external display?

The demo could soon be updated to create Presentation objects and display content on external displays. The demo source code shows how to get a Display object from the DisplayManager and a Display object is all you need to initialize a Presentation object.

I'm curious as to whether or not we can create a Presentation in a Service module and then create other Views in the Service module and add them to the Presentation.
Logic says not - Views and Presentation objects are Activity objects.
But anyone who's played with the Standout library will know that it is possible to create Views in a Service module.
So i might experiment with that and see what happens.

@moster67 I could only think that your android SDK files are not up to date.
But you say you have checked that.
Maybe the version of the android.jar configured in the b4a IDE is not recent enough - can you switch to android API 19 android.jar and try again?

Updated CWAC-Presentation library attached to post #1 and DisplayManagerDemo project attached to this post.
(I've also moved the latest ExternalDisplays library files to post #1).

Martin.

 

[PaulR]

@warwound

Thanks for the info. Unfortunately I don't have another display connected since my phone doesn't have an HDMI/MHL connector.

Would it be possible to use Reflection/Java Object to use the getDisplay(s) methods from DisplayManager? If pointed in the right direction I could do some more testing to make sure nothing funny is happening on that front, because it does seem as though when the screen is being mirrored that an external display is being used since nothing changes on the screen.

edit: this reply was to previous post, not the one directly above. I'll give the demo a whirl.... (which will be later tonight....)

 

[warwound]

I just tried to create a Presentation in the Service module - it force closes the app!
Oh well i wasn't really expecting it to work anyway.

 

[PaulR]

In the new demo I notice that DisplayManager and DisplayManagerDisplayListener are unknowns types... how can this be resolved? edit: I'm on CWAC-Presentation v0.20.. upgrading to 0.3... edit2: yes, that was it.

 

[warwound]

Did you download the 0.3 update from post #1?

 

[bluedude]

Take a look here: http://joaoapps.com/autocast/ This guy seems to be able to cast anything from his Android phone. He actually casts notifications from his phone.

Cheers,