B4A Library External Displays

Android has had official support for external displays since version 4.2.
Prior to version 4.2 some devices had hardware that was capable of outputting a display signal to an external display but control of this output was implemented by the device manufacturer.
Generally the external display would mirror the device's primary display and that was it.

There was no standard API to control external displays until android version 4.2.
Android version 4.2 introduced the Presentation class.
A Presentation is a container for displaying a user interface on an external display.

An external display could now be configured to display any custom user interface - as well as still being able to simply mirror the device's primary display.

An external display is a display that is temporarily connected to a device.
Examples are:

• TV or projector connected to device using an HDMI cable or MHL adapter.
• TV or projector connected to device using the Miracast wireless protocol.
  This includes a Samsung device using Samsung's AllShare Cast technology - AllShare Cast is Samsung's proprietary implementation of the Miracast protocol.
• TV or projector connected to device using a SlimPort cable.

Whereas a primary display is a display that the device normally uses for output.
A TV or projector connected to an android TV stick or OUYA device is still a primary display and not an external display as it is the display that the device normally uses for output.

A Chromecast device is not presently classified as an external display.
A Chromecast device is a media streaming device that can be told to playback a media resource - it will retrieve and display that media resource.
The Chromecast can be told to display a media resource by various methods: an android or iOS application or the desktop Chrome browser can tell a Chromecast device what media resource to display.
But the Chromecast is not currently an external display - i say currently as there is the possibility that Google will update the Chromecast firmware so that it behaves as an external display - but that is just a possibility and may or (more likely) may not happen.

But in this thread i'll publish my attempts to wrap the parts of the android API that allow control over both external displays and Chromecast devices.

If you do not have a device which you can connect to an external display then you are not out of luck - as long as your device is running android 4.2 or later.
In your device's Developer Options under the Drawing category you should find an option to Simulate secondary displays.
Give that option a click and you'll be presented with a list of available emulated secondary displays.
Select an option from that list and a small window will be overlaid on the device display - that small window is a software emulated external display and can be used as an alternative to a real external display.
You can tap and drag the window around the screen so as to not obscure a part of the device's main screen that you need to interact with.

I'll keep this thread tidy by attaching all library files to this first post.

Martin.

 

[warwound]

The CWAC-Presentation library's main object is the PresentationHelper.
PresentationHelper enables you to handle external displays:

• When an external display is available the event ShowPresentation(Display1 As Display) is raised.
  The Display object can be used to initialize a Presentation object.
  The Presentation object is used to display a user interface on the external display.
  Currently the BasicPresentation is the only Presentation object that i have created.
• When an external display is no longer available the event ClearPresentation(ShowInline As Boolean) is raised.
  The Presentation Dismiss method should then be called.
  ShowInline indicates whether the Activity is still in the foreground and therefore the user interface should now be displayed 'inline' on the device's primary display.

The CWAC-Presentation library is based on the open source CommonsWare PresentationHelper library: https://github.com/commonsguy/cwac-presentation

I have compiled this library using android API level 17 and tested it on:

• Galaxy Tab2 running android 4.2.2, using the developer options emulated external display.
• Galaxy S3 running android 4.3 using an MHL to HDMI adapter.
  Tested on a 720p TV and a 1080p tv.
• Moto G running android 4.4, using the developer options emulated external display.

An example project is attached:

• The Main Activity displays a ListView, each ListView item is a link to a picture i have submitted to the Geograph project.
• If an external display is detected then a Presentation object is initialized.
  The Presentation object's Show method is called.
  A WebView is added to the Presentation.
  The WebView fills the Presentation width and height.
• If an external display is no longer available and the Presentation object is initialized then the Presentation object's Dismiss method is called and the Presentation object re-dimmed.
  The Presentation object is no longer initialized.
• Now when a ListView item is clicked the code checks to see is the Presentation object is initialized.
  • If the Presentation object is initialized then the WebView within the Presentation displays the webpage containing the clicked 'geograph'.
  • If the Presentation object is not initialized then a new Activity is started.
    The new Activity displays a WebView which fills it's width and height and the WebView loads the webpage containing the clicked 'geograph'

The example project is basic but functional - it shows how you can display content on an external display if an external display is available, otherwise display the content using a traditional Activity.
It also shows how to handle an external display being connected or disconnected while your application is running - switching display of content from an Activity to a Presentation and vice versa.

Please consider this library to be currently a beta version (maybe even an alpha version), hence it's version number is 0.10.
It's not perfect...
The Presentation object displays a Panel and you add content to that Panel using the standard b4a Panel methods.

The Panel is sized to fill the Presentation width and height - if you log the Panel's width and height you'll get values of -1 x -1.
-1 being an android constant that means 'fill the parent container'.
Now if you try to use the Panel LoadLayout method and in that layout use a Designer script to size a View, for example:

WebView1.Width=100%x
WebView1.Height=100%y

Your WebView will have dimensions of 100% * -1 and be invisible.
So bear in mind that the library is not yet really ready for production use, and hopefully i'll have these issues fixed shortly.

Another thing i want to investigate is whether a Service can detect external displays being connected and display a Presentation without having to create an Activity.

• The CWAC-Presentation library files are attached to the first post in this thread, the attachment contain an HTML reference document you should find useful as well as a copy of the Apache License Version 2.0 which applies to the CommonsWare PresentationHelper library.
• The example project is attached to this post.

I'll turn my attention now to the Chromecast API...
Hopefully have something to share over the next week.

Martin.

 

[DSD]

This is very exciting and I've made a donation to you for 10 GBP, to show my appreciation.
I'm looking forward to the Chromecast API.
Keep up the good work

 

[bluedude]

Pretty interesting, now I know why we could use an MHL adapter. I still think a wireless connection is more convenient in the end

A product like Autocast for Chromecast already supports custom screens so should be possible in a library.

Cheers.

 

[walterf25]

Oh wow, thanks Warwound, i had been trying to wrap samsung's api to get the miracast working, unfortunately my knowledge about Java is very little, fortunately you created this library, and it's very impressive, thanks a lot for your work.

cheers,
Walter

 

[warwound]

@walterf25

So did you test with the Samsung 'All Share Cast' and it worked?
I was thinking of getting a miracast dongle to see how All Share Cast works on my S3 but have read that miracast performance is nowhere near as good as a cable connection so might not bother buying one.

Martin.

 

[warwound]

My miracast dongle arrived today so i thought i'd report my findings so far...

I bought a Measy A2W, just £20 from Ebay.
The dongle is advertised as being compatible with Windows/Mac OS X/iOS and Android.
To configure the dongle you need to install a client program on your device, so on my Galaxy S3 i installed the EZCast application.

I first got EZCast working in 'cast' mode, it enables me to display media, web content and other stuff on my HD TV - the EZCast app casting the content to the external display.

Next i tried the 'mirror' mode - this was the mode that was most important to me and of course it didn't work!
Followed the instructions many times, rebooted a few times but it didn't work.
Everything progressed as expected and the S3 'Screen mirroring' activity showed a toast message saying it'd successfully connected to the dongle.
But that was always immediately followed by a message box that said 'Unable to start Screen Mirroring because of a hardware issue'.
A Google showed it to be a common problem, there's a thread on xda that explains the problem:
A modified (rooted) device fails the HDCP protection check and therefore any attempts to enable screen mirroring fail.
(My S3 is rooted).
That thread shows hows to overwite the default 'libWFD_ENGINE.so' file with a modified version that has this HDCP protection check disabled.
The thread is not aimed at the S3 though and i didn't feel like bricking my S3 with a modification intended for a different Samsung device.

A bit more Googling led me to the ScreenMirroring Patch utility on the Play Store.
Seems that a lot of Samsung users have had a device with working AllShare Cast and then updated their firmware and found that AllShare Cast no longer works.
This seems to have happened with official Samsung firmware updates - it's not something that only applies to custom ROMs.
So the ScreenMirroring Patch patches some system file(s) and hopefully AllShare Cast will now work again.

And a last Google found this thread where it mentions rooted Samsung devices not working with AllShare Cast and suggests using Triangle Away to reset the device's internal 'flash counter' back to zero.
Resetting the flash counter to zero means that the device appears as though it's not been rooted and therefore doesn't fail the HDCP protection check.
Triangle Away is not free but it is only £1.99.
(Though if you're an xda forum member you can search for and find it available for free on the xda forum).

So i ran Triangle Away and the ScreenMirroring Patch and rebooted and at last my dongle worked as a 'normal' mircast dongle mirroring the device screen to my TV.

I then ran the demo project from post #2 in this thread and it worked - the CWAC-Presentation library successfully detecting the external monitor and displaying my custom content on it .

EZCast also worked perfectly in 'cast' mode on both my 7" Tab2 and Moto G but so far i've been unable to get mirror mode to work on these two devices.
Neither device officially supports miracast and both need a bit of a hack to even try to get mirror mode to work:

The Tab2 required me to install the Wifi Display helper, free from the Play Store.
The Wifi Display helper simply starts the 'Wireless display' settings activity, there's no way to start this activity by default on the Tab2.

The Moto G required a hack to the device's 'build.prop' configuration file.
I changed the line:
persist.debug.wfd.enable=0
to:
persist.debug.wfd.enable=1
and rebooted.
By default the Moto G (with KitKat) allows the user to start the 'Wireless display' settings activity, but the all important checkbox to 'Enable wireless display' is not shown.
The build.prop hack enables display of this checkbox.

So the Tab2 and Moto G both now have the available software settings to enable and configure mirror mode, and in theory(!) they both have the required hardware but so far i've not got it working.
I suspect mirror mode has not been enabled on these two devices for a reason (it doesn't work lol) and hacking the device isn't gonna change the fact that it doesn't work.

Anyway - my next experiment is to connect two external displays to my S3!
One connected with my MHL adapter and one with the Measy A2W in mirror mode.

The CWAC-Presentation library requests all 'presentation capable' external displays from the device, the device returns an array of zero or more connected presentation capable external displays.
If this array has a size of more than zero then the CWAC-Presentation library takes the first display in that array and raises the ShowPresentation(Display1 As Display) event.
Any other displays in that array are simply not actioned - the original android library didn't anticipate a device having more than one external display connected at any one time.
So i'll be trying to get two external displays to work at the same time and that might involve an update to the CWAC-Presentation library if successful.

And i've now researched the various methods available to connect to a Chromecast and am planning to start the B4A Chromecast library this weekend.

I'll keep this thread updated.

Martin.

 

[thedesolatesoul]

I received my chromecast today, am impressed.
Have you wrapped anything so far?
It would be nice if you could summarize your leads and I will try to get up to speed.
Thanks

 

[warwound]

@thedesolatesoul

Nothing wrapped so far.

I bought a subscription to The Busy Coder's Guide to Android Development a month or so back and it's got some pretty good info on external displays.
(Well worth the 12 month subscription fee!)

To connect to a Chromecast we need to use the MediaRouter class.

• Android API 16 introduced the android.media.MediaRouter class.
  This class gives access to external displays but not Chromecast devices.
• A 2013 update to the Android Support library introduced the android.support.v7.media.MediaRouter class and this class handles external displays as well as Chromecast devices.
  This is the class we need to use in the b4a Chromecast library.
  android.support.v7.media.MediaRouter depends on the v7 appcompat library library.
  So there's a little work to be done in Eclipse before you can even look at writing any code.
  http://developer.android.com/tools/support-library/setup.html contains some useful info and as always Google is your friend when Eclipse reports missing references etc.
• The CastCompanionLibrary-android is "a library project to enable developers integrate Cast capabilities into their applications faster and easier."
  It seems to contain some helper classes that'll save you having to write lots of code - i've not read the documentation for this library yet.

The v7 MediaRouter example found HERE is probably a good starting point to learn the syntax and usage required.

And the source code that accompanies The Busy Coder's Guide to Android Development is freely available and can be downloaded HERE.
Look at the MediaRouter and Presentation folders for examples.

Finally watch out for a bug that may or may not have been fixed: http://code.google.com/p/android/issues/detail?id=67032.
A callback is supposed to be executed when you stop remote playback of media and this callback seems to not be called.

Martin.

 

[warwound]

@thedesolatesoul

The exact class you want to look at in the CommonsWare demo project is this one:
https://github.com/commonsguy/cw-om.../android/remoteplayback/PlaybackFragment.java.
That should give us some working java code from which we can create a b4a library.

Unfortunately my S3 today is giving me the 'Unable to start Screen Mirroring because of a hardware issue' message - again it knows my S3 is rooted and is refusing to start mirror casting .

Hopefully running Triangle Away again and a reboot will work...

Martin.

[edit]Yep running Triangle Away and a reboot and now mirror cast mode works again [/edit]

 

[warwound]

@thedesolatesoul

Here's the CommonsWare Chromecast demo project compiled - working here on my S3:
http://b4a.martinpearman.co.uk/externaldisplays/MediaRouterRemotePlaybackDemo.apk

Martin.

 

[bluedude]

On my G. Nexus this works perfectly with my Chromecast.

 

[thedesolatesoul]

Thanks @warwound
I read similar documentation to you, however I got confused between the MediaRouter class (API16) and the MediaRouter class in the support libs. The MediaRouter does not seem to be too difficult to work with so far.

@thedesolatesoul
I bought a subscription to The Busy Coder's Guide to Android Development a month or so back and it's got some pretty good info on external displays.
(Well worth the 12 month subscription fee!)

Thank you for this. This is so far the best book/documentation I have seen for Android so far.

The CastCompanionLibrary-android is "a library project to enable developers integrate Cast capabilities into their applications faster and easier."

• It seems to contain some helper classes that'll save you having to write lots of code - i've not read the documentation for this library yet.

This looks good but it seems to only work to cast videos (for e.g. we cannot cast a presentation screen?). At least that is what I found in the documentation, I think there are more classes in the code.

@thedesolatesoul

The exact class you want to look at in the CommonsWare demo project is this one:
https://github.com/commonsguy/cw-om.../android/remoteplayback/PlaybackFragment.java.
That should give us some working java code from which we can create a b4a library.

I had a look at this and so far looks straight forward. I am not sure if we need to override activity_resume events though. If I get time (hardly likely!) I might give it a shot.

Unfortunately my S3 today is giving me the 'Unable to start Screen Mirroring because of a hardware issue' message - again it knows my S3 is rooted and is refusing to start mirror casting .

Havent tried mirror casting, is that an app or a specific option in developer options? I dont see it on my S3 (I am on 4.1)?

@thedesolatesoul

Here's the CommonsWare Chromecast demo project compiled - working here on my S3:
http://b4a.martinpearman.co.uk/externaldisplays/MediaRouterRemotePlaybackDemo.apk

Martin.

Will try this tonight when I get home!

So far I can see we can route audio, video and user/presentation from the MediaRouter in API16 but the support lib only supports routing using remoteplaybackclient i.e. from a uri?

[EDIT] So it seems the we can use a custom channel but it is only passing messages through strings and so forth, the receiver is made up of html/js. [/EDIT]

 

[thedesolatesoul]

Also, does the cast developer console cost $5 or did I make a mistake?
https://cast.google.com/publish/#/signup

 

[warwound]

Thank you for this. This is so far the best book/documentation I have seen for Android so far.

For $45 (12 month subscription) it's a great deal.
Frequently updated to cover the latest android apis with lots of decent working code samples - Mark Murphy certainly knows what he's talking about.
I miss not having a real paper book but a real paper book is generally out of date within months of it being published so i can live with a PDF ebook.

This looks good but it seems to only work to cast videos (for e.g. we cannot cast a presentation screen?). At least that is what I found in the documentation, I think there are more classes in the code.

Chromecast is not an external display that you actually cast anything to...
It's a dongle that can stream various types of media from network resources.
The android api allows you to tell the Chromecast to playback, pause or stop playback of a media resource.
I'm guessing that the Chromecast apps that seem to cast Presentations to the Chromecast actually setup a media server within the app and send the Chromecast a URL to that media derver.
And then the app effectively streams screengrabs to the Chromecast.
Most android Views can be converted to a Bitmap so converting a Presentation to a Bitmap, compressing the Bitmap and serving it as a video stream must be how such apps work.
The Chromecast can also mirror tabs from the desktop Chrome browser and i think the Chrome browser does much the same - runs as a streaming media server and streams a 'screengrab' of a tab to the Chromecast.
But i'm not 100% sure on any of this - i'm happy if someone can clarify exactly how these apps work.

I had a look at this and so far looks straight forward. I am not sure if we need to override activity_resume events though.

Look at the CWAC-Presentation library in post #2, it has methods Pause and Resume which must be called by b4a code in Activity_Pause and Activity_Resume.
It works and is a workaround to us not being able to override the android onPause and onResume methods.

Havent tried mirror casting, is that an app or a specific option in developer options? I dont see it on my S3 (I am on 4.1)?

Samsung implemented their own implementation of the Mircast protocol in the S3.
On my S3 with stock 4.3 ROM under Settings > Connections > Connect and Share there is an option Screen Mirroring.
On previous stock ROMs this option was named AllShare Cast i think.

So far I can see we can route audio, video and user/presentation from the MediaRouter in API16 but the support lib only supports routing using remoteplaybackclient i.e. from a uri?

I'm not 100% sure, i think that the support library MediaRouter performs all tasks and the API16 MediaRouter performs only external display task (no Chromecast).
I'll have to experiment...

[EDIT] So it seems the we can use a custom channel but it is only passing messages through strings and so forth, the receiver is made up of html/js. [/EDIT]

Also, does the cast developer console cost $5 or did I make a mistake?
https://cast.google.com/publish/#/signup

The Chromecast dongle runs an operating system that is a combination/hybrid of both android and Chromium.
The Chromecast user interface is just a Chrome browser display HTML5 web content.
According to the CommonsWare ebook, the Chromecast user interface (referred to as a Receiver) can be either:

• Default Receiver.
  The Default Receiver is a Chrome web app that is the Chromium user interface.
• Styled Receiver.
  A Styled Receiver allows the developer to apply 'light branding information' such as custom logos to the interface.
  It modifies the Default Receiver.
• Custom Receiver.
  A Custom Receiver is your very own Chrome web app - it replaces the Default Receiver.

If you wish to use a Styled Receiver or a Custom Receiver you need to pay Google $5 to register a Cast Developer Console account, otherwise the Default Receiver is used.
You also need to pay $5 to Google for the console account if you want to debug javascript running within the Receiver - but as this probably only applies if you are using a Custom Receiver you'll have already paid the required $5.
(There's only one $5 fee to pay).

As i said the CommonsWare ebook is an endless source of usefula dn up to date info - well worth an investment!
My referral link will earn me an extra month of subscription if anyone uses it...
https://wares.commonsware.com#warwound

Martin.

 

[thedesolatesoul]

I'm guessing that the Chromecast apps that seem to cast Presentations to the Chromecast actually setup a media server within the app and send the Chromecast a URL to that media derver.
And then the app effectively streams screengrabs to the Chromecast.
Most android Views can be converted to a Bitmap so converting a Presentation to a Bitmap, compressing the Bitmap and serving it as a video stream must be how such apps work.
The Chromecast can also mirror tabs from the desktop Chrome browser and i think the Chrome browser does much the same - runs as a streaming media server and streams a 'screengrab' of a tab to the Chromecast.
But i'm not 100% sure on any of this - i'm happy if someone can clarify exactly how these apps work.

I havent seen any app do this yet, maybe I am missing it.
As far as I know the receiver is just html/js so the UI on the chromecast screen will be built from this.
I guess you can take a screenshot of the activity, compress it (static screens compress well) and send it. And the receiver just displays the image, however I would imagine it would be a scaling nightmare. Also, this implies that the screen will be mirrored so you cannot have a different UI on the phone and TV.

Samsung implemented their own implementation of the Mircast protocol in the S3.
On my S3 with stock 4.3 ROM under Settings > Connections > Connect and Share there is an option Screen Mirroring.
On previous stock ROMs this option was named AllShare Cast i think.

I'll try to dig this up, I am on a custom rom. Also I found there is a root app called Mirror that does this too but it is not on the play store i think.

Thanks for all the information, I am also trying to make sense of it. Unfortunately I might not have a lot of time, so I should focus on one part.

 

[bluedude]

As far as I know we need to use the API's in the Google Play library for the Chromecast to work.

 

[warwound]

As far as I know we need to use the API's in the Google Play library for the Chromecast to work.

We have a choice:

• The android support v7 MediaRouter gives basic access to core features.
  You can connect, control playback of media etc.
• The Google Cast API allows more advanced control and more advanced features.

I'm guessing that if you register a Cast Developer Console account (the $5 fee required) and want to style or create your own Chromecast Receiver (user interface) OR you want to access the javascript debugging console then you need to use the Google Cast API.
Otherwise the v7 MediaRouter is the quick and easy way to go.

Martin.

 

[bluedude]

I think basis would be ok for now. Wondering if a webview could be casted. In chrome BETA this still doesn't work like expected.

I think in the end I want to send what's on my Android screen (own app.) to Chromecast. Just like the desktop Chromecast does.

Cheers.

 

[warwound]

The b4a Chromecast library is progressing - today i've managed to connect to and disconnect from my Chomecast, and while connected start playback of an online video.
Still lots of work to be done on it but i suspect i'll be able to upload something for you all to experiment with within the next couple of days.

Martin.