Android Question AWS Device Farm (B4X)

[JackKirk]

I have just discovered something that I believe all community members should be aware of:

AWS Device Farm

I have just used it to solve an iOS problem.

But it has huge potential in the Android development/testing world also.

It appears to be some 2,500 actual devices (about 300 different ones if I have read it correctly) that you can access via the web - you can do both automated and manual testing.

The only real hassle is setting up an AWS account if you don't have one.

Cost is USD0.17 / minute - about USD10 / hr - BUT you get the first 1000 mins (16 hours) FREE

It is incredibly easy to use - for iOS you just take the ipa out of ...\Objects\Archive.zip and paste it onto the device web page - 20 secs later you are running.

I haven't had the need yet to try Android but I suspect it will be just as easy.

So if you get complaints about your app not working on a specific device that you naturally don't have - no longer a problem.

The only shortcoming I have discovered so far is an inability to handle SMS.

I don't doubt there are others which I hope will be documented in this thread with time.

 

[mcqueccu]

Nice one. Does it support taking screenshots as well?

 

[JackKirk]

Does it support taking screenshots as well?

Yes, that was the core of my iOS problem.

 

[JackKirk]

about 300 different ones if I have read it correctly

I may have been a tad too enthusiastic - of the 297 different devices available only about 56 are available for "remote access" - meaning you can interact with them. But these 56 appear to cover all the most popular makes/models.

The rest are accessed by "automated tests written in popular frameworks like Appium, Espresso, and XCTest."

Anyone have any clues on this? Erel?

 

[JackKirk]

Another interesting thing - when you complete a remote access session you can obtain (under the Files tab):

• a high quality video of the session.
• various logs which I assume can be filtered to the more normal logs produced by the B4A/I IDEs.

 

[walterf25]

I may have been a tad too enthusiastic - of the 297 different devices available only about 56 are available for "remote access" - meaning you can interact with them. But these 56 appear to cover all the most popular makes/models.

The rest are accessed by "automated tests written in popular frameworks like Appium, Espresso, and XCTest."

Anyone have any clues on this? Erel?

Great find, thanks for sharing, about the rest of the devices being only available by automated tests, i was wrapping the Appium library, i had a need to do some automated tests on an app i was working on for a Client, and also because i saw that there are a ton of jobs for test script automation in large companies which pay very well, unfortunately i had to stop as I was not able to figure out a way to implement the test reporting part of it, i'm not sure how familiar you are with any of this Test Automation frameworks but i found Appium to be by far the easiest one to learn and implement, basically you would write your test automation tasks in B4J and the B4J app would push the apk file into your Android Device if is not already installed and start executing a series of automated tests, such as checking the response from a website, checking the user input on any view such as password, username etc.
You can run this same task in a loop a number of times and generate a report at the end of the test, the report will record exceptions if something didn't go as expected, i am guessing the rest of the devices are mainly dedicated for this type of automation tests and can only be connected through a URL link they provide once you sign up for that service, that Link is passed to the Appium library which connects the appium server to the device being tested, the server checks the type of device, operating system version etc.. and connects to the device once everything is verified.

I did place a post in the Wish section, hopefully @Erel can find the time one day to add the @test annotation to B4J so that I can continue and finish wrapping this library.

Edit
I wrote this post before looking at the link you shared in the first post, it seems that they have the whole service available, which means there is no need for the library i was wrapping as they already provide all those frameworks integrated into their service, all you would have to do is use their service and you would be able to automate your tests, this is really nice but depending on your need it could get really expensive or not, depending on the length of your automated tests and the number of devices you want your app to be tested across.

Walter

 

[JackKirk]

I wrote this post before looking at the link you shared in the first post, it seems that they have the whole service available, which means there is no need for the library i was wrapping as they already provide all those frameworks integrated into their service, all you would have to do is use their service and you would be able to automate your tests, this is really nice but depending on your need it could get really expensive or not, depending on the length of your automated tests and the number of devices you want your app to be tested across.

Walter, as you seem to be the automation test expert hereabouts would it be asking too much to suggest you create a tutorial for say a B4A app?

I can guarantee you an enthusiast audience of at least 1

 

[walterf25]

Walter, as you seem to be the automation test expert hereabouts would it be asking too much to suggest you create a tutorial for say a B4A app?

Are you talking about a tutorial for the Appium library or using AWS?

 

[JackKirk]

Are you talking about a tutorial for the Appium library or using AWS?

AWS

 

[walterf25]

AWS

I will try to make some time, i can't promise anything but i will try.

Walter

 

[JackKirk]

I will try to make some time, i can't promise anything but i will try.

I think it would be a major plus for the community.

If there is anything I could do to help - testing etc - I would be available.

 

[walterf25]

I think it would be a major plus for the community.

If there is anything I could do to help - testing etc - I would be available.

I took a quick look and all you need to do is upload your apk file, you would still have to write the Tests either using TestNG or Junit, once you upload all this files you can start the Test, it will tell you which devices your apk will be tested across, it will take some time, at the end you will get a report including screenshots of all the tests performed, as i mentioned earlier, it will take some time to make a small tutorial, i still think personally it would be great to be able to do this directly from B4J as you can customize everything single test.

Walter

 

[JackKirk]

It looks to me like you can set up a test environment on multiple devices that stays available for as long as you have an AWS account - then when you have a new release you just plug it in and your testing is largely done - if this is so it has major implications for anyone doing anything serious with B4A/I.

BTW do you live anywhere near Agoura Hills?

 

[walterf25]

It looks to me like you can set up a test environment on multiple devices that stays available for as long as you have an AWS account - then when you have a new release you just plug it in and your testing is largely done - if this is so it has major implications for anyone doing anything serious with B4A/I.

BTW do you live anywhere near Agoura Hills?

I totally agree, Yes i live about 20 minutes from there, I'm in Lake Balboa, next to Encino.

Where are you?

Walter

 

[JackKirk]

Where are you?

I actually live in Booker Bay - an hour N of Sydney NSW Australia.

But my family and I lived in Agoura Hills 1989/90/91 - fond memories - except for the commute to Burbank on the 101 every day.

 

[walterf25]

I actually live in Booker Bay - an hour N of Sydney NSW Australia.

But my family and I lived in Agoura Hills 1989/90/91 - fond memories - except for the commute to Burbank on the 101 every day.

Wow, small world, I've been to Australia a few times, I've been to NSW, Byron Bay, Surfer's paradise, spent a few hours in Sydney beautiful place.

Walter

 

[JackKirk]

spent a few hours in Sydney beautiful place

I've done a lot of travelling and never found somewhere I'd prefer to live - if you are back in Sydney in the future let us know and we will show you around.

 

[walterf25]

I've done a lot of travelling and never found somewhere I'd prefer to live - if you are back in Sydney in the future let us know and we will show you around.

Thanks, will do.

Walter

 

[JackKirk]

TestNG or Junit

Just been doing a bit of googling on these - a couple of questions:

• they both apply to Java/B4A - is there an equivalent for B4I?
• would these need some form of porting?
• I don't really understand how they work - I presume they are embedded in the B4A app

 

[JackKirk]

There is also this as an alternative: Bitbar

Which I can't seem to be able to open an account at...