How to load a url, with username/password

KingKenny

New Member
Licensed User
Longtime User
Hello,

I'd like to load a webpage, that MAY require a username/password (like amazon.com, mail.yahoo.com or any other) from within my application (my app will supply username/pass) either using:

WebView.loadurl("www...")

or

StartActivity(PhoneIntent.OpenBrowser("www..."))

or any other way that is possible, please help

Cheers
 

KingKenny

New Member
Licensed User
Longtime User
See the UserAndPasswordRequired event: Basic4android - Views (Core)

I have tried that Erel, this is the page:
"https://accounts.google.com/ServiceLogin?hl=en&continue=http://www.google.com"

that I am trying to load which requires username/password, but the event UserAndPasswordRequired is never raised for me to return username/pass. I have searched the docs with no joy for an example of this. Please advise.
 
Upvote 0

warwound

Expert
Licensed User
Longtime User
There's some info that might help in this post of mine: http://www.b4x.com/forum/basic4andr...272-sending-keystrokes-webview.html#post74960

Looks like the form on accounts.google.com had an id of 'gaia_loginform', the email input element has a name of 'Email' and the password input element has a name of 'Passwd'.

I tried this code on a Gingerbread device and it works:

B4X:
For some reason the forum will not let me upload the B4A source code, see the attchment instead.

You can use WebViewExtras to execute the javascript statements or just use the javascript: protocol and the LoadUrl method of WebView.

Bear in mind that there is no guarantee that Google will always use these exact ids and names for the form and it's elements.
If at any time Google change the webpage your code will no longer work.

Martin.
 

Attachments

  • WebViewLogin.zip
    6 KB · Views: 1,196
Upvote 0

KingKenny

New Member
Licensed User
Longtime User
Thanks so much warwound (interesting choice of username btw) actually, I was looking for a way to log in to any website, not just google, like single sign on pages, that allow you to log in to multiple sites, I was trying to replicate that from my device. I guess I have to find the credentials fields for each site and make a custom call for each site separately . Thanks for info.
 
Upvote 0

warwound

Expert
Licensed User
Longtime User
Yes it'll be unpredictable.

Some sites will give the login form an id with 'login' included in the form id, others won't.
Similarly the username and password fields could have obvious names such as 'username' and 'password', others won't.

You can use javascript to get an array of all form objects in a web page and look at form ids and input field names to try and guess what form to auto-populate and submit.

And as i mentioned, it's more than likely that a site will get updated and the form id, username and password field names get changed.

Not a lot you can do about any of this - it's all beyond your control.

Martin.
 
Upvote 0

surfuric

Member
Licensed User
Longtime User
I am trying to follow the above example. The website I am trying to input the username and password has the following:

<div class="form_div">
<form action="/worksheets/login/" method="post">
<div class="field"><div class="label">Username</div><input id="username" type="text" name="username" class="text"></div>
<div class="field"><div class="label">Password</div><input type="password" name="password" class="text"></div>
<input type="submit" value="Sign In" class="submit">
</form><br><br>
</div>

I used Javascript.Append("username.value","xxx@yyy.com") and that worked. But I have not been able to access the password or the submit button. Do you think it is possible with WebViewExtras? Or is this website just not "friendly" enough.
 
Upvote 0

warwound

Expert
Licensed User
Longtime User
If a form has no id then you should still be able to get a reference to it using the javascript document object forms property.
You can treat the forms property as an array, so document.forms[0] will be the first form that the page contains:

B4X:
Dim Javascript As StringBuilder
Javascript.Initialize
Javascript.Append("var form=document.forms[0];")   '   this assumes your form is the first form on the page
Javascript.Append("form.username.value='my_email_address@email.com';")
Javascript.Append("form.password.value='my_password';")
Javascript.Append("form.submit()")
Log("Executing javascript: "&Javascript.ToString)
      
Dim WebViewExtras1 As WebViewExtras
WebViewExtras1.executeJavascript(WebView1, Javascript.ToString)

Martin.
 
Upvote 0

surfuric

Member
Licensed User
Longtime User
Warwound,
Sincere Thanks. There is no way I could have solved that without your help!

Once the WebView was loaded, I used WebViewExtras and followed your example for the "B4A.CallSub('processHTML',true, document.documentElement.outerHTML)" to get the HTML. At first I wanted to use the HttpUtils2, but quickly realized that at this point the secure session is only within the Webview (am I correct??)

I then parsed the html to find the fullpath to the worksheet to download (it is a .pdf file). But I cannot use HttpUtils2 to download it. I can only go through WebViewExtras, correct? Is there an easier way? Can you please give me a hint how to download this file?

I really appreciate your time!
 
Upvote 0

warwound

Expert
Licensed User
Longtime User
That's a more complicated one to answer.
It can be done in theory but it involves using Android methods that are not currently exposed in B4A - i think that's true, there may be a library that can help.

Basically once you have logged in to the web site with a WebView, the WebView contains a cookie and that cookie contains values so that the web site knows you are logged in/authenticated.

To download the PDF using an HttpClient would require that you get the cookie from the WebView and add it to your HttpClient.
When the HttpClient requests the PDF the cookie you've added to it would tell the web site that you're logged in and allow the download.

Have a look at this page: Syncing cookies between an HttpClient and a WebView | eshyu's Blog.
That explains the native Android code required - you'll have to search the forum to see if this can be done in B4A.

Martin.
 
Upvote 0

surfuric

Member
Licensed User
Longtime User
Thanks,
I figured it would be difficult. I will look at the reference you have posted.

Just out of curiosity, and I tried this a lot last night with no results, could I write a javascript to download the file and execute that with webviewextras? I am not sure how to write an entire javascript and execute it with webviewextras, but if you think I can I will try. If you say it is a waste of time, I will focus on something else.
 
Upvote 0

warwound

Expert
Licensed User
Longtime User
Hi again.

Using javascript wouldn't allow you to download the PDF without using the WebView - javascript runs entirely in the WebView.

I remembered earlier a library that i started work on a while back and forgot about...
CookieManager is a simple wrapper for the Android <link>CookieManager|http://developer.android.com/reference/android/webkit/CookieManager.html</link> Class.
It enables you to manages the cookies used by your application's WebView instances.

I've recompiled it and put a demo project together:

B4X:
Sub Process_Globals
   'These global variables will be declared once when the application starts.
   'These variables can be accessed from all modules.
   
End Sub

Sub Globals
   'These global variables will be redeclared each time the activity is created.
   'These variables can only be accessed from this module.

   Dim CookieManager1 As CookieManager
   Dim WebView1 As WebView
End Sub

Sub Activity_Create(FirstTime As Boolean)
   CookieManager1.SetAcceptCookies(True)
   WebView1.Initialize("WebView1")
   Activity.AddView(WebView1, 0, 0, 100%x, 100%y)
End Sub

Sub Activity_Resume
   '   load a webpage that requires a login
   Dim Url As String
   Url="http://www.geograph.org.uk/profile/6526"   '   change this to your login page
   WebView1.LoadUrl(Url)
End Sub

Sub Activity_Pause (UserClosed As Boolean)

End Sub

Sub WebView1_PageFinished (Url As String)
   Log("WebView1_PageFinished Url = "&Url)
   If(CookieManager1.HasCookies) Then
      Log("Cookies: "&CookieManager1.GetCookie(Url))
   Else
      Log("No cookies found")
   End If
End Sub

Change the Url that the WebView loads to your secure web page and watch the log as the pages loads - no cookies is likely to be logged.
Now login and watch the log as the WebView reloads - do you see a cookie displayed?
Maybe a cookie with the name PHPSESSID?

If so then this is the info you need to use when requesting the PDF document download using HTTPClient.
I'm not sure exactly what is required, whether you'd use the HttpClient SetHttpParameter method or the HttpRequest SetHeader method.

Can you try the attached code and post again with your results - you might also wanna try a forum search for 'http set cookie' or similar.

Martin.

[edit]CookieManager now available from here: http://www.b4x.com/forum/additional...dates/22968-lib-cookiemanager.html#post133147[/edit]
 
Last edited:
Upvote 0

surfuric

Member
Licensed User
Longtime User
Warwound,
You, sir, are fantastic!
I am not 100% finished, but I wanted to tell you that I used your lib to get the cookie. Then used the following:
req.InitializeGet("https:\\fileIwantedtoDownload.pdf")
req.SetHeader("cookie",CookieManager1.GetCookie(Url))
hc.Execute(req,1)

and it was there! Not perfect yet because I am still using Response.GetString("UTF8") to get it to the log just to see if it worked. I will save it instead to external sdcard to read later with a pdf viewer.

But I wanted to stop everything I was doing and thank you sincerely.
(I am not sure what a shandy is in your sig, but I will be buying you one in the very near future!!!)
Thanks again,
Craig
 
Upvote 0

surfuric

Member
Licensed User
Longtime User
Warwound,
Is there something special to getting webviewextras to work on android 2.3.6? I am testing on older devices (this time a motorola atrix) just in case our clients are using an older version of android. For some reason CookieManager1.HasCookies is always false on this version. (Works fine on 4.x.x.)

Any suggestions?
 
Upvote 0

warwound

Expert
Licensed User
Longtime User
Hi.

Looks like CookieManager isn't supported on older API levels.
I originally tested the code i posted on an ICS device and it worked.
Testing again on the same ICS device fails, the activity crashes as soon as it loads logging: 'Fatal signal 11'.

B4X:
   WebView1.Initialize("WebView1")
   '   call SetAcceptCookie after WebView has been initialized
   CookieManager1.SetAcceptCookies(True)
   Activity.AddView(WebView1, 0, 0, 100%x, 100%y)

Calling SetAcceptCookie after the WebView has been initialized fixes that crash and CookieManager works perfectly.

Now i've tested on a device with 2.3.7 and, as you posted, HasCookies always returns False.
I looked at the official CookieManager documentation: CookieManager | Android Developers
It says CookieManager is supported by Android API level 1+, with the exception of the 2 FileSchemeCookies setter and getter methods which were added in API level 12.
There is another CookieManager class which is only available with API level 9+, java.net.CookieManager: CookieManager | Android Developers but the B4A library doesn't use this java.net class.

I wondered whether HasCookies returns True only if there are cookies in saved storage and the method returns False even if there are cookies in memory but not saved storage.
So i created a new library object CookieSyncManager, here's the updated library reference:

CookieManager
Comment: CookieManager is a simple wrapper for the Android <link>CookieManager|http://developer.android.com/reference/android/webkit/CookieManager.html</link> and <link>CookieSyncManager|http://developer.android.com/reference/android/webkit/CookieSyncManager.html</link> Classes.
It enables you to manages the cookies used by your application's WebView instances.
Author: Martin Pearman
Version: 1.1
  • CookieManager
    Methods:
    • GetAcceptCookies As Boolean
      Returns True if cookies will be accepted by your application.
    • GetCookie (Url As String) As String
      Returns the cookie for the given Url in the format: NAME=VALUE [; NAME=VALUE]
      If no cookie exists for the Url then a null value will be returned.
    • HasCookies As Boolean
      Returns True if any cookies are stored for your application.
    • RemoveAllCookies
      Removes all cookies stored for your application.
    • RemoveExpiredCookies
      Removes all expired cookies stored for your application.
    • RemoveSessionCookies
      Removes all session cookies stored for your application.
    • SetAcceptCookies (Accept As Boolean)
      Set whether cookies will be accepted for your application.
    • SetCookie (Url As String, Value As String)
      Sets the cookie for the Url to the Value.
  • CookieSyncManager
    Methods:
    • Initialize
      Initialize the CookieSyncManager.
      The CookieSyncManager is used to synchronise the browser cookie store between RAM and permanent storage.
      To get the best performance, browser cookies are saved in RAM.
      A separate thread saves the cookies between, driven by a timer with a 5 minute interval.
    • ResetSync
      Resets the CookieSyncManager timer.
    • StartSync
      Requests the CookieSyncManager to start synchronisation.
      Typically called in Activity_Resume.
    • StopSync
      Requests the CookieSyncManager to stop synchronisation.
      Typically called in Activity_Pause.
    • Sync
      Forces the CookieSyncManager to synchronise now.
      This method is asynchronous, there is no guarantee it will synchronise immediately.

And example B4A code:

B4X:
'Activity module
Sub Process_Globals
   
End Sub

Sub Globals
   Dim CookieManager1 As CookieManager
   Dim CookieSyncManager1 As CookieSyncManager
   Dim WebView1 As WebView
End Sub

Sub Activity_Create(FirstTime As Boolean)
   CookieSyncManager1.Initialize
   
   WebView1.Initialize("WebView1")
   Activity.AddView(WebView1, 0, 0, 100%x, 100%y)
   '   call SetAcceptCookie after WebView has been initialized
   CookieManager1.SetAcceptCookies(True)
   
   '   load a webpage that requires a login
   Dim Url As String
   Url="http://www.geograph.org.uk/profile/6526"   '   change this to your login page
   WebView1.LoadUrl(Url)
   
End Sub

Sub Activity_Resume
   CookieSyncManager1.StartSync   '   not really required for this test example
End Sub

Sub Activity_Pause (UserClosed As Boolean)
   CookieSyncManager1.StopSync   '   not really required for this test example
End Sub

Sub WebView1_PageFinished (Url As String)
   '   force the CookieSyncManager to synchronise
   CookieSyncManager1.Sync
   CookieSyncManager1.ResetSync
   
   Log("WebView1_PageFinished Url = "&Url)
   
   If CookieManager1.HasCookies Then
      Log("Cookies: "&CookieManager1.GetCookie(Url))
   Else
      Log("No cookies found")
   End If
End Sub

Still the Gingerbread device always logs 'No cookies found'.
The cookie does though seem to exist, if i edit the example:

B4X:
Sub WebView1_PageFinished (Url As String)
   '   force the CookieSyncManager to synchronise
   CookieSyncManager1.Sync
   CookieSyncManager1.ResetSync
   
   Log("WebView1_PageFinished Url = "&Url)
   
   Log("Cookies: "&CookieManager1.GetCookie(Url))
      
End Sub

The log shows a cookie with a name of PHPSESSID after logging in to a website, before logging in the log reports 'Cookies: null'.
This behaviour is the same with and without the CookieSyncManager code, the CookieSyncManager seems to have no useful purpose here.

Can you test to see if GetCookie returns a value after logging in to a website and returns null if not logged in?
If might be that you'll have to test if GetCookie is Null or not instead of using the HasCookies method.

Martin.
 
Last edited:
Upvote 0

surfuric

Member
Licensed User
Longtime User
You are correct that the HasCookies always returns false for me on an old version of android. Changing to testing whether or not the CookieManager1.GetCookie(Url) actually had a value or not fixed the problem. I did not have to encorporate the CookieSyncManager to get this to work.
I have tested also on my 4.0.4 as well and it seems to work.

So . . . as you mention:
This behaviour is the same with and without the CookieSyncManager code, the CookieSyncManager seems to have no useful purpose here.
Should I just leave well enough alone and not use the Sync Manager?
 
Upvote 0

warwound

Expert
Licensed User
Longtime User
I think the SyncManager enables you to persist cookies when your app is restarted.
Unless you need that then there's no need to use the SyncManager.

Martin.
 
Upvote 0
Top