HttpUtils code returns different results every time it's run

rleiman

rleiman

Well-Known Member
Licensed User
Longtime User
Hi Everyone,

Can you tell me why this code returns a different set of results each time the app is run?

This code downloads web pages from dictionary.com it lets you know if a particular word is a noun or not a noun.

The words that are sent to dictionary.com are:
this
is
a
test

I was expecting that "a" and "test" would be flagged as a noun but the results returned are not consistant. If I go to dictionary.com and do it manually, it returns the correct results.

Can you tell me if I did the coding wrong?

Is the fault with my coding or with dictionary.com maybe?

If you run this coding you can see the strange behaviour.

B4X: 
'Activity module
Sub Process_Globals
   'These global variables will be declared once when the application starts.
   'These variables can be accessed from all modules.

End Sub

Sub Globals
   'These global variables will be redeclared each time the activity is created.
   'These variables can only be accessed from this module.

    Dim b4a As String
   Dim strDownloadList As List
   Dim listWords As List
End Sub

Sub Activity_Create(FirstTime As Boolean)

   b4a = "http://dictionary.reference.com/browse/"

   listWords.Initialize
   listWords.Add("this")
   listWords.Add("is")
   listWords.Add("a")
   listWords.Add("test")

   strDownloadList.Initialize
   strDownloadList.Add(b4a & "this")
   strDownloadList.Add(b4a & "is")
   strDownloadList.Add(b4a & "a")
   strDownloadList.Add(b4a & "test")

   HttpUtils.CallbackActivity = "Main" 'Current activity name.
    HttpUtils.CallbackJobDoneSub = "JobDone"
   HttpUtils.CallbackUrlDoneSub = "UrlDone"
   HttpUtils.DownloadList("JobDone", strDownloadList)
End Sub

Sub Activity_Resume

End Sub

Sub Activity_Pause (UserClosed As Boolean)

End Sub

Sub JobDone (Job As String)

     ToastMessageShow("Processing all done.", False)
End Sub

Sub UrlDone(Url)

   Dim strEntireWebPage As String
   Dim strSearchWord As String

'   ToastMessageShow("Processing URL: " & Url, False)
'   ToastMessageShow("URL number: " & HttpUtilsService.countWorking, False)

   If HttpUtils.IsSuccess(Url) Then
      s = HttpUtils.GetString(Url)

'      ToastMessageShow(s,True)
      
      ' Replace double quotes with single quotes for easy handling because I am using strings.
      '---------------------------------------------------------------------------------------
      s = s.Replace(QUOTE, "'") 

      ' Trim down the entire web page so only the deffinition is left.
      '---------------------------------------------------------------
      s = s.SubString(s.IndexOf("<div class='pbk'><span class='pg'>") + 34) 'Add 34 so we grab the definition after the tag
      s = s.SubString2(0, s.IndexOf("</span>")) 'Grab only the definition enclosed

      ' See if the word is a noun.
      '---------------------------
      If s.ToUpperCase.Trim = "NOUN" Then
         ToastMessageShow(listWords.Get(HttpUtilsService.countWorking) & " *** is a noun.", False)
      Else
         ToastMessageShow(listWords.Get(HttpUtilsService.countWorking) & " *** is NOT a noun.", False)
      End If
   End If
End Sub
 
Last edited:
Sort by date Sort by votes
rleiman

rleiman

Well-Known Member
Licensed User
Longtime User
Hi Erel,

It's ok if the ordering arrive in a different sequence. I'm only interested with pulling out the nouns from a list of words, for example:
is
test
a
this

But on a particular run it will say that the word "this" is a noun and on another run, it will say "is" was a noun when in reality the word "test" is a noun and "a" is a noun, not the other words.

If you click these links you will see what results I would like:

Test | Define Test at Dictionary.com
A | Define A at Dictionary.com
This | Define This at Dictionary.com
Is | Define Is at Dictionary.com

If you look in the code, I have some toast messages so that shows I know what download is for a particular word.

We hope to get it to download "a" as being a noun every time the app is run.

Can you correct the sample coding to show what you were talking about?

Thanks in advance.

Erel said:
The requests are sent in parallel. You cannot assume that the results will arrive in the same order.
You should use the Url parameter to match the request.
Unless you want to show the results as they arrive, it is simpler to use JobDone and go over all requests. This will also solve this ordering issue.
Click to expand...
 
Last edited:
Upvote 0
rleiman

rleiman

Well-Known Member
Licensed User
Longtime User
Is there a way to find the order so I can match it up with the downloaded results?

It's ok if the order would be 3,1,2,4 because I can figure out which word each of those would represent.

If not, I will try to scrape the downloaded web page to see what word was downloaded.

Erel said:
This code is wrong:
B4X: 
ToastMessageShow(listWords.Get(HttpUtilsService.countWorking) & " ***
You are assuming that the order of results is as the order of the requests.
Click to expand...
 
Upvote 0
rleiman

rleiman

Well-Known Member
Licensed User
Longtime User
Thanks for the help.

I was able to get consistant results by some additional web page scraping.

Hopefully dictionary.com won't change anything.

Here is the working code:

B4X: 
'Activity module
Sub Process_Globals
   'These global variables will be declared once when the application starts.
   'These variables can be accessed from all modules.

End Sub

Sub Globals
   'These global variables will be redeclared each time the activity is created.
   'These variables can only be accessed from this module.

    Dim b4a As String
   Dim strDownloadList As List
End Sub

Sub Activity_Create(FirstTime As Boolean)

   b4a = "http://dictionary.reference.com/browse/"

   strDownloadList.Initialize
   strDownloadList.Add(b4a & "this")
   strDownloadList.Add(b4a & "is")
   strDownloadList.Add(b4a & "a")
   strDownloadList.Add(b4a & "test")

   HttpUtils.CallbackActivity = "Main" 'Current activity name.
    HttpUtils.CallbackJobDoneSub = "JobDone"
   HttpUtils.CallbackUrlDoneSub = "UrlDone"
   HttpUtils.DownloadList("JobDone", strDownloadList)
End Sub

Sub Activity_Resume

End Sub

Sub Activity_Pause (UserClosed As Boolean)

End Sub

Sub JobDone (Job As String)

   HttpUtils.Complete = True

     ToastMessageShow("Processing all done.", False)
End Sub

Sub UrlDone(Url)

   Dim strEntireWebPage As String
   Dim strSearchWord As String
   Dim strDownloadedWebPage As String
   Dim strPartOfSpeach As String
   Dim strLookupWord As String

'   ToastMessageShow("Processing URL: " & Url, False)
'   ToastMessageShow("URL number: " & HttpUtilsService.countWorking, False)

   If HttpUtils.IsSuccess(Url) Then
      
      strDownloadedWebPage = HttpUtils.GetString(Url)

'      ToastMessageShow(s,True)
      
      ' Replace double quotes with single quotes for easy handling because I am using strings.
      '---------------------------------------------------------------------------------------
      strDownloadedWebPage = strDownloadedWebPage.Replace(QUOTE, "'") 

      ' Trim down the entire web page so only the deffinition is left for the part of speach found.
      '--------------------------------------------------------------------------------------------
      strPartOfSpeach = strDownloadedWebPage.SubString(strDownloadedWebPage. _
         IndexOf("<div class='pbk'><span class='pg'>") + 34) ' Add 34 so we grab the definition after the tag.
      strPartOfSpeach = strPartOfSpeach.SubString2(0, strPartOfSpeach.IndexOf("</span>")) ' Grab only the definition enclosed.

      ' Trim down the entire web page so only the deffinition of the lookup word is found.
      '-----------------------------------------------------------------------------------
      strLookupWord = strDownloadedWebPage.SubString(strDownloadedWebPage. _
         IndexOf("Listen to the pronunciation of ") + 31) ' Add 31 so we grab the definition after the text.
         
      strLookupWord = strLookupWord.SubString2(0, strLookupWord.IndexOf("'")) 'Grab only the lookup word.

      Msgbox("The word: " & QUOTE & strLookupWord & QUOTE & " is " & strPartOfSpeach.ToUpperCase & ".", "Part of Speach")   
   End If
End Sub
 
Upvote 0
You must log in or register to reply here.
Top