B4J Question How to know or control web site responses

[PABLO2013]

Greetings
I am scraping information from my consecutive pages and then the information I put in a database.
This code does it well for me, although the answers are added to the database in a disorderly or random way, but when I make a second query, I can not fit the data (images) with the corresponding records, because the web response is random or disorderly.

1. how you can request text and images in the same query.
2 How can I correct this?
3. what options do I have

Sub AppStart (Args() As String)

sql1.InitializeSQLite(File.DirApp, "1.db", True)

LA
LAFOTO
StartMessageLoop 

End Sub



Sub LA

For i=6000 To 6009
Log(i)
DIR(12)="https://www.la.com/tienda/"&i&"-la-"&i&"-.html"
'Log(DIR(12))
Dim job As HttpJob
job.Initialize("LA", Me)
job.Download(DIR(12))
job.GetRequest.Timeout = 900000
Next
End Sub


Sub LAFOTO
Dim Cursor As ResultSet
FOR i=6000 to 6009
Cursor = sql1.ExecQuery("SELECT FOTO FROM BASE WHERE CODIGO="&i)
'FOTO have the address of image
DIR(13)=Cursor.GetString("FOTO")
Dim job1 As HttpJob
job1.Initialize("FOTOLA1", Me)
job1.Download(DIR(13))
job1.GetRequest.Timeout = 900000
next
         
End Sub

Sub JobDone(Job As HttpJob)
Select Job.JobName
 
Case "LA"
 
If Job.Success = True Then

DIR(12)=Job.GetString

This code scrapes image direction and a data of a price
And stores it in T2, T3, T4, T5, T6, T7, T8
Buffer i put in this momento to NULL

............
sql1.ExecNonQuery2("INSERT INTO BASE VALUES (?, ?, ?, ?, ?, ?, ?, ?,?)", Array As Object(Null, T2, T3,T4,T5,T6,T7,T8,Buffer))
End If

   Else
      Log("Error: " & Job.ErrorMessage)

   End If
   Job.Release

Case "FOTOLA1"

If Job.Success = True Then

Try 
Dim InputStream1 As InputStream

InputStream1 = Job.GetInputStream

Dim OutputStream1 As OutputStream
OutputStream1.InitializeToBytesArray(1000)

File.Copy2(InputStream1, OutputStream1)

Dim Buffer() As Byte 'declares an empty array
Buffer = OutputStream1.ToBytesArray
'OutputStream1.Close

Catch
Buffer=Null 
End Try
sql1.ExecNonQuery2("UPDATE BASE Set FOTO1 = ? WHERE CODIGO ="&numero, Array As Object(Buffer))

   Else
      Log("Error: " & Job.ErrorMessage)

End If

End Select

End Sub

 

[Erel]

It is very difficult to understand this code.

how you can request text and images in the same query.

You can't. Each http request returns a single resource.

You can set Job.Tag before the job is sent and use the tag value to organize the responses.

 

[PABLO2013]

Thanks
This is just the missing piece.
Paste the code from a text editor and remove the indentation.
I'm going to copy it with what you said in case it works, thank you

Sub AppStart (Args() As String)

sql1.InitializeSQLite(File.DirApp, "1.db", True)

LA
LAFOTO
StartMessageLoop

End Sub



Sub LA

For i=6000 To 6009
Log(i)
DIR(12)="https://www.la.com/tienda/"&i&"-la-"&i&"-.html"
'Log(DIR(12))
Dim job As HttpJob
job.Initialize("LA", Me)
job.Download(DIR(12))
job.tag=i
job.GetRequest.Timeout = 900000
Next
End Sub


Sub LAFOTO
Dim Cursor As ResultSet
FOR i=6000 to 6009
Cursor = sql1.ExecQuery("SELECT FOTO FROM BASE WHERE CODIGO="&i)
'FOTO have the address of image
DIR(13)=Cursor.GetString("FOTO")
Dim job1 As HttpJob
job1.Initialize("FOTOLA1", Me)
job1.Download(DIR(13))
job1.GetRequest.Timeout = 900000
next
      
End Sub

Sub JobDone(Job As HttpJob)
Select Job.JobName
Case "LA"
If Job.Success = True Then
log(job.tag)
DIR(12)=Job.GetString

'This code scrapes image direction and a data of a price
'And stores it in T2, T3, T4, T5, T6, T7, T8
'Buffer i put in this momento to NULL
'............
sql1.ExecNonQuery2("INSERT INTO BASE VALUES (?, ?, ?, ?, ?, ?, ?, ?,?)", Array As Object(Null, T2, T3,T4,T5,T6,T7,T8,Buffer))
End If

   Else
      Log("Error: " & Job.ErrorMessage)

   End If
   Job.Release

Case "FOTOLA1"

If Job.Success = True Then

Try
Dim InputStream1 As InputStream

InputStream1 = Job.GetInputStream

Dim OutputStream1 As OutputStream
OutputStream1.InitializeToBytesArray(1000)

File.Copy2(InputStream1, OutputStream1)

Dim Buffer() As Byte 'declares an empty array
Buffer = OutputStream1.ToBytesArray
'OutputStream1.Close

Catch
Buffer=Null
End Try
sql1.ExecNonQuery2("UPDATE BASE Set FOTO1 = ? WHERE CODIGO ="&numero, Array As Object(Buffer))

   Else
      Log("Error: " & Job.ErrorMessage)

End If

End Select

End Sub
Last edited: Yesterday at 5:26 PM