Android Question Gemini recognize scribbling from b4x signature template- curl to b4x - Help!

[personalapps]

How do I convert this curl to b4x? I am trying to post the scribbling from b4x signature template and trying to get reply from Gemini:

curl https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-flash:generateContent?key=${GOOGLE_API_KEY} \
-H 'Content-Type: application/json' \
-d '{
"contents":[{
"parts":[
{"text": "What is this picture?"},
{"inline_data": {
"mime_type":"image/png",
"data": "'"$(base64 -i 'path/to/image/image.png')"'"
}}
]}
]
}'

This is my b4xcode:

Sub SendSignatureToGemini
    ' Capture the signature as an image
    Dim bmp As B4XBitmap = SignatureTemplate.Bitmap
    
    ' Convert the bitmap to a byte array
    Dim ImageBytes() As Byte = ImageToBytes(bmp, "PNG")
    
    If ImageBytes = Null Then
        Log("Failed to convert image to bytes")
        Return
    End If
    
    ' Convert the byte array to a Base64 string
    Dim Base64Image As String = su.EncodeBase64(ImageBytes)
    
    ' Prepare the JSON payload
    Dim JSON As JSONGenerator
    Dim partsList As List
    partsList.Initialize
    
    ' Part 1: Text part
    Dim part1 As Map
    part1.Initialize
    part1.Put("text", "What is this picture?")
    partsList.Add(part1)
    
    ' Part 2: Inline data part
    Dim part2 As Map
    part2.Initialize
    part2.Put("type", "inline_data")
    
    Dim inline_data As Map
    inline_data.Initialize
    inline_data.Put("mime_type", "image/png")
    inline_data.Put("data", Base64Image)
    
    part2.Put("inline_data", inline_data)
    partsList.Add(part2)
    
    ' Contents List
    Dim contentsList As List
    contentsList.Initialize
    contentsList.Add(partsList)
    
    Dim rootMap As Map
    rootMap.Initialize
    rootMap.Put("contents", contentsList)
    
    JSON.Initialize(rootMap)
    Dim JSONString As String = JSON.ToString
    
    ' Set up the HttpJob
    Dim Job As HttpJob
    Job.Initialize("GeminiJob", Me)
    Job.PostString($"https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-flash:generateContent?key=${YOUR_GOOGLE_API_KEY}"$, JSONString)
    Job.GetRequest.SetContentType("application/json")
End Sub

Sub JobDone(Job As HttpJob)
    If Job.Success Then
        Dim response As String = Job.GetString
        Log("Response: " & response)
        ' Handle the response here
    Else
        Log("Error: " & Job.ErrorMessage)
    End If
    Job.Release
[QUOTE]
End Sub

The error I get is:

*** Receiver (httputils2service) Receive (first time) ***
tagSocket(104) with statsTag=0xffffffff, statsUid=-1
ResponseError. Reason: , Response: {
"error": {
"code": 400,
"message": "Invalid JSON payload received. Unknown name \"text\" at 'contents[0]': Cannot find field.\nInvalid JSON payload received. Unknown name \"type\" at 'contents[1]': Cannot find field.\nInvalid JSON payload received. Unknown name \"inline_data\" at 'contents[1]': Cannot find field.",
"status": "INVALID_ARGUMENT",
"details": [
{
"@type": "type.googleapis.com/google.rpc.BadRequest",
"fieldViolations": [
{
"field": "contents[0]",
"description": "Invalid JSON payload received. Unknown name \"text\" at 'contents[0]': Cannot find field."
},
{
"field": "contents[1]",
"description": "Invalid JSON payload received. Unknown name \"type\" at 'contents[1]': Cannot find field."
},
{
"field": "contents[1]",
"description": "Invalid JSON payload received. Unknown name \"inline_data\" at 'contents[1]': Cannot find field."
}
]
}
]
}
}
Error: {

[/QUOTE]

 

[teddybear]

Check your b4xcode, the generated JSONString is different from curl- d {jsonstring}.

 

[Andrew (Digitwell)]

Log your JSONString after line 50

It looks like you are missing the "parts" part of the JSON .
Lines 42,43 should probably be:

    Dim contentsList As List
    contentsList.Initialize
    contentsList.Add(CreateMap("parts":partsList))

 

[personalapps]

Log your JSONString after line 50

It looks like you are missing the "parts" part of the JSON .
Lines 42,43 should probably be:

    Dim contentsList As List
    contentsList.Initialize
    contentsList.Add(CreateMap("parts":partsList))

Unfortunately, the error still persists:

ResponseError. Reason: , Response: {
"error": {
"code": 400,
"message": "Invalid JSON payload received. Unknown name \"type\" at 'contents[0].parts[1]': Cannot find field.",
"status": "INVALID_ARGUMENT",
"details": [
{
"@type": "type.googleapis.com/google.rpc.BadRequest",
"fieldViolations": [
{
"field": "contents[0].parts[1]",
"description": "Invalid JSON payload received. Unknown name \"type\" at 'contents[0].parts[1]': Cannot find field."
}
]
}
]
}
}
Error: {
"error": {
"code": 400,
"message": "Invalid JSON payload received. Unknown name \"type\" at 'contents[0].parts[1]': Cannot find field.",
"status": "INVALID_ARGUMENT",
"details": [

 

[teddybear]

Unfortunately, the error still persists:

Obviously,the element type is inluded in Json payload. it is at line 30 in your b4xcode. but there is no such element in curl -d, as I said at post#2, find out the differences between b4x and curl json string.

 

[aeric]

Similar issue with -d parameter.

Curl Command using httpjob

Can anyone help me execute this curl command using httpjob. The Dropbox API documentation refers to this curl command: curl https://api.dropbox.com/oauth2/token \ -d grant_type=refresh_token \ -d refresh_token= \ -d client_id= \ -d client_secret= I have found a few httpjob...

www.b4x.com

 

[Andrew (Digitwell)]

Unfortunately, the error still persists:

Where is the log output of the produced JsonString from line 50. That would help a lot.

Also this is a different error. It now reads more of the JSON string but as @teddybear says you have extra elements which need to be corrected. It's all in the error message.

 

[personalapps]

Obviously,the element type is inluded in Json payload. it is at line 30 in your b4xcode. but there is no such element in curl -d, as I said at post#2, find out the differences between b4x and curl json string.

Obviously removing 'part2.Put("type", "inline_data")' works!

Sub SendSignatureToGemini
    ' Capture the signature as an image
    
    Dim bmp As B4XBitmap = SignatureTemplate.Bitmap
    
    ' Convert the bitmap to a byte array
    Dim ImageBytes() As Byte = ImageToBytes(bmp, "PNG")
    
    If ImageBytes = Null Then
        Log("Failed to convert image to bytes")
        Return
    End If
    
    ' Convert the byte array to a Base64 string
    Dim Base64Image As String = su.EncodeBase64(ImageBytes)
    
    ' Prepare the JSON payload
    Dim JSON As JSONGenerator
    Dim partsList As List
    partsList.Initialize
    
    ' Part 1: Text part
    Dim part1 As Map
    part1.Initialize
    part1.Put("text", "do math")'changed prompt to do some useful stuff
    partsList.Add(part1)
    
    ' Part 2: Inline data part
    Dim part2 As Map
    part2.Initialize
    Dim inline_data As Map
    inline_data.Initialize
    inline_data.Put("mime_type", "image/png")
    inline_data.Put("data", Base64Image)
    
    part2.Put("inline_data", inline_data)
    partsList.Add(part2)
        
    ' Contents List
    Dim contentsList As List
    contentsList.Initialize
    'contentsList.Add(partsList)
    contentsList.Add(CreateMap("parts":partsList))
    
    Dim rootMap As Map
    rootMap.Initialize
    rootMap.Put("contents", contentsList)
    
    JSON.Initialize(rootMap)
    Dim JSONString As String = JSON.ToString
    
    ' Set up the HttpJob
    Dim Job As HttpJob
    Job.Initialize("GeminiJob", Me)
    Job.PostString($"https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-flash:generateContent?key=${YOUR_GOOGLE_API_KEY}"$, JSONString)
    Job.GetRequest.SetContentType("application/json")
End Sub