I have been fighting with this problem for 2 days.
I have checked every google search result I can find.
I am trying to follow the instructions here: https://platform.openai.com/docs/api-reference/uploads/add-part
The curl command to upload a part of a file is here:
curl https://api.openai.com/v1/uploads/upload_abc123/parts
-F data="aHR0cHM6Ly9hcGkub3BlbmFpLmNvbS92MS91cGxvYWRz..."
I have used Perplexity AI and OpenAI to try to convert that to B4J code correctly.
I have use this converter to convert it to okhttp which B4J supports:
https://curlconverter.com/java-okhttp/
I have put the response from there into Perplexity and OpenAI to try to get workable code.
With innumerable iterations and changes, I have not gotten it to work.
The code works properly to initialize the upload of parts, but the uploading of the parts themselves returns an error.
The closest I have come is this code:
For all other attempts, I have gotten a response that the server can’t understand the request.
For the above code I got this error:
09/10/2024 12:19:12 - OpenAI Error: uploadPartJob: {
"error": {
"message": "'data' is a required property",
"type": "invalid_request_error",
"param": null,
"code": null
}
}
Even though I have the “data” property in the request, I still get that error as you can see.
There are no examples online that I can find for doing multi-part uploads. There is plenty of examples about uploading files of 512MB or less, but nothing about multi-part uploads (which are required for files greater than 512 MB).
I have tried to find examples here to convert to B4J as well with no luck: https://github.com/openai/openai-python/tree/main
I would sure appreciate any help you can give on this. Attached is a small program to show the problem. It has my OpenAI key in it so please don't abuse it. I will disable that key when this is resolved.
I have checked every google search result I can find.
I am trying to follow the instructions here: https://platform.openai.com/docs/api-reference/uploads/add-part
The curl command to upload a part of a file is here:
curl https://api.openai.com/v1/uploads/upload_abc123/parts
-F data="aHR0cHM6Ly9hcGkub3BlbmFpLmNvbS92MS91cGxvYWRz..."
I have used Perplexity AI and OpenAI to try to convert that to B4J code correctly.
I have use this converter to convert it to okhttp which B4J supports:
https://curlconverter.com/java-okhttp/
I have put the response from there into Perplexity and OpenAI to try to get workable code.
With innumerable iterations and changes, I have not gotten it to work.
The code works properly to initialize the upload of parts, but the uploading of the parts themselves returns an error.
The closest I have come is this code:
B4J code to upload part of a file to OpenAI:
#Region Shared Files
#CustomBuildAction: folders ready, %WINDIR%\System32\Robocopy.exe,"..\..\Shared Files" "..\Files" ' This line defines a custom build action that uses Robocopy to synchronize files from the "Shared Files" directory to the "Files" directory.
'Ctrl + click to sync files: ide://run?file=%WINDIR%\System32\Robocopy.exe&args=..\..\Shared+Files&args=..\Files&FilesSync=True ' This line provides a clickable link in the IDE to manually trigger the file synchronization using Robocopy.
#End Region
'Ctrl + click to export as zip: ide://run?File=%B4X%\Zipper.jar&Args=Project.zip ' This line provides a clickable link in the IDE to export the project as a zip file using Zipper.jar.
Sub Class_Globals
Private Root As B4XView ' Root view of the page.
Private xui As XUI ' Cross-platform UI library.
Private OpenAIapiKey As String = "My API Key is in the attached zip file. Please don't abuse it."
Private Label1 As Label
End Sub
Public Sub Initialize ' Initialization method for the class.
Button1_Click
Label1.Initialize("")
Label1.Text = "Please watch the logs. It takes 5 or 10 minutes to get the final response." & CRLF & "See line 30 and change it to a folder you like and line 36 to a file you like." & CRLF & "Also, see line 70 of the code. I limited it to 1 part becuse I run out of memory otherwise. I will fix that after I get an appropriate response from a part upload."
End Sub
'This event will be called once, before the page becomes visible.
Private Sub B4XPage_Created (Root1 As B4XView)
Root = Root1 ' Assign the root view.
Root.LoadLayout("MainPage") ' Load the layout named "MainPage".
End Sub
'You can see the list of page related events in the B4XPagesManager object. The event name is B4XPage.
Private Sub Button1_Click ' Event handler for Button1 click.
Private uploadApiUrl As String = "https://api.openai.com/v1/uploads"
Dim Folder As String = "D:\Downloads"
Log("File.DirApp = " & File.DirApp)
Log("File.DirAssets = " & File.DirAssets) ' This is the Files Folder.
Log("File.DirTemp = " & File.DirTemp)
Dim FileName As String = "uploadtest.zip"
Dim Purpose As String = "assistants"
Dim Job As HttpJob
Job.Initialize("initializeUploadJob", Me)
Dim FilePath As String = Folder & "\" & FileName
Job.Tag = FilePath
Dim fileSize As Long = File.Size(Folder, FileName)
Dim mimeType As String = GetMimeType1(FileName) ' Adjust MIME type as needed
Dim json As String = $"{"filename": "${FileName}", "purpose": "${Purpose}", "bytes": ${fileSize}, "mime_type": "${mimeType}"}"$
Job.PostString(uploadApiUrl, json)
Job.GetRequest.SetHeader("Authorization", "Bearer " & OpenAIapiKey)
Job.GetRequest.SetContentType("application/json")
Job.GetRequest.SetHeader("OpenAI-Beta", "assistants=v2")
Wait For (Job) JobDone (Job As HttpJob) ' Wait for the job to complete.
If Job.Success Then
LogColor("Job = " & Job,0xFF006D11) ' Log the job
Log("Initialize Upload Job Response = " & Job.GetString) ' Log the job result.
' Convert JSON string to map
Dim parser As JSONParser
parser.Initialize(Job.GetString)
Dim result As Map = parser.NextObject
Dim FileID As String = result.Get("id")
Log("File ID = " & FileID)
' Proceed to upload parts
Dim fileSize As Long = File.Size(Folder, FileName)
Log("File Size = " & fileSize)
Dim partSize As Long = 64 * 1024 * 1024 ' 64 MB
Dim partCount As Int = Ceil(fileSize / partSize)
Log("PartCount = " & partCount)
For i = 0 To 1 'partCount - 1
Dim start As Long = i * partSize
Dim end1 As Long = Min((i + 1) * partSize, fileSize)
'Log("Uploadpart called: " & start & "," & end1 & "," & Folder & "," & FileName & "," & FileID & "," & apiKey)
LogColor("***",0xFFFF3700)
' *** THE CODE WORKS PROPERLY UP TO HERE ***
'Upload Part
Dim partJob As HttpJob
partJob.Initialize("uploadPartJob", Me)
Dim url As String = "https://api.openai.com/v1/uploads/" & FileID & "/parts"
Dim length As Long = end1 - start
Dim raf As RandomAccessFile
raf.Initialize(Folder, FileName, False)
Log("Uploading " & length & " bytes")
Dim partData(length) As Byte
raf.ReadBytes(partData, 0, length, start)
raf.Close
' Create a MultipartFileData for the part data
Dim mfd As MultipartFileData
mfd.Initialize
mfd.Dir = Folder ' No directory needed since we're using raw bytes
mfd.FileName = FileName ' No file name needed
mfd.KeyName = "data" ' The key expected by the server
mfd.ContentType = "multipart/form-data" ' Set appropriate content type
' Create a map for the multipart form data
Dim multipartData As Map
multipartData.Initialize
multipartData.Put("data", partData)
' Post the multipart request
partJob.PostMultipart(url, multipartData, Array(mfd))
partJob.GetRequest.SetHeader("Authorization", "Bearer " & OpenAIapiKey)
partJob.GetRequest.SetContentType("multipart/form-data")
LogColor("Part Job = " & partJob,0xFF006D11) ' Log request for troubleshooting
Wait For (Job) JobDone (Job As HttpJob) ' Wait for the job to complete.
LogColor("PartUpload Job Response = " & Job.GetString,0xFFFF3700) ' Log the job result.
If Job.Success Then
' Convert JSON string to map
Dim parser As JSONParser
parser.Initialize(Job.GetString)
Dim result As Map = parser.NextObject
Dim FileID As String = result.Get("id")
Log("File ID = " & FileID)
Else
' Call error sub for error handling
xui.MsgboxAsync(Job.JobName & ": " & Job.ErrorMessage,"Error")
End If
Job.Release
Next
Else
' Call error sub for error handling
xui.MsgboxAsync(Job.JobName & ": " & Job.ErrorMessage,"Error")
End If
Job.Release
End Sub
Private Sub GetMimeType1(fileName As String) As String
Dim extension As String = fileName.SubString(fileName.LastIndexOf(".") + 1).ToLowerCase
Select extension
Case "txt"
Return "text/plain"
Case "html", "htm"
Return "text/html"
Case "jpg", "jpeg"
Return "image/jpeg"
Case "png"
Return "image/png"
Case "pdf"
Return "application/pdf"
Case "doc"
Return "application/msword"
Case "docx"
Return "application/vnd.openxmlformats-officedocument.wordprocessingml.document"
Case "xls"
Return "application/vnd.ms-excel"
Case "xlsx"
Return "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"
Case Else
Return "application/octet-stream"
End Select
End SubFor all other attempts, I have gotten a response that the server can’t understand the request.
For the above code I got this error:
09/10/2024 12:19:12 - OpenAI Error: uploadPartJob: {
"error": {
"message": "'data' is a required property",
"type": "invalid_request_error",
"param": null,
"code": null
}
}
Even though I have the “data” property in the request, I still get that error as you can see.
There are no examples online that I can find for doing multi-part uploads. There is plenty of examples about uploading files of 512MB or less, but nothing about multi-part uploads (which are required for files greater than 512 MB).
I have tried to find examples here to convert to B4J as well with no luck: https://github.com/openai/openai-python/tree/main
I would sure appreciate any help you can give on this. Attached is a small program to show the problem. It has my OpenAI key in it so please don't abuse it. I will disable that key when this is resolved.
Last edited:
