Android Question Best way to upload and download files to internet? Simple example?

RB Smissaert · Feb 18, 2025

Have a large amount of data that would be too much to go into the .apk file.
What would be the best way to upload this data (ideally from the phone) and then download it and process it?
The data will be in a simple .csv or .txt file.
Once I have the data in File.DirRootExternal I can process the .csv or .txt file and move the data to SQLite.
Did quite a bit of searching the forum, but not clear to me what the best way would be.

RBS

aeric · Feb 18, 2025

Where is the server or cloud storage?
There are many options. FTP, API, webservices...
Which one have you tried?

DonManfred · Feb 18, 2025

RB Smissaert said:
Once I have the data in File.DirRootExternal

- You CAN NOT write to this folder. It is restricted in higher androids.
- Use okhttputils2 to download the file(s).

[B4X] OkHttpUtils2 with Wait For

Downloading resources is simpler with the new Resumable Subs feature. Using Wait For we can wait for the JobDone event in the same sub that started the download. No longer is it needed to have a single sub that handles all requests results. Simplest example: Dim j As HttpJob j.Initialize(""...

www.b4x.com

RB Smissaert · Feb 18, 2025

aeric said:
Where is the server or cloud storage?
There are many options. FTP, API, webservices...
Which one have you tried?

Tried nil yet, just starting up with this.
Ideally it would be a free option, which as I understand should be possible.

RBS

RB Smissaert · Feb 18, 2025

DonManfred said:
- You CAN NOT write to this folder. It is restricted in higher androids.
- Use okhttputils2 to download the file(s).

[B4X] OkHttpUtils2 with Wait For

Downloading resources is simpler with the new Resumable Subs feature. Using Wait For we can wait for the JobDone event in the same sub that started the download. No longer is it needed to have a single sub that handles all requests results. Simplest example: Dim j As HttpJob j.Initialize(""...

www.b4x.com

Ah, yes. I moved files to there from a Windows PC, but that is different.
Will have a look at okhttputils2.

RBS

aeric · Feb 18, 2025

RB Smissaert said:
Ideally it would be a free option, which as I understand should be possible.

Cloud storages like Google Drive and Dropbox are free. You may need to use a library or OkHttpUtils2 to work with the API.
You may also look for free web hosting. Create a PHP webservice that let you upload and download your files.

RB Smissaert · Feb 18, 2025

aeric said:
Cloud storages like Google Drive and Dropbox are free. You may need to use a library or OkHttpUtils2 to work with the API.
You may also look for free web hosting. Create a PHP webservice that let you upload and download your files.

How about FirebaseStorage?
There are a lot of options, which makes it all a bit more tricky.

RBS

RB Smissaert · Feb 18, 2025

DonManfred said:
- You CAN NOT write to this folder. It is restricted in higher androids.
- Use okhttputils2 to download the file(s).

[B4X] OkHttpUtils2 with Wait For

Downloading resources is simpler with the new Resumable Subs feature. Using Wait For we can wait for the JobDone event in the same sub that started the download. No longer is it needed to have a single sub that handles all requests results. Simplest example: Dim j As HttpJob j.Initialize(""...

www.b4x.com

I can do this OK:

Dim su As StringUtils
su.SaveCSV(File.DirRootExternal, "1.csv", ",", table)

This is with Android version 14

Manifest:
AddManifestText(
<uses-sdk android:minSdkVersion="5" android:targetSdkVersion="34"/>

What exactly do you mean with you can not write to this folder?
Did you mean download to that folder?

RBS

klaus · Feb 18, 2025

RB Smissaert said:
su.SaveCSV(File.DirRootExternal, "1.csv", ",", table)

Google does no more allow a direct access to File.DirRootExternal.
You could use RuntimePermissions.GetSafeDirDefaultExternal.

RB Smissaert · Feb 18, 2025

klaus said:
Google does no more allow a direct access to File.DirRootExternal.
You could use RuntimePermissions.GetSafeDirDefaultExternal.

Ok, so it is Google problem, not an Android problem? Never came across that one. Thanks for clearing that up.

RBS

emexes · Feb 18, 2025

RB Smissaert said:
a large amount of data

What is "large", approximately? Tens or hundreds of megabytes or gigabytes?

RB Smissaert said:
The data will be in a simple .csv or .txt file.

Should compress down nicely. APK files are compressed. Is APK still too large, even after this?

Bear in mind that HTTP works for any type of file, not just HTML, TXT, etc.

If it is CSV data and you're not worried about user download quota or cost then... Google Docs has a public download link feature that delivers a Sheet in CSV or TSV format. Works great. But it's in uncompressed cleartext.

Another free-hosting way of doing it would be to compress the data, and then store it as pixel data in a lossless image format, and host that on a picture sharing site. If site has a picture file size limit, then can store as multiple picture files ie bigdata1.png bigdata2.png bigdata3.png... or maybe holiday1.png holiday2.png holiday3.png... to keep things innocuous

GitHub projects let you store files, and I'm pretty sure they can be directly linked to and download using HTTP.

Dumb question, but just in case: data is the same for all users, right?

RB Smissaert · Feb 18, 2025

emexes said:
What is "large", approximately? Tens or hundreds of megabytes or gigabytes?

Should compress down nicely. APK files are compressed. Is APK still too large, even after this?

Bear in mind that HTTP works for any type of file, not just HTML, TXT, etc.

If it is CSV data and you're not worried about user download quota or cost then... Google Docs has a public download link feature delivers a Sheet in CSV or TSV format. Works great. But it's in uncompressed cleartext.

Another free-hosting way of doing it would be to compress the data, and then store it as pixel data in a lossless image format, and host that on a picture sharing site. If site has a picture file size limit, then can store as multiple picture files ie bigdata1.png bigdata2.png bigdata3.png... or maybe holiday1.png holiday2.png holiday3.png... to keep things innocuous

GitHub projects let you store files, and I'm pretty sure they can be directly linked to and download using HTTP.

Dumb question, but just in case: data is the same for all users, right?

It is actually not that much, 15 .cvs files, altogether uncompressed 102 Mb, compressed (7-zip) 11Mb.
Then there are also 4 SQLite databases, altogether a bit less than 3Mb, uncompressed, not sure they compress well.
Perhaps that is OK in an apk file.

RBS

RB Smissaert · Feb 18, 2025

RB Smissaert said:
It is actually not that much, 15 .cvs files, altogether uncompressed 102 Mb, compressed (7-zip) 11Mb.
Then there are also 4 SQLite databases, altogether a bit less than 3Mb, uncompressed, not sure they compress well.
Perhaps that is OK in an apk file.

RBS

Forgot to say that the current apk, without those 19 added data files is 14Mb.

RBS

emexes · Feb 18, 2025

I probably should have mentioned that I've recently had a similar use-case involving an AutoCAD DXF file, which is plain ASCII text of mostly-numeric data, and it compressed up and travelled to the phone via APK no problem:

DIR:

30/01/2025  05:53 PM       225,285,470 test.dxf
10/02/2025  12:57 AM        29,739,665 testdxf.zip
10/02/2025  03:21 PM        29,469,332 testdxf.dxf.gz

ie it is a method that works.

emexes · Feb 18, 2025

If you use the included-with-B4X gzip compression, then you can decompress it on the fly via a Stream, so that it looks to your program like you're reading the uncompressed file directly. Saves on needlessly chewing up phone storage space. And, depending on the phone storage transfer rate, might even read faster too.

I compressed test.dxf on a Windows computer using a ~~five~~ ten-line B4J program. Worked a treat.

B4X:

Log("Hello world!!!")

Dim fdir As String = "d:\dxf\test"
Dim fname As String = "test.dxf"
Dim cname As String = fname & ".gz"

Dim compress As CompressedStreams

Dim b() As Byte = File.ReadBytes(fdir, fname)
Dim c() As Byte = compress.CompressBytes(b, "gzip")
File.WriteBytes(fdir, cname, c)

Log(File.Size(fdir, fname) & TAB & fname)
Log(File.Size(fdir, cname) & TAB & cname)

RB Smissaert · Feb 18, 2025

emexes said:
I probably should have mentioned that I've recently had a similar use-case involving an AutoCAD DXF file, which is plain ASCII text of mostly-numeric data, and it compressed up and travelled to the phone via APK no problem:

DIR:

30/01/2025 05:53 PM 225,285,470 test.dxf 10/02/2025 12:57 AM 29,739,665 testdxf.zip 10/02/2025 03:21 PM 29,469,332 testdxf.dxf.gz

ie it is a method that works.

Ok, that sounds promising then.
I take it B4A has no trouble unzipping those files.

RBS

RB Smissaert · Feb 18, 2025

emexes said:
If you use the included-with-B4X gzip compression, then you can decompress it on the fly via a Stream, so that it looks to your program like you're reading the uncompressed file directly. Saves on needlessly chewing up phone storage space. And, depending on the phone storage transfer rate, might even read faster too.

Ok, thanks, will try that then.

RBS

emexes · Feb 18, 2025

RB Smissaert said:
I take it B4A has no trouble unzipping those files.

No problem. Worked first go too, which is always nice, albeit tingles my spidey sense.

I would expect same of B4I also, but haven't actually done it.

emexes · Feb 18, 2025

RB Smissaert said:
will try that then

Which reminds me: didn't the APK compression (pretty sure it's a Zip file) work as (un)advertised? You might be doing stuff you don't need to do. Although it'll chew up storage on the phone compared to if you decompress on-the-fly whilst reading, so...

Unless you can read directly from File.DirAssets... tbh I don't think I've ever tried that... always copied to File.DirInternal first... I think was because RandomAccessFile operations didn't work with DirAssets files. Maybe sequential reading via a Stream works ok.

emexes · Feb 19, 2025

Just confirming for anyone who might find it useful to know:

.APK and .JAR files are Zip files, same as with Microsoft Office .DOCX .XLSX etc files and Open Office .ODT .ODS etc files.

7-Zip File Manager will open them directly.

Other Zip File Manager programs I've used, needed the file extension renamed to .ZIP first.

I just realised that you could probably ~~hide~~ transport files surruptitiously by embedding them within an otherwise-innocuous spreadsheet etc. Bonus!

Android Question Best way to upload and download files to internet? Simple example?

Well-Known Member

Expert

Expert

Well-Known Member

Well-Known Member

Expert

Well-Known Member

Well-Known Member

Expert

Well-Known Member

Expert

Well-Known Member

Well-Known Member

Expert

Expert

Well-Known Member

Well-Known Member

Expert

Expert

Expert

Similar Threads

Privacy & Transparency

Privacy & Transparency