Android Question how to write text file with append in csv style

[bvonlaar]

Hi all,
I´m working on an app for gps based field data collection. A similiar program I made some years ago for a pda. I have to collect a record like this : 1;53.756932;11.802078;;Zauneidechse;Subadult;
By writeln in VB I get a text file like this:

1;53.756932;11.802078;;Zauneidechse;Subadult;
2;53.756917;11.802033;;Waldeidechse;Juvenil;
...

The file is opened just when the record is collected. After writing the file is closed. The next record is written with append to the file, without reading the complete file in.

Here I thought I´m on a good way:

Dsent = Din1 & ";" & Din2 & ";" & Din3 & ";" & Din4 & ";" & Din5 & ";" & Din6 & ";" & Din7
   
    tw1.Initialize(File.openoutput(File.DirRootExternal, "Mologfiles/MologACSV.csv", True))
    'File.WriteList(File.DirRootExternal, "Mologfiles/MologACSV.csv", Dsent) 'True))
   
    tw1.WriteLine(Dsent)                       
    tw1.Close

By checking the CSV-file I found this output:

1;53.756932;11.802078;;Zauneidechse;Subadult;2;53.756917;11.802033;;Waldeidechse;Juvenil;3;53.756898;11.802057;;Zauneidechse;Weibchen;

Another way could be to read everytime the existing file into a list, adding the new line and write it back as list.
But this wouldn´t be very helpful, because the csv-file could become very big.
Is there a way to realize the file writing similiar like the oldscholl VB?

I checked a lot of threads, but no solution for this problem.
Any help is welcome

Regards,
Benedikt

 

[klaus]

Have you tryed to add a line feed character at the end?
Dsent = Din1 & ";" & Din2 & ";" & Din3 & ";" & Din4 & ";" & Din5 & ";" & Din6 & ";" & Din7 & CRLF

You may consider using a SQLite database.

 

[bvonlaar]

Have you tryed to add a line feed character at the end?
Dsent = Din1 & ";" & Din2 & ";" & Din3 & ";" & Din4 & ";" & Din5 & ";" & Din6 & ";" & Din7 & CRLF

You may consider using a SQLite database.

I will try your suggestion with the line feed character. If this don´t work I think I´ll try it with writing a List.

SQLite database I think is to fat for this small project. I have also to take in consideration who will use this
app. The handling all in all should be very easy. And it should be simple to implement the collected field data
into a GIS like QGIS.

However, I tried just your suggestion with CRLF, but the result was the same.
If you have another idea, please let me know.

Benedikt

 

[stevel05]

How are you checking the CSV file? According to the documentation, WriteLine appends a new line to the end of each written line. Notepad doesn't recognise it, but Programmers Notepad2 does.

 

[Mahares]

As Steve suggested, use Notepad++ to open the file, not Notepad. You can also open it via Excel. Also, you are much better off using WriteList and readList. See below full code for your project:

Sub Globals
    Dim MyList As List
    Dim Dsent As String
    Dim Din1 As Int
    Dim Din2, Din3 As Double
    Dim Din4, Din5, Din6, Din7 As String
End Sub

Sub Activity_Create(FirstTime As Boolean)
    MyList.Initialize   
   
    WriteText  'write to the file
   
    ReadText 'read from the file

End Sub

Sub WriteText   
    Din1=1 
    Din2=53.756932 
    Din3=11.802078 
    Din4="" 
    Din5="Zauneidechse" 
    Din6="Subadult"
    Din7=""
    Dsent = Din1 & ";" & Din2 & ";" & Din3 & ";" & Din4 & ";" & Din5 & ";" & Din6 & ";" & Din7
    MyList.Add(Dsent)
   
    Din1=2 :Din2=53.756917 :Din3=11.802033 :Din4="" :Din5="Waldeidechse" :Din6="Juvenil" :Din7=""
    Dsent = Din1 & ";" & Din2 & ";" & Din3 & ";" & Din4 & ";" & Din5 & ";" & Din6 & ";" & Din7
    MyList.Add(Dsent)
    File.WriteList(File.DirRootExternal, "afile.txt",MyList)
End Sub

Sub ReadText
    File.ReadList(File.DirRootExternal, "afile.txt")
    For Each s As String In MyList
        Log(s)
    Next
End Sub

 

[bvonlaar]

How are you checking the CSV file? According to the documentation, WriteLine appends a new line to the end of each written line. Notepad doesn't recognise it, but Programmers Notepad2 does.

HI Steve05,
now you are confusing me a little bit. I will check my environment. For opening the csv-files I ever use only the editor, because I knew that there exist some problems with
Notepad and other text editors sometimes. I believed I´m on the safe sight using the simple editor.
Well, I just tried the csv-file with Notepad++ and the csv-file appeared in the right manner. Now my confusion is complete, because I never had a problem before with the
Editor and with the PDA-version I work since 7 years without any problems. I also tested the import into QGIS and the file was also correct recognized and displayed.

Any idea about this effect?

But I found also a difference by using CRLF like Klaus suggested. It produce an empty line. But this is only seen in the Notepad++ not seen in the Editor (still display only one line).

I think I have run a lot of tests. But your focus upon the used text editor was very helpful.

Benedikt

 

[bvonlaar]

As Steve suggested, use Notepad++ to open the file, not Notepad. You can also open it via Excel. Also, you are much better off using WriteList and readList. See below full code for your project:

Sub Globals
    Dim MyList As List
    Dim Dsent As String
    Dim Din1 As Int
    Dim Din2, Din3 As Double
    Dim Din4, Din5, Din6, Din7 As String
End Sub

Sub Activity_Create(FirstTime As Boolean)
    MyList.Initialize  
  
    WriteText  'write to the file
  
    ReadText 'read from the file

End Sub

Sub WriteText  
    Din1=1
    Din2=53.756932
    Din3=11.802078
    Din4=""
    Din5="Zauneidechse"
    Din6="Subadult"
    Din7=""
    Dsent = Din1 & ";" & Din2 & ";" & Din3 & ";" & Din4 & ";" & Din5 & ";" & Din6 & ";" & Din7
    MyList.Add(Dsent)
  
    Din1=2 :Din2=53.756917 :Din3=11.802033 :Din4="" :Din5="Waldeidechse" :Din6="Juvenil" :Din7=""
    Dsent = Din1 & ";" & Din2 & ";" & Din3 & ";" & Din4 & ";" & Din5 & ";" & Din6 & ";" & Din7
    MyList.Add(Dsent)
    File.WriteList(File.DirRootExternal, "afile.txt",MyList)
End Sub

Sub ReadText
    File.ReadList(File.DirRootExternal, "afile.txt")
    For Each s As String In MyList
        Log(s)
    Next
End Sub

The comment of Steve was good, because he put again the focus on a theme which I thought to be cleared.
Thanks for your suggestion. This I´ll also try. I think for the moment is it the only way. I wanted to avoid this, because when I want to append the file, I have to read the complete existing file into a list, then add the new line and write it down again. The list can grow easily up to 500 lines. So I have for saving 1 line to push 500 lines, if I understood all right.
This seems to me a little bit resources consuming. I think I´ll read at program start the existing csv-file in and add every new line to the list and write after collecting a new line the expanded list down. The problem is I have to save the list after every new collected line. Only to save the list when the program will be finished is to riskful, because in cause of a technical accident the data are lost and in due to the kind of job the events aren´t reproduceable.

Benedikt

 

[klaus]

If you want to use the file in Windows you should use the windows character set encoding.
Instead of
tw1.Initialize(File.openoutput(File.DirRootExternal, "Mologfiles/MologACSV.csv", True))
you should use:
tw1.Initialize2(File.openoutput(File.DirRootExternal, "Mologfiles/MologACSV.csv", True), "Windows-1252")

Attached my small test program.
You can open the file csv with Excel.

Changed the project, according to post#10.

 

[Mahares]

you should use:
tw1.Initialize2(File.openoutput(File.DirRootExternal, "Mologfiles/MologACSV.csv", True), "Windows-1252")

Even changing the file output format to what @klaus suggested above, when you open the csv file using Notepad, all records are shown in one line. But, using Notepad++, each line is separate as it is supposed to be. Also, in order to parse the file in Excel, the file needs to be called: test.txt instead of test.csv.

 

[klaus]

Thank you Mahares.
Concerning Excel, I can read it as a csv file and all items are shown in a cell.
I had't tested with NotePad, I don't have it on my current computer.
I tested it with Windows NotePad on an old computer with Windows XP and it showed the lines on each line, but with a square at the end.
Using this Notepad on the current computer shows them in one line as you reported.
Then I looked at the file with TotalCommander and saw the the CR character is missing.
I added it:
Dsent = Din1 & ";" & Din2 & ";" & Din3 & ";" & Din4 & ";" & Din5 & ";" & Din6 & ";" & Din7 & Chr(13)
Changed the test project in post#8.

NotePad:

Excel:

 

[Mahares]

Concerning Excel, I can read it as a csv file and all items are shown in a cell.

Yes, you can open and read it via Excel, but if you want to parse it in Excel where each item occupies its own cell, you need to change the extension to txt, Excel wizzard walks you through the parsing process and in this case the delimiter is a semi colon. It will be much more readable.

 

[klaus]

The file is saved with semi colons as the separator character, which is the default in Excel.
I can load it directly with the csv extension and each item is in its own cell!
Look at post#10, I added a screenshot.

 

[bvonlaar]

The file is saved with semi colons as the separator character, which is the default in Excel.
I can load it directly with the csv extension and each item is in its own cell!
Look at post#10, I added a screenshot.

Much thanks Klaus,
I think you got it. I tested it with my program and it works as wanted. Also the simple Editor displays the data correct. For future I learned not to trust only in the Editor ;-).
Now the append data work also fine. If there would be only the way with the list, I would have had a bad feeling, because I know the practise and the situation of the data collectors as well and what all could go wrong.
Also the import to QGIS works. Really good work.
Also thanks to Mahares and Steve. I think this discussion could also help other which come from VB.

Thanks!!
Benedikt