Android Question Read only last line in a file

pazzokli · Dec 30, 2018

Hi forum,
I' ve a big text file with a lot of line. I want read only last line or last x lines without read all the lines.
This file store date and room temperature every 10 minutes and add a line each time. I want display a graph of some hours only, but problem is that I don't know how many lines there are in this file and I implemented a cicle to read all the line and use last 144 (for show 24h) only.
This take a lot of time.
There is a fastest way?

Thanks

Mahares · Dec 31, 2018

pazzokli said:
I want read only last line or last x lines without read all the lines

pazzokli said:
There is a fastest way?

If you are comfortable with SQLite, it is the way to go instead a large text file. You can import the text file into a table.
Suppose you have a table named: tblTemp with 2 cols: TEMP_DATE And TEMP, you can sort the table by any column in any direction and extract any number of records. Example to extract the most recent 144 records:

B4X:

Dim strQuery As String="SELECT * FROM tblTemp ORDER BY TEMP_DATE DESC LIMIT 144"
cursor=Starter.sql1.ExecQuery(strQuery)

mangojack · Dec 31, 2018

If your not up with SQLite .. you could experiment with File.ReadList method .. writing your text file to a list first for easier interaction.

this is only basic hint code .. unsure how and what your writing to the text file. * You would have to include some checks for list size etc.

B4X:

Dim List1 As List
    List1 = File.ReadList(File.DirDefaultExternal, "Text.txt")    
    Log($"List size =  ${List1.Size}"$)
    Log($"This is the last item  ...  ${List1.Get(List1.Size -1}"$))
   
    For i = List1.Size - 144 To List1.Size -1
        Log($"This is item #   ${List1.Get(i}"$))   
    Next

But Personally I would be following @Mahares suggestion above and go the SQLite Database approach.

pazzokli · Dec 31, 2018

I don't know database and SQL. I used access on vb.net only with simple interaction. This is why I store in a text file.
But database is the best solution. I know...
I will try the second solution

Erel · Dec 31, 2018

Tip: You should never use File.DirDefaultExternal.
See the runtime permissions tutorial.

Sandman · Dec 31, 2018

Using a database for this seems like a huge overkill to me. Just run a shell command:

B4X:

tail -n 5 path/to/file

Where you obviously change the path to be correct, and change the 5 to the number of lines you want from the end.

Sorry, can't give better code example now, writing this using mobile, on the go. Search forum for how to run shell commands and get result back.

Erel · Dec 31, 2018

Shell commands in an Android app?! Much better to use a database or just read the whole file with File.ReadList assuming that it is not too large.

Sandman · Dec 31, 2018

Erel said:
Shell commands in an Android app?!

Sure, why not? Is that so bad?

Erel said:
assuming that it is not too large

The poster said it was a big file, so I assumed it was big.

Erel · Dec 31, 2018

Sandman said:
Sure, why not? Is that so bad?

Shell commands can cause all kinds of problems including permission problems, missing commands and others. It will also be slower than reading the file directly from the app.

Sandman · Dec 31, 2018

I'm not going to argue with you about this, you most likely know this a lot better than me. But just to explain my reasoning...

Erel said:
missing commands

That might be correct as a general rule, but tail is such a standard command that I assumed it to always be present.

Erel said:
It will also be slower than reading the file directly from the app.

This surprised me. I would have imagined that tail would be a lot faster considering it's specialized for the job of getting lines from the end, especially for very large files.

Erel said:
permission problems

Yeah, I didn't think of that.

Erel · Dec 31, 2018

Sandman said:
would have imagined that tail would be a lot faster

Remember that the OS needs to start a new process for this. An efficient implementation inside your app will be faster. Though this is not a real issue if you are reading the file every 10 minutes.

Android <> Linux. Accessing the underlying shell commands is not something that should normally be done.

MarkusR · Dec 31, 2018

just save the last 144 rows (for the last 24h) in a separated file from your .net app.
u can use the same file name or for each day the date.
in .net you can store the data in a list.

Sandman · Dec 31, 2018

Erel said:
Accessing the underlying shell commands is not something that should normally be done.

Well, ok then.

pazzokli · Dec 31, 2018

MarkusR said:
just save the last 144 rows (for the last 24h) in a separated file from your .net app.
u can use the same file name or for each day the date.
in .net you can store the data in a list.

I Made the sample of 24h but I could choose 3 days or a week so number of lines is not a fix parameter.

mangojack · Dec 31, 2018

While using SQL db for this might be slight overkill ... it just opens so many possible interactions with your data on a time / date basis.

Trust me .. compared with VB / .Net ... using a SQL database within B4x is an absolute breeze.

Chapter 5 / Page22 of the user guide has an excellent write up to get you started ... and there are many excellant startup tutorials within the forum.

B4A Users Guide ....

klaus · Dec 31, 2018

Chapter 5 / Page22 of the user guide has an excellent write up to get you started

The User's Guide is a bit outdated.
You may have a look at the B4X SQLite Database Booklet.
It has examples with source code.

udg · Dec 31, 2018

Another way could be to open the file and skip the first "x" records.
On the assumption that each row is of equal length, you could adapt the attached code where a couple of alternatives are showed. You should time each one against your real data file in order to understand which one is faster and less resource consuming. Along with the already suggested solutions.

Happy new year!

@Erel : I think that the addition of the "skip" method to the standard File I/O methods could be a nice one; I'm thinking about chunk-reading operations..

Erel · Jan 1, 2019

You can use RandomAccessFile to do it. Though it will be an overkill for a file with a few thousand lines.

udg · Jan 2, 2019

Hi Erel, I must admit that at first I didn't fully understand your comment.
Yesterday I wrote a quick and dirty hexeditor in B4J and the light shed on me again.
So, InputStream is OK for one-way only reading (and skip could be of some help like in the OP), while RAF is a b-oriented, rich featurs mechanism although "heavier" than the former (FileChannel?).

Erel · Jan 2, 2019

udg said:
So, InputStream is OK for one-way only reading (and skip could be of some help like in the OP), while RAF is a b-oriented, rich featurs mechanism although "heavier" than the former (FileChannel?).

That's true.

Android Question Read only last line in a file

Active Member

Expert

Expert

Active Member

B4X founder

Expert

B4X founder

Expert

B4X founder

Expert

B4X founder

Well-Known Member

Expert

Active Member

Expert

Expert

Expert

Attachments

B4X founder

Expert

B4X founder

Similar Threads

Privacy & Transparency

Privacy & Transparency