Android Question Number of lines in a text file

[Sergey_New]

I need to read a text file.
For the progressbar to work, I need to know the number of lines scount.

    Try
        Dim rd As TextReader:
        rd.Initialize(File.OpenInput(Starter.myFolder, FileName))
        scount=Regex.Split(CRLF,rd.ReadAll).Length
        rd.Close
    Catch
        Log(LastException)
    End Try

If the number of lines is less than 90,000, then everything works, but if, for example, there are 1,500,000 lines, the application crashes on line #4.

What restrictions might there be?

 

[DonManfred]

(ErrnoException) android.system.ErrnoException: open failed: ENOENT (No such file or directory)

did you rename the file to be lowercased?
It is named Royal.ged in original... must be royal.ged when reading from assets

Edit: I did not tested in debug at all. I am always using Release

 

[klaus]

It is named Royal.ged in original... must be royal.ged when reading from assets

In Release mode it works with Royal.ged and with royal.ged.
In Debug mode, even with royal.ged the error message is the same.

 

[Daestrum]

What if you change

 Dim strLines As String= rd.ReadLine

to

 Dim strLines As Object= rd.ReadLine

and then add that to the list

 

[Sergey_New]

In reality, populating a special database is much more difficult than

     Dim lst As List
     lst.Initialize
     lst=File.ReadList(Starter.myFolder, FileName)
     Log(lst.Size)

Reading of a large file stops depending on its size at 30% read or more.
All attempts to catch a memory overflow exception have failed.
Is it possible to get an exception when memory is full?

 

[LucaMs]

You could try using the Application_Error event in the Starter service, but then... What?

 

[DonManfred]

Is it possible to get an exception when memory is full?

your app crash when the memory is full. You can not intercept that.
I already told you that you are better of with writing a parser for the textfile which will parse the file line by line and create the db entries. Especially when the files can be big.
You are just ignoring such hints.

It is´nt fun to read this thread too as it has 3 pages already

 

[Sergey_New]

No, I don't ignore it.
Exactly as you advise, the application analyzes each line and fills the database. Below is the entire code for reading the file:

Sub ReadFile(FileName As String, cpb As CircularProgressBar)
    TagsMap.Initialize
    ecount=0
    Starter.IsChanged=False
    Dim c1, progress=1, scount As Int, c2, c3 As String
    Dim isconc As Boolean
    myStack.Initialize
    db.Initialize
    UserTags.Initialize
    UncnowMap.Initialize
    Dim h As Int=0
    fileError=FileName
    Try
        Dim rd As TextReader
        scount=File.ReadList(Starter.myFolder, FileName).Size
        rd.Initialize(File.OpenInput(Starter.myFolder, FileName))
        Dim strLines As String= rd.ReadLine
        Dim r As record
        r.Initialize
        Do While strLines <> Null
            ecount=ecount+1
            Dim sb As StringBuilder: sb.Initialize
            Dim m As Matcher = Regex.Matcher(pat,strLines)
            Do While m.Find
                c1=m.Group(1): c2=m.Group(2): c3=m.Group(3)
            Loop
            Select Case c2
                Case "CONC"
                    r.c3=sb.Append(r.c3).Append(c3)
                    strLines = rd.ReadLine
                Case "CONT"
                    r.c3=sb.Append(r.c3).Append(CRLF).Append(c3)
                    strLines = rd.ReadLine
                Case Else
                    If isconc Then
                        FillDB(r)
                    End If
                    r.c1=c1: r.c2=c2: r.c3=c3
                    isconc=True
                    strLines = rd.ReadLine
            End Select
            If cpb.IsInitialized Then
                Dim x As Int=100*progress/(scount)
                If x>h Then
                    cpb.Value=x
                    Sleep(0)
                    h=x
                End If
                progress=progress+1
            End If
        Loop
    Catch
        Log(LastException)
    End Try
    rd.Close
End Sub

 

[DonManfred]

Below is the entire code for reading the file

do you still get OOM with it?

 

[Sergey_New]

do you still get OOM with it?

I don't understand what it means to get OOM (Out-Of-Memory). I don't find anything similar in the debug log. The application just stops.

 

[MicroDrie]

In addition to the Out-Of-Memory (OOM) problem, there is another problem, namely time. The OS considers running your program to be a problem after a long time and quietly kills it after showing a pop-up message several times. With text reader I could read about 22,698 lines, but without text reader it was after 142,730 lines that the OS thought it had enough and quietly killed my debug mode program.

 

[agraham]

All attempts to catch a memory overflow exception have failed.

You can't catch an OOM exception as the only thing that can be done is to kill the process as it is now in a probably unstable state.

 

[Sergey_New]

and quietly kills it after showing a pop-up message several times

There were no messages.

 

[Sergey_New]

kill the process

How do you tell an application or user that a process needs to be terminated?

 

[RB Smissaert]

I don't understand what it means to get OOM (Out-Of-Memory). I don't find anything similar in the debug log. The application just stops.

Likely if there are no error messages then there are OOM problems or as suggested by MicroDrie

There were no messages.

If there are no messages then likely there are OOM problems or as MicroDrie suggested the OS kills the app as it considers it to be stuck/frozen or whatever it thinks it is doing wrong.
If you are interested I can post code that produces in a loop, small enough lists of arrays that you can then process with no problems.
Maybe best to start a new thread with a more accurate title.

RBS

 

[73Challenger]

This isn't exactly what you asked but... could you estimate the number of lines by getting the SIZE of the file instead of trying to read al the lines?
FileSize = 1,000 bytes
Line size = 100 bytes
Lines = 10
If the lines are all similar in length, this might work.

Second thought, could you keep a count of the lines before you write them out to the file? If you need to store this across sessions you could put it in a sqllite table or a simple text file to restore when the app opens?

A progress bar, especially if you are talking about a massive amount of lines doesn't have to be exactly perfectly accurate. If it's off by 10 your users won't notice.