Android Question XmlSax is crashing when using Hebrew

Shay · May 10, 2016

Using this code:

Getting the XML file from server:

B4X:

        Select Job.JobName
            Case "Post1"

                Dim OutStream As OutputStream
                OutStream = File.OpenOutput(File.DirInternal, "Global.xml", False) ' Job.Tag is read to set the Original Filename we specify earlier in the creation of the Job
                File.Copy2(Job.GetInputStream,OutStream) ' save the file
                OutStream.Close
                in = File.OpenInput(File.DirInternal, "Global.xml")
                parser.Parse(in, "Parser")
                in.Close

XML pharsing

B4X:

Sub Parser_EndElement (Uri As String, Name As String, Text As StringBuilder)
    If parser.Parents.IndexOf("item") > -1 Then
        If Name = "title" Then
            Title = Text.ToString
        Else If Name = "link" Then
            Link = Text.ToString
        Else if Name = "color" Then
            TitleColor = Text.ToString   
        End If
    End If
    If Name = "item" Then
        clv1.AddTextItem(Title, Link) 'add the title as the text and the link as the value   
    End If
End Sub

XML file:

B4X:

<?xml version="1.0" encoding="windows-1255"?>

<rss version="2.0">
    <channel>
       
        <item>
            <title>ששש</title>
            <color>5</color>
            <link>1</link>
            <description>none</description>
        </item>

        <item>
            <title>bbb</title>
            <color>6</color>
            <link>2</link>
            <description>none</description>
        </item>

    </channel>
</rss>

logs:

B4X:

MainMenu, Response from server: <?xml version="1.0" encoding="windows-1255"?>
<rss version="2.0">
    <channel>
      
        <item>
            <title>���</title> <--------- should be Hebrew
            <color>5</color>
            <link>1</link>
            <description>none</description>
        </item>
        <item>
            <title>bbb</title>
            <color>6</color>
            <link>2</link>
            <description>none</description>
        </item>
    </channel>
</rss>
Error occurred on line: 103 (MainMenu)
org.apache.harmony.xml.ExpatParser$ParseException: At line 7, column 10: not well-formed (invalid token)
    at org.apache.harmony.xml.ExpatParser.parseFragment(ExpatParser.java:515)
    at org.apache.harmony.xml.ExpatParser.parseDocument(ExpatParser.java:474)
    at org.apache.harmony.xml.ExpatReader.parse(ExpatReader.java:321)
    at org.apache.harmony.xml.ExpatReader.parse(ExpatReader.java:279)
    at anywheresoftware.b4a.objects.SaxParser.parse(SaxParser.java:80)
    at anywheresoftware.b4a.objects.SaxParser.Parse(SaxParser.java:73)
    at java.lang.reflect.Method.invokeNative(Native Method)
    at java.lang.reflect.Method.invoke(Method.java:511)
    at anywheresoftware.b4a.shell.Shell.runMethod(Shell.java:702)
    at anywheresoftware.b4a.shell.Shell.raiseEventImpl(Shell.java:339)
    at anywheresoftware.b4a.shell.Shell.raiseEvent(Shell.java:246)
    at java.lang.reflect.Method.invokeNative(Native Method)
    at java.lang.reflect.Method.invoke(Method.java:511)
    at anywheresoftware.b4a.ShellBA.raiseEvent2(ShellBA.java:134)
    at anywheresoftware.b4a.BA$2.run(BA.java:328)
    at android.os.Handler.handleCallback(Handler.java:615)
    at android.os.Handler.dispatchMessage(Handler.java:92)
    at android.os.Looper.loop(Looper.java:137)
    at android.app.ActivityThread.main(ActivityThread.java:4921)
    at java.lang.reflect.Method.invokeNative(Native Method)
    at java.lang.reflect.Method.invoke(Method.java:511)
    at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:1027)
    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:794)
    at dalvik.system.NativeStart.main(Native Method)
** Service (starter) Destroy **
** Activity (mainmenu) Pause, UserClosed = true **
** Service (widget) Start **

Erel · May 10, 2016

The file is not encoded in UTF8. You need to use TextReader to load it with the correct encoding:

B4X:

Dim tr As TextReader
tr.Initialize2(in, "Windows-1255")
parser.Parse2(tr, ...)

moster67 · May 10, 2016

You should be able to use my ICUB4A-library to help you to determine the character-encoding.
https://www.b4x.com/android/forum/threads/icub4a-detecting-character-encoding-formats.65411/

Shay · May 10, 2016

parser.Parse (this must be inputSteam - I am getting compile error

B4X:

Compiling generated Java code.    Error
B4A line: 140
parser.Parse(tr, \
javac 1.8.0_73
src\Av\co\il\mainmenu.java:536: error: incompatible types: BufferedReader cannot be converted to InputStream
mostCurrent._parser.Parse((java.io.InputStream)(_tr.getObject()),"Parser");

B4X:

    Dim OutStream As OutputStream
                OutStream = File.OpenOutput(File.DirInternal, "Global.xml", False) ' Job.Tag is read to set the Original Filename we specify earlier in the creation of the Job
                File.Copy2(Job.GetInputStream,OutStream) ' save the file
                OutStream.Close
                in = File.OpenInput(File.DirInternal, "Global.xml")
                in.Close
                Dim tr As TextReader
                tr.Initialize2(in, "Windows-1255")
                parser.Parse(tr, "Parser")
                tr.Close

I also tried in = tr

DonManfred · May 10, 2016

i guess tr should be a inputstream? The XX in parser.Parse(XX,yyy)
Why not using Job.GetInputstream at this point then? In fact you dont nee to save the stream to a xml first.

Forget it. i think i am wrong.

Shay · May 10, 2016

I bypass the above issue using this, so now this is not hebrew issue, still it is crashing with different stacks, if title is hebrew or title is english
I think I will move to DB instead of XML files, this library is not stable

code I used:

B4X:

ub ReadTextReader (FileName As String ,NewFileName As String)
    Dim TextReader1 As TextReader
    Dim TextWriter1 As TextWriter
    TextReader1.Initialize2(File.OpenInput(File.DirInternal, FileName ),"Windows-1255")
    TextWriter1.Initialize2(File.OpenOutput(File.DirInternal, NewFileName , False),"Windows-1255")
    Dim line As String
    line = TextReader1.ReadLine   
    Do While line <> Null
        Log(line) 'write the line to LogCat
        line = TextReader1.ReadLine
        TextWriter1.WriteLine(line)
    Loop
    TextReader1.Close
    TextWriter1.Close
End Sub

and then I am using
ReadTextReader("Global.xml", "Global1.xml")
in = File.OpenInput(File.DirInternal, "Global1.xml")

stack while using hebrew:

B4X:

MainMenu, Response from server: <?xml version="1.0" encoding="windows-1255"?>
<rss version="2.0">
    <channel>
       
        <item>
            <title>���</title>
            <color>5</color>
            <link>1</link>
            <description>none</description>
        </item>
        <item>
            <title>bbb</title>
            <color>6</color>
            <link>2</link>
            <description>none</description>
        </item>
    </channel>
</rss>
<?xml version="1.0" encoding="windows-1255"?>
<rss version="2.0">
    <channel>
       
        <item>
            <title>ששש</title>
            <color>5</color>
            <link>1</link>
            <description>none</description>
        </item>
        <item>
            <title>bbb</title>
            <color>6</color>
            <link>2</link>
            <description>none</description>
        </item>
    </channel>
</rss>
Error occurred on line: 104 (MainMenu)
org.apache.harmony.xml.ExpatParser$ParseException: At line 6, column 10: not well-formed (invalid token)
    at org.apache.harmony.xml.ExpatParser.parseFragment(ExpatParser.java:515)
    at org.apache.harmony.xml.ExpatParser.parseDocument(ExpatParser.java:474)
    at org.apache.harmony.xml.ExpatReader.parse(ExpatReader.java:321)
    at org.apache.harmony.xml.ExpatReader.parse(ExpatReader.java:279)
    at anywheresoftware.b4a.objects.SaxParser.parse(SaxParser.java:80)
    at anywheresoftware.b4a.objects.SaxParser.Parse(SaxParser.java:73)
    at java.lang.reflect.Method.invokeNative(Native Method)
    at java.lang.reflect.Method.invoke(Method.java:511)
    at anywheresoftware.b4a.shell.Shell.runMethod(Shell.java:702)
    at anywheresoftware.b4a.shell.Shell.raiseEventImpl(Shell.java:339)
    at anywheresoftware.b4a.shell.Shell.raiseEvent(Shell.java:246)
    at java.lang.reflect.Method.invokeNative(Native Method)
    at java.lang.reflect.Method.invoke(Method.java:511)
    at anywheresoftware.b4a.ShellBA.raiseEvent2(ShellBA.java:134)
    at anywheresoftware.b4a.BA$2.run(BA.java:328)
    at android.os.Handler.handleCallback(Handler.java:615)
    at android.os.Handler.dispatchMessage(Handler.java:92)
    at android.os.Looper.loop(Looper.java:137)
    at android.app.ActivityThread.main(ActivityThread.java:4921)
    at java.lang.reflect.Method.invokeNative(Native Method)
    at java.lang.reflect.Method.invoke(Method.java:511)
    at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:1027)
    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:794)
    at dalvik.system.NativeStart.main(Native Method)

Erel · May 10, 2016

The library is stable. Please create a small project that demonstrates the problem and upload it (with the XML file).

Shay · May 10, 2016

Erel just use the demo project that is attached to the library and use the XML I posted above, change first title to Hebrew
The XML need to be downloaded from some web server

Erel · May 11, 2016

The error you encounter is related to text encoding. The only way to test it is to either download the XML directly from the web server or if you upload the downloaded file.

Shay · May 11, 2016

What do you mean direct download, this is what I did, first downloaded the file

Erel · May 11, 2016

If you want me to help you then you need to upload a project that demonstrates the problem. Either include the code that download the XML file or include the XML file itself and load it from File.DirAssets.

ilan · May 11, 2016

Shay said:
parser.Parse (this must be inputSteam - I am getting compile error

Erel said:
Dim tr As TextReader
tr.Initialize2(in, "Windows-1255")
parser.Parse2(tr, ...)

you can see that erel is using parser.Parse2(...,...) "2"!!!

Android Question XmlSax is crashing when using Hebrew

Shay

Well-Known Member

Erel

B4X founder

moster67

Expert

Shay

Well-Known Member

DonManfred

Expert

Shay

Well-Known Member

Erel

B4X founder

Shay

Well-Known Member

Erel

B4X founder

Shay

Well-Known Member

Erel

B4X founder

ilan

Expert

Similar Threads