B4J Tutorial Building a mini "Email based server"

Christos Dorotheou · Feb 14, 2014

manios said:
Hi Christos,
Thanks for the Sub. It helps and works OK.
However something is still wrong with the coding of the mails or the decoding. I get intermittend some "=" and "=20" characters/strings in the message body from the Mailparser. This finally screws up the conversion and contents.
I will try to get the raw contents of the mail with an old VB program and see what is wrong

You can store the Raw mail contents, from within the ParseMultipartBody() routine by saving it in a folder of your choice and a file name
with a .txt extension and then you can open it with any text editor and inspect it.

You can even give it an .eml extension and this way is saved as an email which can be opened by many email clients such as Outlook Express.

This is also one way to backup locally your received emails.

Please see bellow:

B4X:

Sub ParseMultipartBody (Mail As String, Msg As Message)
    'find first boundary
    index = Mail.IndexOf2("--" & boundary, index)
    ReadNextLine(Mail)
    Dim headers As StringBuilder
    headers.Initialize
    Do While index < Mail.Length
        Dim line As String
        line = ReadNextLine(Mail)
        If line.Length > 0 Then
            headers.Append(line).Append(" ")
        Else If index < Mail.Length Then
            Dim nextPart As Int
            nextPart = Mail.IndexOf2("--" & boundary, index)
            If nextPart-4 > index Then
                HandlePart(headers.ToString, Mail.SubString2(index, nextPart-4), Msg)
            End If
            If nextPart = -1 Then Return
            index = nextPart
            ReadNextLine(Mail)
            headers.Initialize
        End If
    Loop
   
    File.WriteString("C:\Temp", "AnyName.eml", Mail)
    File.WriteString("C:\Temp", "AnyName.txt", Mail)
   
End Sub

manios · Feb 14, 2014

Checking the raw contents of the messages, I finally found out that the message body does contain incorrect characters. However displaying the .eml file with an email-client these characters are not shown. The mails are generated from a form on a TYPO3 website, need to go back to the originator. Its very strange!
Manios

Christos Dorotheou · Feb 14, 2014

manios said:
Hi Christos,
Thanks for the Sub. It helps and works OK.
However something is still wrong with the coding of the mails or the decoding. I get intermittend some "=" and "=20" characters/strings in the message body from the Mailparser. This finally screws up the conversion and contents.
I will try to get the raw contents of the mail with an old VB program and see what is wrong

Manios,

Please bare in mind that UTF2UNI() routine works flawsely for encoded sentences with UTF-8 Character Set for the time being, but probably will return some strange results back if the character set being used to encode the text is ISO-8859-x.
I am working on a more robust Routine which will handle the encoded sentences based on the Character Set used.

As far as the "=" and "=20" is concerned characters you have mentioned, these have a special purpose defined in RFC 822.
Generally when there is a multi-line encoded text, each line must be terminated with "=" to help parsers, and as far "=20" is concerned this is a SPACE Character.

Ok this is the upgraded UTF2UNI() routine which will decode the encoded text passed to the routine, based on the Character Set used.

B4X:

Sub UTF2UNI(s2match As String, CharSet As String) As String
Dim m As Matcher
Dim Parts As List
Dim cnv As ByteConverter
Dim i, j As Int
Dim i2s As String
Dim SupportedEncodings() As String
Dim booCharSetIsValid As Boolean

Parts.Initialize
SupportedEncodings = cnv.SupportedEncodings

booCharSetIsValid = False
For j = 0 To SupportedEncodings.Length - 1
    If CharSet.ToUpperCase = SupportedEncodings(j).ToUpperCase Then
      booCharSetIsValid = True
    End If       
Next

If booCharSetIsValid = True Then
  If CharSet.Trim.ToLowerCase = "utf-8" Then
      m = Regex.Matcher("=[0-9a-fA-F][0-9a-fA-F]=[0-9a-fA-F][0-9a-fA-F]",s2match)

      i = -1
      Do While m.Find = True
        i = i + 1
        i2s = i
     
        Do While i2s.Trim.Length < 2
            i2s = "0" & i2s
        Loop   
     
        Parts.Add(m.Group(0).Replace("=",""))
        s2match = s2match.Replace(m.Group(0),"~" & i2s)
        m = Regex.Matcher("=[0-9a-fA-F][0-9a-fA-F]=[0-9a-fA-F][0-9a-fA-F]",s2match)
      Loop

      If Parts.Size > 0 Then
        For i = 0 To Parts.Size - 1
            i2s = i
            Do While i2s.Trim.Length < 2
                i2s = "0" & i2s
            Loop
            s2match = s2match.Replace("~" & i2s, cnv.StringFromBytes(cnv.HexToBytes(Parts.Get(i)), CharSet))
        Next
      End If
  Else
      s2match = s2match.Trim
      s2match = s2match.Replace(" ","=20")
 
      If s2match.EndsWith("=") = True Then
        s2match = s2match.SubString2(0,s2match.Length-1)
      End If     
 
      m = Regex.Matcher("=[0-9a-fA-F][0-9a-fA-F]",s2match)

      i = -1
      Do While m.Find = True
        i = i + 1
        i2s = i
        Do While i2s.Trim.Length < 2
            i2s = "0" & i2s
        Loop
     
        Parts.Add(m.Group(0).Replace("=",""))
        s2match = s2match.Replace(m.Group(0),"~" & i2s)
     
        m = Regex.Matcher("=[0-9a-fA-F][0-9a-fA-F]",s2match)
      Loop

      If Parts.Size > 0 Then
        For i = 0 To Parts.Size - 1
            i2s = i
            Do While i2s.Trim.Length < 2
                i2s = "0" & i2s
            Loop         
            s2match = s2match.Replace("~" & i2s, cnv.StringFromBytes(cnv.HexToBytes(Parts.Get(i)), CharSet))
        Next
      End If 
  End If
End If

Return s2match

End Sub

Christos Dorotheou · Feb 14, 2014

Christos Dorotheou said:

Manios,

Please bare in mind that UTF2UNI() routine works flawsely for encoded sentences with UTF-8 Character Set for the time being, but probably will return some strange results back if the character set being used to encode the text is ISO-8859-x.
I am working on a more robust Routine which will handle the encoded sentences based on the Character Set used.

As far as the "=" and "=20" is concerned characters you have mentioned, these have a special purpose defined in RFC 822.
Generally when there is a multi-line encoded text, each line must be terminated with "=" to help parsers, and as far "=20" is concerned this is a SPACE Character.

Ok this is the upgraded UTF2UNI() routine which will decode the encoded text passed to the routine, based on the Character Set used.

B4X:

Sub UTF2UNI(s2match As String, CharSet As String) As String
Dim m As Matcher
Dim Parts As List
Dim cnv As ByteConverter
Dim i, j As Int
Dim i2s As String
Dim SupportedEncodings() As String
Dim booCharSetIsValid As Boolean

Parts.Initialize
SupportedEncodings = cnv.SupportedEncodings

booCharSetIsValid = False
For j = 0 To SupportedEncodings.Length - 1
    If CharSet.ToUpperCase = SupportedEncodings(j).ToUpperCase Then
      booCharSetIsValid = True
    End If      
Next

If booCharSetIsValid = True Then
  If CharSet.Trim.ToLowerCase = "utf-8" Then
      m = Regex.Matcher("=[0-9a-fA-F][0-9a-fA-F]=[0-9a-fA-F][0-9a-fA-F]",s2match)

      i = -1
      Do While m.Find = True
        i = i + 1
        i2s = i
    
        Do While i2s.Trim.Length < 2
            i2s = "0" & i2s
        Loop  
    
        Parts.Add(m.Group(0).Replace("=",""))
        s2match = s2match.Replace(m.Group(0),"~" & i2s)
        m = Regex.Matcher("=[0-9a-fA-F][0-9a-fA-F]=[0-9a-fA-F][0-9a-fA-F]",s2match)
      Loop

      If Parts.Size > 0 Then
        For i = 0 To Parts.Size - 1
            i2s = i
            Do While i2s.Trim.Length < 2
                i2s = "0" & i2s
            Loop
            s2match = s2match.Replace("~" & i2s, cnv.StringFromBytes(cnv.HexToBytes(Parts.Get(i)), CharSet))
        Next
      End If
  Else
      s2match = s2match.Trim
      s2match = s2match.Replace(" ","=20")

      If s2match.EndsWith("=") = True Then
        s2match = s2match.SubString2(0,s2match.Length-1)
      End If    

      m = Regex.Matcher("=[0-9a-fA-F][0-9a-fA-F]",s2match)

      i = -1
      Do While m.Find = True
        i = i + 1
        i2s = i
        Do While i2s.Trim.Length < 2
            i2s = "0" & i2s
        Loop
    
        Parts.Add(m.Group(0).Replace("=",""))
        s2match = s2match.Replace(m.Group(0),"~" & i2s)
    
        m = Regex.Matcher("=[0-9a-fA-F][0-9a-fA-F]",s2match)
      Loop

      If Parts.Size > 0 Then
        For i = 0 To Parts.Size - 1
            i2s = i
            Do While i2s.Trim.Length < 2
                i2s = "0" & i2s
            Loop        
            s2match = s2match.Replace("~" & i2s, cnv.StringFromBytes(cnv.HexToBytes(Parts.Get(i)), CharSet))
        Next
      End If
  End If
End If

Return s2match

End Sub

Since I have now, got a better understanding of B4J and MIME inner workings, I've rewritten the UTF2UNI() routine
to be more compact and simplified. Sorry if I've got carried away.

Here is the compact version of it:

B4X:

Sub UnMIME(s2match As String, CharSet As String) As String
Dim m As Matcher
Dim cnv As ByteConverter
Dim SupportedEncodings() As String
Dim booCharSetIsValid As Boolean
Dim c2match As String
Dim j As Int

SupportedEncodings = cnv.SupportedEncodings

booCharSetIsValid = False
For j = 0 To SupportedEncodings.Length - 1
    If CharSet.ToUpperCase = SupportedEncodings(j).ToUpperCase Then
      booCharSetIsValid = True
    End If       
Next

If booCharSetIsValid = True Then
  If CharSet.Trim.ToLowerCase = "utf-8" Then
      c2match = "=[0-9a-fA-F][0-9a-fA-F]=[0-9a-fA-F][0-9a-fA-F]"
  Else
      c2match = "=[0-9a-fA-F][0-9a-fA-F]"
      s2match = s2match.Trim
      s2match = s2match.Replace(" ","=20")
      If s2match.EndsWith("=") = True Then
        s2match = s2match.SubString2(0,s2match.Length-1)
      End If     
  End If 
  m = Regex.Matcher(c2match,s2match)
  Do While m.Find = True
      s2match = s2match.Replace(m.Group(0), cnv.StringFromBytes(cnv.HexToBytes(m.Group(0).Replace("=","")), CharSet))
      m = Regex.Matcher(c2match,s2match)
  Loop
End If

Return s2match

End Sub

incendio · Nov 3, 2014

I have an app to send email that based on this example.

My app has an email distribution list, store in sql database.

Codes something like this :

B4X:

Cursor = Sql.ExecQuery("select * from table where tag = 'N' )
Do While Cursor.NextRow
  Id = Cursor.GetInt("ID") 
  To = Cursor.GetString("TO")
  Subject = Cursor.GetString("SUBJECT")
  Body = Cursor.GetString("BODY")
  SMTP.To.Add(To)
  SMTP.Subject = Subject
  SMTP.Body    = Body
  SMTP.Send
Loop
Cursor.Close

Since Cursor can return more than 1 row, should I paused a few second for each rows after email sent or the code above is ok?

Erel · Nov 3, 2014

Please start a new thread for this question.

Beja · Mar 10, 2015

Hello guys.
Today I tried to run the example in the first post and got the following message:

03/09/2015 21:36:23: Error: (RuntimeException) java.lang.RuntimeException: Error during login: -ERR [AUTH] Web login required: https://support.google.com/mail/bin/answer.py?answer=78754

Any explanation appreciated.

Erel · Mar 10, 2015

You need to follow these instructions with gmail: https://support.google.com/accounts/answer/6010255

Beja · Mar 11, 2015

Yep!
That's it.. thanks Erel.
I also got email from Google support saying:

Some examples of apps that do not support the latest security standards include:
•The Mail app on your iPhone or iPad with iOS 6 or below
•The Mail app on your Windows phone preceding the 8.1 release
•Some Desktop mail clients like Microsoft Outlook and Mozilla Thunderbird

does that means Android apps like this one is safe and I can go ahead and set the less-security?

Beja · Mar 11, 2015

Now the app is receiving the messages, but...
I am not sure if this already addressed but I get this error message:

03/11/2015 15:29:49: Found messages: 1
03/11/2015 15:29:49: Message from: #### #### <####@####.###>, subject: 13
03/11/2015 15:29:49: Sending response
03/11/2015 15:29:49: Error sending message: (SSLException) javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection?

Erel · Mar 12, 2015

Can you post the code that initializes the SMTP object?

Beja · Mar 12, 2015

I owe you one Erel,
that was my mistake.. I was supposed to put the complete email address in the User variable instead of just the user ID. now working like a charm and I love B4J.

Xie Xie

codie01 · Jan 31, 2016

Hi All, I have tried the example and get an error that "message" is not a type, "is a Library reference missing" I have jnet installed.

Thanks Phil

Erel · Feb 1, 2016

Message is a a custom type declared in MailParser module.

codie01 · Feb 5, 2016

Good Day All,

I am trying to use this mini server example in B4J and am getting a type error "Message" in the code at the beginning of the following sub:

B4X:

Sub HandleMessage(msg As Message)

What Library is missing for this type please, cannot seem to find a reference.

Kind Regards,
Philip

DonManfred · Feb 5, 2016

codie01 said:
What Library is missing for this type please, cannot seem to find a reference.

Message is declared in MailParser.bas

You already got the answer on Monday (the post before your last one).

Why you are asking the same question again?

3394509365 · Oct 11, 2016

Erel, have you an example, "mail server" for B4A?

Erel · Oct 13, 2016

Check this example: https://www.b4x.com/android/forum/threads/11310/#content

ilan · Dec 31, 2020

thank you very much for this example erel, its perfect!

is it possible to set POP3 to read hebrew letters too? (UTF-8)
what i get now is something like this:

Date: Fri, 17 Jun 2016 09, Message from: ServiceMail@milgam.co.il, subject: =?utf-8?B?15DXmdep15XXqCDXqtep15zXlded?=

Click to expand...

and also i would like to know if the message was read or is still in unread status, is this possible?

Erel · Jan 1, 2021

ilan said:
and also i would like to know if the message was read or is still in unread status, is this possible?

You can delete it from the server. There is no read status that you can change.

B4J Tutorial Building a mini "Email based server"

Attachments

Member

Active Member

Member

Member

Well-Known Member

B4X founder

Expert

B4X founder

Expert

Attachments

Expert

B4X founder

Expert

Active Member

B4X founder

Active Member

Expert

Active Member

B4X founder

Expert

B4X founder