Android Question net library, smtp section

[drgottjr]

there is an issue with sending mail from @yahoo.com accounts through non-yahoo servers. yahoo has a policy, which, if you happen to be an smtp server and you agree to cooperate with yahoo, you will reject the mail because it was not sent by a yahoo server. (if you don't follow the policy - and you don't have to - you deliver the mail as normal.) my app uses the net library to send mail through a third party server, and i happen to have an @yahoo.com account. any mail i send to servers cooperating with yahoo reject the mail. (gmail cooperates, so mail "from" yahoo to gmail, but not going through yahoo servers, will be bounced. the policy is "dmarc".)

setting aside some obvious workarounds for the moment, i wanted to know how the smtp dialog in the net library is coded. specifically, what is the string that follows the "ehlo" statement when the library begins the session with the stmp server? i see the "ehlo" in the $stmp class file and a reference to a string that follows. what does that string point to? "ehlo" is supposed to be followed by a fully qualified domain name, but technically you can get away with putting nothing. the receiving server does a reverse lookup even if you say nothing. anyway, my third party smtp server says my "software" is showing @yahoo.com account as "return-path" even though i have manually set every header i can (either through net or smtpextras) to something else. my "software", of course is the b4a app and the net library (enhanced with smtpextras). i say i've done what i can with the app. what i can't speak to is what the net library does. so, i'm asking: what does it do when it sends "ehlo"? and what does it do when it sends "mail from:" (and i'm making the distinction between "From:" and "mail from:". "From:" is not part of the smtp dialog between net and the server.) it must be getting the strings that follow those commands from somewhere. my guess is it's taking something from smtp.initilize(). in my case, it's the only place where a reference to @yahoo.com is made. all the other headers which i set or add do not refer to yahoo.

thanks for any help.
-go

 

[Erel]

Note that it is quite difficult to read your past because of the lack of spaces.

The exact messages sent depend on several factors. You can use B4J together with Wireshark (or a similar tool) to see the messages.

 

[drgottjr]

Note that it is quite difficult to read your past because of the lack of spaces.

The exact messages sent depend on several factors. You can use B4J together with Wireshark (or a similar tool) to see the messages.

thanks. sorry for any difficulty. i can see what's going over the wire. i was hoping to be able to learn how a variable was being assigned and where (i imagined that when a wrapper is written, the author might be looking at java code to see where hooks could go.)

but let me ask my question a different way: might i be able to use the reflector somehow? in the net library (smtp section), the default seems to be to use the username (from smtp.initialize()) in the "MAIL FROM:" statement. it also uses the username in the "From:" mail header. these, of course, are not the same thing. i can change the "From:" header (smtp.sender = ), but i can't change the "MAIL FROM:" smtp header. there seems to be no hook for that. is something exposed from the net library that i might be able to use in b4a?

writing a simple smtp dialog is easy enough; i would prefer to use a robust library. i just need that one tweak because of yahoo's (understandable) concern with spam. thanks again.

-go

 

[Erel]

Reflection will not help in this case. NetExtras library allows you to add headers. Try to add the MAIL FROM header with that library.

 

[nwhitfield]

The Return Path in mail is usually set by the receiving SMTP server from the envelope info given in the SMTP 'MAIL FROM' dialogue. But from reading up on the changes, it looks like there's not really anything you can do - Yahoo isn't going to let users submit their own mail servers as legitimate for yahoo.com addresses, because that defeats the point of what they're trying to to (stop the avalanche of spam with yahoo.com addresses)

A primer on DMARC that specifically explains what Yahoo is up to is at https://wordtothewise.com/2014/04/brief-dmarc-primer/

 

[drgottjr]

The Return Path in mail is usually set by the receiving SMTP server from the envelope info given in the SMTP 'MAIL FROM' dialogue. But from reading up on the changes, it looks like there's not really anything you can do - Yahoo isn't going to let users submit their own mail servers as legitimate for yahoo.com addresses, because that defeats the point of what they're trying to to (stop the avalanche of spam with yahoo.com addresses)

A primer on DMARC that specifically explains what Yahoo is up to is at https://wordtothewise.com/2014/04/brief-dmarc-primer/

to respond to erel first: thanks. i'm already using netextras. the problem is - as is alluded to by nwhitfield, next, in his comments - "mail from:" is not a mail header. it's part of the smtp dialog (and ends up setting the return-path.) the net library only allows me to change or add headers.

to nwhitfield - thanks for your comments. it's not that i want to defeat dmarc. i'm specifically trying to make my dialog with my smtp server NOT say "mail from: my@yahoo.com". i have a right to use my yahoo account to receive mail. i just don't have a right to say that my smtp server is yahoo. which is what's happening, and that's what i'm trying to change.

without dragging this topic out (although others might find themselves in the same boat and wondering why some mail may not be delivered when using the net library), suffice it to say that the net(extras) library issues its "MAIL FROM:" smtp command using the userid id given when the smtp object is initialized. this is quite logical and normal. but it can cause a problem when yahoo is involved. although the library provides hooks to change or add mail headers, it does not allow changes to the smtp dialog. the library is not at fault. part of the problem involves the use of third party smtp servers (perhaps only my third party smtp server). an option might be a different server. another might be not to use yahoo accounts. but how useful is an app that says it's not good for yahoo account holders? i have to make it work, so i started with the tools at hand, namely the net library.

i appreciate the interest.
-go

 

[nwhitfield]

I suppose the only way to avoid this would be to take advantage of the possibilty in SMTP to set no return address, ie to persuade the library to start with

MAIL FROM: <>

But can you initialize it with an empty user id that will generate that?

If so, that sets a null return path, which is what MTAs do to avoid cascades of bounce messages. If you do want to provide some valid error reporting to the users of your apps, you can then add an Errors-To: header, which will in some circumstances do the trick, though there are lots of decidedly wonky email systems out there that do insane things like always reply to the Envelope From (SMTP) address, rather than anything in the headers, and other such lunacies.

If you can't persuade the SMTP object to behave as you wish, provided you trap the most common response codes and you're not doing anything really complicated, you could probably write a fairly simple SMTP submission tool using a socket and AsyncStreamsText - it's not the most complex of protocols, after all.

 

[drgottjr]

I suppose the only way to avoid this would be to take advantage of the possibilty in SMTP to set no return address, ie to persuade the library to start with

MAIL FROM: <>

But can you initialize it with an empty user id that will generate that?

If so, that sets a null return path, which is what MTAs do to avoid cascades of bounce messages. If you do want to provide some valid error reporting to the users of your apps, you can then add an Errors-To: header, which will in some circumstances do the trick, though there are lots of decidedly wonky email systems out there that do insane things like always reply to the Envelope From (SMTP) address, rather than anything in the headers, and other such lunacies.

If you can't persuade the SMTP object to behave as you wish, provided you trap the most common response codes and you're not doing anything really complicated, you could probably write a fairly simple SMTP submission tool using a socket and AsyncStreamsText - it's not the most complex of protocols, after all.

that's exactly what i wanted to do, but the smtp object in the net library is initialized with my userid, and the library uses this in its smtp dialog. since my userid IS my yahoo email address, it's a self-fulfilling bounce (at least in cases where the receiving smtp server adheres to yahoo's dmarc policy.) unfortunately, i can't have things both ways; either my login is successful but "mail from: @yahoo" fails, or "mail from: <>" works, but my login fails ("<>" is not my login.) i need to change a variable after the object has been initialized, which requires access to the library at some apparently unexposed level. my question was to find out how much exposure there was.

writing my own simple dialog is trivial, and the original version of the app used it and my own smtp server as mta, but who wants his own server being pounded? for a few dollars, i can use a high-powered smtp server and have replies sent to yahoo. they're both set up to handle traffic, and since a lot of people have yahoo accounts, it followed that, at the very least, i would need to test with one. anyway, thanks for opportunity to vent.

-go