Add an option to silently reject#50
Conversation
|
I hope I understand the situation correctly. So gmail is forwarding messages coming to your @gmail.com account to your new address hosted on mox. And mox is rejecting some of those messages as spam. I suppose this causes gmail to put the forwarding rule on pause? Is the forwarding rule configured as "send a copy to this address", or more a "don't store anything in gmail anymore, just forward to my new address"? (I don't know if gmail has the second option). If I understand it correctly, gmail doesn't spamfilter the messages it forwards, or it doesn't do a good job? (I would expect it would). I think the current approach in the PR would mostly work. But I think you would want to disable DMARC checks. Because gmail is sending messages to mox with a "mesage From" address for which it won't have an "aligned SPF pass" (either it uses the original "smtp mail from" that is aligned but not passes SPF, or it uses its own "smtp mail from" that passes SPF but isn't aligned). If there is also no (passing) DKIM header in the message, the DMARC check will fail. With the current PR changes, I think that would put such messages in the Rejects mailbox. I think you only want to do the soft rejecting based on sender/sender-domain reputation and content (bayesian). The ListAllowDomain option in the Ruleset is related, but disables all spam checking, it is intended for messages from mailing lists. The assumption is that mailing lists do good enough spam filtering. If we were to reject messages from a mailing list, our subscription would likely be cancelled. So it's quite similar to your situation, and you could probably use it, if only if gmail's spam filtering was good enough. I would think gmails were good enough, but perhaps those of other providers/setups would perhaps not be. There are a few complications. Although you can recognize email coming from gmail (with a "VerifiedDomain", probably at least matched on verified SPF, perhaps also verified DKIM (does gmail add a DKIM header when forwarding?)), I don't know if you can be sure that it is a forwarded message versus a message someone sent directly to your new address from their own gmail address. They both come from gmail's mail servers. Your original gmail address doesn't necessarily have to be in the message To/Cc/Bcc header. Perhaps it is good enough to match for that, though the Ruleset currently doesn't allow for that matching (we can add it). I think you would have to accept that there could be some legitimate forwarded messages that mox would silently put in the Rejects mailbox. Currently, this PR would add a knob "softreject" (the name doesn't feel quite right). Perhaps (I'm not sure), it could be better to name it after the scenario. The scenario being: For forwarded messages (matched on verified domain of forwarding server, matched original/forwarded addressee in message (in to/cc/bcc headers of original address), and matched forward-destination address), always accept messages from the forwarding server, but still apply some of mox's spam-checks and possibly deliver to the rejects mailbox instead of regular mailbox. Another thought: If the previous KeepRejects-change was meant to work together with this, perhaps it would make sense to put the soft-rejected messages in a different mailbox than Rejects and not use the KeepRejects config? We could add a field in the Ruleset for the mailbox for such forwarded rejects, e.g. "ForwardRejects". |
|
Your understanding is generally correct. To add more background, I've set up Gmail auto-forwarding with a filter that forwards all incoming mail then deletes it. This ignores all spam filtering. Gmail is perfectly capable of detecting spam but I don't want to have to log in to Gmail to check my spam inbox for false-positives, I'd prefer all spam be forwarded to mox and handled there, in one place. Right now though, mox (correctly) detects the spam but (incorrectly) rejects it, causing Gmail to attempt to send it again. I'd prefer to accept delivery in these cases so that Gmail and mox don't waste their energy. This PR is working though, this is how I've configured mox: Gmail adds both SPF and DKIM when forwarding, so The previous KeepRejects option is orthogonal to this, it's about spam in general, this is about a specific category. |
|
Thanks, clear, those delivered-to and return-path make sense, hadn't thought about those. Doesn't this still leave an issue with mox rejecting due to DMARC fails? Or is gmail rewriting the message From header to your own gmail address? |
|
Hmm, for a forward from Gmail, DMARC passes: The From header is retained as-is. But I have another email from a different address and authentication fails: I'll wait and see if I see rejections. |
|
If a message has a passing DKIM header in the message, the DMARC check will pass (assuming the "message From" address isn't rewritten). This is quite common nowadays, but not guaranteed. Plenty of mailing lists modify the message so that the DKIM signature becomes invalid. But I suspect you would update mailing list subscriptions to go directly to your new address. So that leaves SPF. The Authentication-Results show a pass, but it won't be an aligned pass. The second Authentication-Results has a dmarc=none. That probably means the "message From" domain doesn't have a dmarc (reject) policy, which makes sense for spammers. But valid domains may have dmarc reject policies. One would hope they would also add DKIM signatures to their messages, but there is no guarantee. Anyway, it seems this option make sense to have, will get back to it soon. |
|
I gave this some more thought. I think we have to be more careful about how forwarded messages influence mox's Ideas: Or more high-level (my preference): In case of forwarding, we would also skip the DMARC evaluation. In the future, mox should also implement ARC With a ForwardedFrom config option, we can explain more fully in the sconf-doc Btw, have you tried matching the ruleset with SMTPMailFromRegexp? If gmail sets Some more notes:
I can help with/implement these changes, it's becoming more than you probably |
|
I've been thinking about this too but my conclusion is that it might make milter/Sieve support more important. I imagine there are going to be other people in the future who have cases like mine and to add explicit handling for those cases to Mox seems unmaintainable to me. Better would be to make Mox configurable enough that everyone can set it up for their particular use-case and adding milters and/or Sieve seems like the better solution to me. Speaking of Sieve, it looks like maddy, a similar project, has Sieve support in pure Go through this library. |
|
I think mox will need to be more configurable in the future. But I also don't want every user to have to figure out the right combination of knobs to handle common cases. I think accepting forwarded emails is quite common. And interactions between email protocols/mechanisms is a bit more tricky than users would perhaps think. So I prefer to start with making it easy to configure this common case, and we'll get to the fully-configurable later. |
… mailbox soon, we can have multiple rejects mailboxes. and checking against the configured rejects mailbox name wasn't foolproof to begin with, because it may have changed between delivery to the rejects mailbox and the message being moved. after upgrading, messages currently in rejects mailboxes don't have IsReject set, so they don't get the special rejecs treatment when being moved. they are removed from the rejects mailbox after some time though, and newly added rejects will be treated correctly. so this means some existing messages wrongly delivered to the rejects mailbox, and moved out, aren't used (for a positive signal) for future deliveries. saves a bit of complexity in the implementation. i think the tradeoff is worth it. related to discussion in issue #50
…red mailbox this is based on @bobobo1618's PR #50. bobobo1618 had the right idea, i tried including an "is forwarded email" configuration option but that indeed became too tightly coupled. the "is forwarded" option is still planned, but it is separate from the "accept rejects to mailbox" config option, because one could still want to push back on forwarded spam messages. we do an actual accept, delivering to a configured mailbox, instead of storing to the rejects mailbox where messages can automatically be removed from. one of the goals of mox is not pretend to accept email while actually junking it. users can still configure delivery to a junk folder (as was already possible), but aren't deleted automatically. there is still an X-Mox-Reason header in the message, and a log line about accepting the reject, but otherwise it is registered and treated as an (smtp) accept. the ruleset mailbox is still required to keep that explicit. users can specify Inbox again. hope this is good enough for PR #50, otherwise we'll change it.
…odifying how junk analysis is done part of PR #50 by bobobo1618
|
hi @bobobo1618, i went back to use pretty much your approach. you were right that these cases were better handled decoupled. so there is now the "AcceptRejectsToMailbox" option, that is quite close to this PR, but it is more a real accept (see commit message for reasoning). the second change is adding the IsForward field. i tested this with a gmail account, set up forwarding like you described. the SMTP MAIL FROM is of the form myuser+.*@gmail.com, so i think it's best to set up a ruleset matching for SMTPMailFromRegexp for that. interested in hearing if this works for you, otherwise we can make changes. |
|
Sounds good to me! I'll close this then. |
Spam forwarded from my Gmail account is redelivered because mox rejects it. I want mox to process it as if it's spam but not tell Gmail, so it doesn't try to redeliver it.