discourse/app/services/email_settings_validator.rb

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

178 lines
5.9 KiB
Ruby
Raw Permalink Normal View History

# frozen_string_literal: true
require "net/imap"
require "net/smtp"
require "net/pop"
# Usage:
#
# begin
# EmailSettingsValidator.validate_imap(host: "imap.test.com", port: 999, username: "test@test.com", password: "password")
#
# # or for specific host preset
# EmailSettingsValidator.validate_imap(**{ username: "test@gmail.com", password: "test" }.merge(Email.gmail_imap_settings))
#
FEATURE: Improve group email settings UI (#13083) This overhauls the user interface for the group email settings management, aiming to make it a lot easier to test the settings entered and confirm they are correct before proceeding. We do this by forcing the user to test the settings before they can be saved to the database. It also includes some quality of life improvements around setting up IMAP and SMTP for our first supported provider, GMail. This PR does not remove the old group email config, that will come in a subsequent PR. This is related to https://meta.discourse.org/t/imap-support-for-group-inboxes/160588 so read that if you would like more backstory. ### UI Both site settings of `enable_imap` and `enable_smtp` must be true to test this. You must enable SMTP first to enable IMAP. You can prefill the SMTP settings with GMail configuration. To proceed with saving these settings you must test them, which is handled by the EmailSettingsValidator. If there is an issue with the configuration or credentials a meaningful error message should be shown. IMAP settings must also be validated when IMAP is enabled, before saving. When saving IMAP, we fetch the mailboxes for that account and populate them. This mailbox must be selected and saved for IMAP to work (the feature acts as though it is disabled until the mailbox is selected and saved): ### Database & Backend This adds several columns to the Groups table. The purpose of this change is to make it much more explicit that SMTP/IMAP is enabled for a group, rather than relying on settings not being null. Also included is an UPDATE query to backfill these columns. These columns are automatically filled when updating the group. For GMail, we now filter the mailboxes returned. This is so users cannot use a mailbox like Sent or Trash for syncing, which would generally be disastrous. There is a new group endpoint for testing email settings. This may be useful in the future for other places in our UI, at which point it can be extracted to a more generic endpoint or module to be included.
2021-05-27 18:28:18 -05:00
# rescue *EmailSettingsExceptionHandler::EXPECTED_EXCEPTIONS => err
# EmailSettingsExceptionHandler.friendly_exception_message(err, host)
# end
class EmailSettingsValidator
FEATURE: Improve group email settings UI (#13083) This overhauls the user interface for the group email settings management, aiming to make it a lot easier to test the settings entered and confirm they are correct before proceeding. We do this by forcing the user to test the settings before they can be saved to the database. It also includes some quality of life improvements around setting up IMAP and SMTP for our first supported provider, GMail. This PR does not remove the old group email config, that will come in a subsequent PR. This is related to https://meta.discourse.org/t/imap-support-for-group-inboxes/160588 so read that if you would like more backstory. ### UI Both site settings of `enable_imap` and `enable_smtp` must be true to test this. You must enable SMTP first to enable IMAP. You can prefill the SMTP settings with GMail configuration. To proceed with saving these settings you must test them, which is handled by the EmailSettingsValidator. If there is an issue with the configuration or credentials a meaningful error message should be shown. IMAP settings must also be validated when IMAP is enabled, before saving. When saving IMAP, we fetch the mailboxes for that account and populate them. This mailbox must be selected and saved for IMAP to work (the feature acts as though it is disabled until the mailbox is selected and saved): ### Database & Backend This adds several columns to the Groups table. The purpose of this change is to make it much more explicit that SMTP/IMAP is enabled for a group, rather than relying on settings not being null. Also included is an UPDATE query to backfill these columns. These columns are automatically filled when updating the group. For GMail, we now filter the mailboxes returned. This is so users cannot use a mailbox like Sent or Trash for syncing, which would generally be disastrous. There is a new group endpoint for testing email settings. This may be useful in the future for other places in our UI, at which point it can be extracted to a more generic endpoint or module to be included.
2021-05-27 18:28:18 -05:00
def self.validate_as_user(user, protocol, **kwargs)
DistributedMutex.synchronize("validate_#{protocol}_#{user.id}", validity: 10) do
self.public_send("validate_#{protocol}", **kwargs)
end
end
##
# Attempts to authenticate and disconnect a POP3 session and if that raises
# an error then it is assumed the credentials or some other settings are wrong.
#
# @param debug [Boolean] - When set to true, any errors will be logged at a warning
# level before being re-raised.
def self.validate_pop3(
host:,
port:,
username:,
password:,
ssl: SiteSetting.pop3_polling_ssl,
openssl_verify: SiteSetting.pop3_polling_openssl_verify,
FEATURE: Improve group email settings UI (#13083) This overhauls the user interface for the group email settings management, aiming to make it a lot easier to test the settings entered and confirm they are correct before proceeding. We do this by forcing the user to test the settings before they can be saved to the database. It also includes some quality of life improvements around setting up IMAP and SMTP for our first supported provider, GMail. This PR does not remove the old group email config, that will come in a subsequent PR. This is related to https://meta.discourse.org/t/imap-support-for-group-inboxes/160588 so read that if you would like more backstory. ### UI Both site settings of `enable_imap` and `enable_smtp` must be true to test this. You must enable SMTP first to enable IMAP. You can prefill the SMTP settings with GMail configuration. To proceed with saving these settings you must test them, which is handled by the EmailSettingsValidator. If there is an issue with the configuration or credentials a meaningful error message should be shown. IMAP settings must also be validated when IMAP is enabled, before saving. When saving IMAP, we fetch the mailboxes for that account and populate them. This mailbox must be selected and saved for IMAP to work (the feature acts as though it is disabled until the mailbox is selected and saved): ### Database & Backend This adds several columns to the Groups table. The purpose of this change is to make it much more explicit that SMTP/IMAP is enabled for a group, rather than relying on settings not being null. Also included is an UPDATE query to backfill these columns. These columns are automatically filled when updating the group. For GMail, we now filter the mailboxes returned. This is so users cannot use a mailbox like Sent or Trash for syncing, which would generally be disastrous. There is a new group endpoint for testing email settings. This may be useful in the future for other places in our UI, at which point it can be extracted to a more generic endpoint or module to be included.
2021-05-27 18:28:18 -05:00
debug: Rails.env.development?
)
begin
pop3 = Net::POP3.new(host, port)
# Note that we do not allow which verification mode to be specified
# like we do for SMTP, we just pick TLS1_2 if the SSL and openSSL verify
# options have been enabled.
if ssl
if openssl_verify
pop3.enable_ssl(max_version: OpenSSL::SSL::TLS1_2_VERSION)
else
pop3.enable_ssl(OpenSSL::SSL::VERIFY_NONE)
end
end
# This disconnects itself, unlike SMTP and IMAP.
pop3.auth_only(username, password)
rescue => err
log_and_raise(err, debug)
end
end
##
# Attempts to start an SMTP session and if that raises an error then it is
# assumed the credentials or other settings are wrong.
#
# @param domain [String] - Used for HELO, should be the FQDN of the server sending the mail
# localhost can be used in development mode.
# See https://datatracker.ietf.org/doc/html/rfc788#section-4
# @param debug [Boolean] - When set to true, any errors will be logged at a warning
# level before being re-raised.
def self.validate_smtp(
host:,
port:,
username:,
password:,
FEATURE: Improve group email settings UI (#13083) This overhauls the user interface for the group email settings management, aiming to make it a lot easier to test the settings entered and confirm they are correct before proceeding. We do this by forcing the user to test the settings before they can be saved to the database. It also includes some quality of life improvements around setting up IMAP and SMTP for our first supported provider, GMail. This PR does not remove the old group email config, that will come in a subsequent PR. This is related to https://meta.discourse.org/t/imap-support-for-group-inboxes/160588 so read that if you would like more backstory. ### UI Both site settings of `enable_imap` and `enable_smtp` must be true to test this. You must enable SMTP first to enable IMAP. You can prefill the SMTP settings with GMail configuration. To proceed with saving these settings you must test them, which is handled by the EmailSettingsValidator. If there is an issue with the configuration or credentials a meaningful error message should be shown. IMAP settings must also be validated when IMAP is enabled, before saving. When saving IMAP, we fetch the mailboxes for that account and populate them. This mailbox must be selected and saved for IMAP to work (the feature acts as though it is disabled until the mailbox is selected and saved): ### Database & Backend This adds several columns to the Groups table. The purpose of this change is to make it much more explicit that SMTP/IMAP is enabled for a group, rather than relying on settings not being null. Also included is an UPDATE query to backfill these columns. These columns are automatically filled when updating the group. For GMail, we now filter the mailboxes returned. This is so users cannot use a mailbox like Sent or Trash for syncing, which would generally be disastrous. There is a new group endpoint for testing email settings. This may be useful in the future for other places in our UI, at which point it can be extracted to a more generic endpoint or module to be included.
2021-05-27 18:28:18 -05:00
domain: nil,
authentication: nil,
enable_starttls_auto: GlobalSetting.smtp_enable_start_tls,
enable_tls: GlobalSetting.smtp_force_tls,
openssl_verify_mode: GlobalSetting.smtp_openssl_verify_mode,
FEATURE: Improve group email settings UI (#13083) This overhauls the user interface for the group email settings management, aiming to make it a lot easier to test the settings entered and confirm they are correct before proceeding. We do this by forcing the user to test the settings before they can be saved to the database. It also includes some quality of life improvements around setting up IMAP and SMTP for our first supported provider, GMail. This PR does not remove the old group email config, that will come in a subsequent PR. This is related to https://meta.discourse.org/t/imap-support-for-group-inboxes/160588 so read that if you would like more backstory. ### UI Both site settings of `enable_imap` and `enable_smtp` must be true to test this. You must enable SMTP first to enable IMAP. You can prefill the SMTP settings with GMail configuration. To proceed with saving these settings you must test them, which is handled by the EmailSettingsValidator. If there is an issue with the configuration or credentials a meaningful error message should be shown. IMAP settings must also be validated when IMAP is enabled, before saving. When saving IMAP, we fetch the mailboxes for that account and populate them. This mailbox must be selected and saved for IMAP to work (the feature acts as though it is disabled until the mailbox is selected and saved): ### Database & Backend This adds several columns to the Groups table. The purpose of this change is to make it much more explicit that SMTP/IMAP is enabled for a group, rather than relying on settings not being null. Also included is an UPDATE query to backfill these columns. These columns are automatically filled when updating the group. For GMail, we now filter the mailboxes returned. This is so users cannot use a mailbox like Sent or Trash for syncing, which would generally be disastrous. There is a new group endpoint for testing email settings. This may be useful in the future for other places in our UI, at which point it can be extracted to a more generic endpoint or module to be included.
2021-05-27 18:28:18 -05:00
debug: Rails.env.development?
)
begin
if enable_tls && enable_starttls_auto
raise ArgumentError, "TLS and STARTTLS are mutually exclusive"
end
if username || password
authentication = SmtpProviderOverrides.authentication_override(host) if authentication.nil?
authentication = authentication.to_sym
if !%i[plain login cram_md5].include?(authentication)
raise ArgumentError, "Invalid authentication method. Must be plain, login, or cram_md5."
end
else
authentication = nil
end
FEATURE: Improve group email settings UI (#13083) This overhauls the user interface for the group email settings management, aiming to make it a lot easier to test the settings entered and confirm they are correct before proceeding. We do this by forcing the user to test the settings before they can be saved to the database. It also includes some quality of life improvements around setting up IMAP and SMTP for our first supported provider, GMail. This PR does not remove the old group email config, that will come in a subsequent PR. This is related to https://meta.discourse.org/t/imap-support-for-group-inboxes/160588 so read that if you would like more backstory. ### UI Both site settings of `enable_imap` and `enable_smtp` must be true to test this. You must enable SMTP first to enable IMAP. You can prefill the SMTP settings with GMail configuration. To proceed with saving these settings you must test them, which is handled by the EmailSettingsValidator. If there is an issue with the configuration or credentials a meaningful error message should be shown. IMAP settings must also be validated when IMAP is enabled, before saving. When saving IMAP, we fetch the mailboxes for that account and populate them. This mailbox must be selected and saved for IMAP to work (the feature acts as though it is disabled until the mailbox is selected and saved): ### Database & Backend This adds several columns to the Groups table. The purpose of this change is to make it much more explicit that SMTP/IMAP is enabled for a group, rather than relying on settings not being null. Also included is an UPDATE query to backfill these columns. These columns are automatically filled when updating the group. For GMail, we now filter the mailboxes returned. This is so users cannot use a mailbox like Sent or Trash for syncing, which would generally be disastrous. There is a new group endpoint for testing email settings. This may be useful in the future for other places in our UI, at which point it can be extracted to a more generic endpoint or module to be included.
2021-05-27 18:28:18 -05:00
if domain.blank?
if Rails.env.development?
domain = "localhost"
else
# Because we are using the SMTP settings here to send emails,
# the domain should just be the TLD of the host.
domain = MiniSuffix.domain(host)
end
end
smtp = Net::SMTP.new(host, port)
# These SSL options are cribbed from the Mail gem, which is used internally
# by ActionMailer. Unfortunately the mail gem hides this setup in private
# methods, e.g. https://github.com/mikel/mail/blob/master/lib/mail/network/delivery_methods/smtp.rb#L112-L147
#
# Relying on the GlobalSetting options is a good idea here.
#
# For specific use cases, options should be passed in from higher up. For example
# Gmail needs either port 465 and tls enabled, or port 587 and starttls_auto.
if openssl_verify_mode.kind_of?(String)
openssl_verify_mode = OpenSSL::SSL.const_get("VERIFY_#{openssl_verify_mode.upcase}")
end
ssl_context = Net::SMTP.default_ssl_context
ssl_context.verify_mode = openssl_verify_mode if openssl_verify_mode
smtp.enable_starttls_auto(ssl_context) if enable_starttls_auto
smtp.enable_tls(ssl_context) if enable_tls
FEATURE: Improve group email settings UI (#13083) This overhauls the user interface for the group email settings management, aiming to make it a lot easier to test the settings entered and confirm they are correct before proceeding. We do this by forcing the user to test the settings before they can be saved to the database. It also includes some quality of life improvements around setting up IMAP and SMTP for our first supported provider, GMail. This PR does not remove the old group email config, that will come in a subsequent PR. This is related to https://meta.discourse.org/t/imap-support-for-group-inboxes/160588 so read that if you would like more backstory. ### UI Both site settings of `enable_imap` and `enable_smtp` must be true to test this. You must enable SMTP first to enable IMAP. You can prefill the SMTP settings with GMail configuration. To proceed with saving these settings you must test them, which is handled by the EmailSettingsValidator. If there is an issue with the configuration or credentials a meaningful error message should be shown. IMAP settings must also be validated when IMAP is enabled, before saving. When saving IMAP, we fetch the mailboxes for that account and populate them. This mailbox must be selected and saved for IMAP to work (the feature acts as though it is disabled until the mailbox is selected and saved): ### Database & Backend This adds several columns to the Groups table. The purpose of this change is to make it much more explicit that SMTP/IMAP is enabled for a group, rather than relying on settings not being null. Also included is an UPDATE query to backfill these columns. These columns are automatically filled when updating the group. For GMail, we now filter the mailboxes returned. This is so users cannot use a mailbox like Sent or Trash for syncing, which would generally be disastrous. There is a new group endpoint for testing email settings. This may be useful in the future for other places in our UI, at which point it can be extracted to a more generic endpoint or module to be included.
2021-05-27 18:28:18 -05:00
smtp.open_timeout = 5
# Some SMTP servers have a higher delay to respond with errors
# as a tarpit measure that slows down clients who are sending "bad" commands.
# 10s is the minimum, we might need to increase this in the future.
smtp.read_timeout = 10
FEATURE: Improve group email settings UI (#13083) This overhauls the user interface for the group email settings management, aiming to make it a lot easier to test the settings entered and confirm they are correct before proceeding. We do this by forcing the user to test the settings before they can be saved to the database. It also includes some quality of life improvements around setting up IMAP and SMTP for our first supported provider, GMail. This PR does not remove the old group email config, that will come in a subsequent PR. This is related to https://meta.discourse.org/t/imap-support-for-group-inboxes/160588 so read that if you would like more backstory. ### UI Both site settings of `enable_imap` and `enable_smtp` must be true to test this. You must enable SMTP first to enable IMAP. You can prefill the SMTP settings with GMail configuration. To proceed with saving these settings you must test them, which is handled by the EmailSettingsValidator. If there is an issue with the configuration or credentials a meaningful error message should be shown. IMAP settings must also be validated when IMAP is enabled, before saving. When saving IMAP, we fetch the mailboxes for that account and populate them. This mailbox must be selected and saved for IMAP to work (the feature acts as though it is disabled until the mailbox is selected and saved): ### Database & Backend This adds several columns to the Groups table. The purpose of this change is to make it much more explicit that SMTP/IMAP is enabled for a group, rather than relying on settings not being null. Also included is an UPDATE query to backfill these columns. These columns are automatically filled when updating the group. For GMail, we now filter the mailboxes returned. This is so users cannot use a mailbox like Sent or Trash for syncing, which would generally be disastrous. There is a new group endpoint for testing email settings. This may be useful in the future for other places in our UI, at which point it can be extracted to a more generic endpoint or module to be included.
2021-05-27 18:28:18 -05:00
smtp.start(domain, username, password, authentication)
smtp.finish
rescue => err
log_and_raise(err, debug)
end
end
##
# Attempts to login, logout, and disconnect an IMAP session and if that raises
# an error then it is assumed the credentials or some other settings are wrong.
#
# @param debug [Boolean] - When set to true, any errors will be logged at a warning
# level before being re-raised.
def self.validate_imap(
host:,
port:,
username:,
password:,
FEATURE: Improve group email settings UI (#13083) This overhauls the user interface for the group email settings management, aiming to make it a lot easier to test the settings entered and confirm they are correct before proceeding. We do this by forcing the user to test the settings before they can be saved to the database. It also includes some quality of life improvements around setting up IMAP and SMTP for our first supported provider, GMail. This PR does not remove the old group email config, that will come in a subsequent PR. This is related to https://meta.discourse.org/t/imap-support-for-group-inboxes/160588 so read that if you would like more backstory. ### UI Both site settings of `enable_imap` and `enable_smtp` must be true to test this. You must enable SMTP first to enable IMAP. You can prefill the SMTP settings with GMail configuration. To proceed with saving these settings you must test them, which is handled by the EmailSettingsValidator. If there is an issue with the configuration or credentials a meaningful error message should be shown. IMAP settings must also be validated when IMAP is enabled, before saving. When saving IMAP, we fetch the mailboxes for that account and populate them. This mailbox must be selected and saved for IMAP to work (the feature acts as though it is disabled until the mailbox is selected and saved): ### Database & Backend This adds several columns to the Groups table. The purpose of this change is to make it much more explicit that SMTP/IMAP is enabled for a group, rather than relying on settings not being null. Also included is an UPDATE query to backfill these columns. These columns are automatically filled when updating the group. For GMail, we now filter the mailboxes returned. This is so users cannot use a mailbox like Sent or Trash for syncing, which would generally be disastrous. There is a new group endpoint for testing email settings. This may be useful in the future for other places in our UI, at which point it can be extracted to a more generic endpoint or module to be included.
2021-05-27 18:28:18 -05:00
open_timeout: 5,
ssl: true,
debug: false
)
begin
imap = Net::IMAP.new(host, port: port, ssl: ssl, open_timeout: open_timeout)
imap.login(username, password)
begin
imap.logout
rescue StandardError
nil
end
imap.disconnect
rescue => err
log_and_raise(err, debug)
end
end
def self.log_and_raise(err, debug)
if debug
Rails.logger.warn(
"[EmailSettingsValidator] Error encountered when validating email settings: #{err.message} #{err.backtrace.join("\n")}",
)
end
raise err
end
end