Roadmap

From Proxmox Mail Gateway
Revision as of 08:38, 15 July 2021 by Martin (talk | contribs) (Proxmox Mail Gateway 7.0 release)
Jump to navigation Jump to search

Roadmap

Release History

See also Announcement forum

Proxmox Mail Gateway 7.0

Released 15. July 2021

  • Based on Debian Bullseye (11)
  • SpamAssassin 3.4.6 (with updated rule-set)
  • Kernel 5.11
  • PostgreSQL 13

Changelog Overview

  • Enhancements in the web interface (GUI)
    • Make dashboard status panel more detailed, showing, among other things, uptime, kernel version, CPU info and a high level repository status overview.
    • New APT repository management panel in the Administration tab shows an in-depth status and a list of all configured repositories.
      Basic repository management, for example, activating or deactivating a repository, is also supported.
    • Updated ExtJS JavaScript framework to latest GPL release 7.0
    • Added advanced task-log filtering
    • Improved translations, including:
      • Arabic
      • French
      • German
      • Japanese
      • Polish
      • Turkish
  • ACME/Let's Encrypt
    • Support the use of wildcard domains with the DNS plugins
    • API: nodeconfig: validate ACME config before writing
  • API
    • pmgproxy: allow setting LISTEN_IP parameter
    • The "Authentication mode" setting of LDAP for the quarantine interface no longer contains the ticket-link in the report-mails - thus, quarantine users need to provide their LDAP credentials to access the quarantine.
  • Installer:
    • Rework the installer environment to use switch_root instead of chroot, when transitioning from initrd to the actual installer.
      This improves module and firmware loading, and slightly reduces memory usage during installation.
    • Automatically detect HiDPI screens, and increase console font and GUI scaling accordingly. This improves UX for workstations with Proxmox VE (for example, for passthrough).
    • Improve ISO detection:
      • Support ISOs backed by devices using USB Attached SCSI (UAS), which modern USB3 flash drives often do.
      • Linearly increase the delay of subsequent scans for a device with an ISO image, bringing the total check time from 20s to 45s. This allows for the detection of very slow devices, while continuing faster in general.
    • Use zstd compression for the initrd image and the squashfs images.
    • Update to busybox 1.33.1 as the core-utils provider.
  • libarchive-perl
    • The perl-bindings to libarchive have been updated to match libarchive version 3.4.3 (shipped in Debian Bullseye) - the library interface was kept backwards-compatible
  • libxdgmime-perl
    • The perl-bindings to xdgmime have been updated to match current upstream - the library interface was kept backwards-compatible

Breaking Changes

  • New default bind address for pmgproxy, unifying the default behavior with Proxmox VE and Proxmox Backup Server
    • In making the LISTEN_IP configurable, the daemon now binds to both wildcard addresses (IPv4 0.0.0.0:8006 and IPv6 [::]:8006) by default.
    Should you wish to prevent it from listening on IPv6, simply configure the IPv4 wildcard as LISTEN_IP in /etc/default/pmgproxy:
    LISTEN_IP="0.0.0.0"
    • Additionally, the logged IP address format changed for IPv4 in pmgproxy's access log (/var/log/pmgproxy/pmgproxy.log). They are now logged as IPv4-mapped IPv6 addresses. Instead of:
    192.0.2.1- root@pam [01/06/2021:01:19:03 +0200] "GET /api2/json/config/ruledb/digest HTTP/1.1" 200 51
    the line now looks like:
    ::ffff:192.0.2.1- root@pam [01/06/2021:01:19:03 +0200] "GET /api2/json/config/ruledb/digest HTTP/1.1" 200 51
    If you want to restore the old logging format, also set LISTEN_IP="0.0.0.0"
  • ClamAV has deprecated the SafeBrowsing feature:
    • These options have been removed from the shipped freshclam.conf.in template.
    • The safebrowsing config key in /etc/pmg/pmg.conf is currently ignored and will be dropped at some point in the future.
  • Changes to the database layout:
    • The host column of the cgreylist table, which has not been used since Proxmox Mailgateway 6.2, has been dropped from the schema and will be dropped from existing databases during the upgrade.
  • API deprecations, moves and removals
    • The upgrade parameter of the /nodes/{node}/termproxy API method has been replaced by providing upgrade as cmd parameter.
    • The domain parameter of the /config/tlspolicy API method has been replaced by the destination parameter.
    • The /quarantine/whitelist/{address} and /quarantine/blacklist/{address} API methods, that take the address as part of the path, have been deprecated in favor of explicitly providing the parameter in the request to /quarantine/whitelist and /quarantine/blacklist respectively.
    • The API methods for detailed statistics per e-mail address, which take the address as part of the path (/statistics/contact/{contact}, /statistics/sender/{sender} and /statistics/receiver/{receiver} have been deprecated in favor of /statistics/detail, which takes the address as an explicit parameter.

Known Issues

  • Network: Due to the updated systemd version, and for most upgrades, the newer kernel version (5.4 to 5.11), some network interfaces might change upon reboot:
    • Some may change their name. For example, due to newly supported functions, a change from enp33s0f0 to enp33s0f0np0 could occur.
      We observed such changes with high-speed Mellanox models.
    • Bridge MAC address selection has changed in Debian Bullseye - it is now generated based on the interface name and the machine-id (5) of the system.
      Note that by default, Proxmox Mail Gateway does not use a Linux Bridge for networking, so most setups are unaffected.
  • Machine-id: Systems installed using the Proxmox Mail Gateway 5.0 to 5.4 ISO may have a non-unique machine-id. These systems will have their machine-id re-generated automatically on upgrade, to avoid a potentially duplicated bridge MAC and other issues.
If you do the upgrade remotely, make sure you have a backup method of connecting to the host (for example, IPMI/iKVM, tiny-pilot, another network accessible by a cluster node, or physical access), in case the network used for SSH access becomes unreachable, due to the network failing to come up after a reboot.

Proxmox Mail Gateway 6.4

Released 30. March 2021

  • Based on Debian Buster (10.9)
  • SpamAssassin 3.4.5 (with update ruleset)
  • Kernel 5.4.106
  • ACME integration
    • Proxmox Mail Gateway now offers full integration of the ACME protocol via the GUI, enabling administrators to create valid and trusted certificates for their domains with the Let's Encrypt certificate authority, in the same way as with Proxmox VE.
    • Full support for the http-01 and dns-01 challenges, with all plugins from acme.sh.
    • Easily configurable from the GUI.
  • General Certificate Management via the GUI
    • It is now possible to upload custom certificates from the web interface, or set up a cluster-wide ACME account to automatically get and renew certificates from an ACME provider.
  • Support for external SpamAssassin update channels (regular automated updates).
    • By providing a short configuration file containing a SpamAssassin rule channel's URL and GPG key, Proxmox Mail Gateway will now fetch verified updates from that channel, along with the updates from updates.spamassassin.org.
    • The KAM ruleset channel is now available, and a suitable configuration file is shipped with proxmox-spamassassin.
  • Improved Quarantine Management
    • The admin view of the Spam Quarantine can now display quarantined mail of all users at once.
    • All Quarantine views (admin and user) allow you to filter for subject or sender.
    • The spam quarantine can now process huge amounts of mails at once (> 3200).
  • TLS-logging improvements to the Tracking Center
    • The Tracking Center now shows when an outbound connection is established over TLS.
  • Enhancements to the Integration of Proxmox Backup Server
    • It is now possible to get notified about the result of a scheduled backup to a configured Proxmox Backup Server Remote.
    • Inclusion of the (potentially large) statistics database is now configurable per Remote.
  • Notable Bugfixes:
    • Support for '/' in the local part of an e-mail address (quarantine and statistics view).

Proxmox Mail Gateway 6.3

Released 19. November 2020

  • Based on Debian Buster (10.6)
  • Updated SpamAssassin rules
  • Kernel 5.4.73
  • Proxmox Backup Server Integration
    Proxmox Mail Gateway is fully supported by the new Proxmox Backup Server 1.0, released on November 11, 2020:
    • Backing up to multiple remote backup servers: You can define multiple remote instances of Proxmox Backup Server to store backups on. In case of a large-scale disaster, they can be quickly restored.
    • Scheduled Backups: You can schedule regular backups via the GUI, which will then be automatically triggered by a systemd-timer unit. This removes the need for manual backup creation and individual, scripted solutions.
  • Quarantine Link via login-page
    Users can request mails containing a link to their quarantineview, if enabled by the Admin. This enables users to edit their individual blocklists, even if no mails are in their quarantine. Until now this was only possible for sites using LDAP.
  • Improvements to the Tracking Center
    To further improve user experience in the tracking center, pmg-log-tracker handles certain cases better:
    • The case sensitivity has been removed from the search box.
    • In case the pmg-smtp-filter fails to process emails due to misconfiguration, they are now marked as rejected.
  • Notable Bugfixes:
    • DKIM signing now uses the longest matching domain for the 'd=' tag.
    • Mails held in the Attachment Quarantine are assigned a new Message-ID - fixing interoperability with certain downstream servers (for example, MS Exchange), which silently discard messages with duplicate Message-IDs.

Proxmox Mail Gateway 6.2

Released 28. April 2020

  • Based on Debian Buster (10.3)
    • Proxmox Mail Gateway is based on the latest stable release of Debian 10.3 (Buster).
  • SpamAssassin 3.4.4
    • Proxmox ships the latest upstream release of Apache SpamAssassin with a updated and enhance ruleset (KAM rules added)
  • Kernel 5.4
    • Proxmox Mail Gateway shares the kernel with Proxmox VE and is based on the 5.4 series from Ubuntu 20.04
  • pmg-log-tracker in Rust
    • pmg-log-tracker has been extended and reimplemented in the Rust programming language. pmg-log-tracker is the binary at the core of the Message Tracking Center, providing live searchable and grouped logs in the GUI.
    • The new pmg-log-tracker has support for parsing and grouping logs in before-queue filtering mode.
    • The refresh of the code base of pmg-log-tracker provides an optimized performance and more stability.
  • Support for before-queue filtering in the GUI
    • With the added support for displaying before-queue filtering logs in the GUI and fixing some minor glitches in that area, the before-queue filtering can now be comfortably enabled in the GUI.
  • Improved IPv6 support
    • The Mail Proxy's SPF checker also verifies SPF records for those remote mail servers connecting via IPv6.
    • Greylisting support for IPv6 addresses (with variable netmask, defaulting to '/64') - needs to be explicitly enabled.
    • Who-objects containing IPv6 literal address work now.
  • Customizable netmask length for greylist matching
    • Instead of fixing a greylist network to a '/24' the administrator can now configure which hosts should be considered as belonging to the same network by setting a larger (or smaller) prefix.
    • This can help with receiving mail from some cloud-providers, who send out one mail from different ip addresses within a large network, which usually leads to a rather long delay and sometimes even to a legitimate mail being rejected.
    • Due to the changed database layout partial upgrades of clusters will prevent nodes running the older version from syncing the greylist database until they are upgraded.
  • Better UX for the User Spam Quarantine interface
    • If selected in the Quarantine view, the From header and the Subject are now displayed on top of the mail body.
    • It is now possible to delete mail addresses containing certain special characters (for example '/') from a users' black- or whitelist.
    • Users can set their preferred language directly in the quarantine interface instead of having to log out to change the setting.
    • Fixed a bug in the selection of multiple e-mails.
  • Handling of changes to overridden templates with ucf
    • Starting with this release all service configuration templates, copied and modified in /etc/pmg/templates get registered with ucf. Should a overridden template change with a new package version the administrator is asked and can accept or reject the changes.
    • All users who have templates in /etc/pmg/templates will be asked about the current changes for the initial registration.
  • New What Object: 'Match Archive Filename'
    • In addition to match files in archives (zip, tar.gz, rar,...) based on the file's content-type, it is also possible to look for particular filename patterns inside of archives.
    • This completes the feature matrix of matching files based on content-type or filename, as plain attachments, or inside archives.
  • Support for downstream LMTP servers
    • In certain setups there is no advantage in having a dedicated SMTP server for receiving e-mails from Proxmox Mail Gateway, since all used functionality is provided by a MTA, which speaks IMAP and LMTP (e.g., Dovecot).
    • It is now possible to configure Proxmox Mail Gateway to send e-mails directly to a LMTP relay, both as default transport and only as transport for certain domains.
  • Improvements to recently added features
    • Before-queue filtering and DKIM signing, both implemented with Proxmox Mail Gateway 6.1, have a better user experience and are considered stable now.
    • Some remaining glitches and bugs fixed for both.
    • DKIM selector handling can handle the existence of multiple selectors and in the GUI, users can comfortably switch between the active selector.
  • TLS policy selection for internal downstream servers
    • It is now possible to specify a desired level of encryption and authentication for the opportunistic TLS-encryption (STARTTLS) for downstream servers entered in your transports.
    • This can help to ensure that your internal communication is not sent in the clear over the network. It can also be used to work around broken TLS implementations in legacy downstream servers.
  • Improvements to general usability
    • The unbounded growth of the Quarantine disk usage for non-master nodes in clustered setups is fixed.
    • It's now possible to switch to incremental updates of the AV signatures for ClamAV via GUI, alleviating the problem that both methods fail in certain cases for some users.

Proxmox Mail Gateway 6.1

Released 27. November 2019

  • Based on Debian Buster (10.2)
    • Proxmox Mail Gateway is based on the latest stable release of Debian 10.2 (Buster).
  • Updated SpamAssassin rules
  • Kernel 5.3
    • Proxmox Mail Gateway shares the kernel with Proxmox VE and is based on the 5.3 series from Ubuntu 19.10
  • DKIM-Signing
    • Support for adding DomainKeys Identified Mail (DKIM) Signatures (RFC 6376) to outbound emails
    • Configuration via GUI
    • Signing happens after processing the email with the rule system, thus ensuring that it leaves the Proxmox Mail Gateway with a valid signature
    • Flexible control of which domains should get signed with sensible defaults (the relay domains)
    • Inside a cluster, one common selector minimizes the overhead for adding required DNS entries
  • Attachment Quarantine
    • The Remove Attachments action can now optionally deliver the complete email to the Attachment Quarantine
    • The Attachment Quarantine offers a comfortable GUI for selectively downloading parts of the email for further analysis, or delivering the mail to the original recipient
    • Accessible only by the administrators, it offers safety from accidentally open malicious executables, doc-files, and other attachments infected with malware
  • Adjustable SpamAssassin Rule Scores via GUI
    • Adapt the scores of individual SpamAssassin rules directly in the GUI
    • Enables you to adapt the scoring to your environment, thus achieving better ham/spam detection rates
    • Mostly for adding a bit of weight to rules, which are good indicators for Spam in your environment
    • Selectively disable Rules, which cause false positives for your environment
  • Improved handling of Configuration and Rule changes in clustered environments
    • The Filtering Engine gets notified about a range of configuration changes which require a reload
    • The notification is propagated during the cluster sync
    • This reduces the situations where you had to manually restart pmg-smtp-filter
  • Experimental Support for Before Queue filtering
    • Proxmox Mail Gateway can now optionally reject an email during the SMTP dialogue, instead of accepting it and silently discarding unwanted email
    • This is a requirement in certain situations
    • By answering with a permanent failure code (554), there is no need to generate a Non-Delivery Report, which could cause your system to get blacklisted, due to Backscatter
    • Currently incompatible with the Tracking Center, thus needs to be explicitly enabled in /etc/pmg/pmg.conf

Proxmox Mail Gateway 6.0

Released 27. August 2019

  • Proxmox Mail Gateway is based on the latest stable release of Debian 10.0 (Buster)
  • This major version update provides an easy to follow step-by-step upgrade path - https://pmg.proxmox.com/wiki/index.php/Upgrade_from_5.x_to_6.0
  • Rule name logging - each final action now logs the name of the rule which triggered it to the system log
  • The system logs get displayed faster in the GUI because they now use the Proxmox `mini-journalreader` instead of `journalctl`
  • ClamAV 0.101.4 (fix for https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=934359)
  • Postgres 11 (new major version backing the rule system)
  • OpenSSL 1.1.1c with support for TLS 1.3
  • Updated shipped SpamAssassin Ruleset
  • Countless bugfixes and improvements in the GUI labels

Proxmox Mail Gateway 5.2

Released 20. March 2019

  • Mobile Quarantine Interface
    • based on the small and modern framework7
    • Deliver/Delete/Whitelist/Blacklist mails in your Quarantine from your mobile device
  • Improvements in the LDAP integration
    • allow the use of FQDNs instead of IPs in the WebUI
    • add support for certificate verification (and enable it for new deployments)
    • add support for LDAP+starttls
  • PMG-Appliance template
    • Install PMG as a (unprivileged) Linux Container (e.g. in PVE)
    • Introduces the new 'proxmox-mailgateway-container' metapackage, which does not depend on a kernel, and results in a vastly reduced size (and fewer updates)
  • Improvements in Logging
    • pmg-smtp-filter now logs each SA-Rules score in addition to the rule names - simplifying the analysis of the spam filter's performance without the need to access the mail's source
  • Improvements in the WebUI's TLS configuration
  • pmgproxy can now be configured via '/etc/default/pmgproxy' to disable/enable certain ciphers, compression, cipher selection preference.
  • new command: `pmg-system-report`
    • Provides a overview of key characteristics of PMG's setup and performance
    • Improves the initial diagnosis for our Enterprise support
  • .eml download from the (non-mobile) Quarantine Interface
    • Lets you download the complete source of a quarantined message in .eml format for further analysis
  • Add support for custom checks
    • Enable users to integrate their own custom check logic by providing a defined interface, which can optionally be enabled, and runs a custom check before the mail gets handed to the virus scanner and rule system.
  • Improvements of Blacklist/Whitelist handling in the Quarantine Interface
    • multiselect for removing multiple entries at once
  • proxmox-spamassassin
    • Update the shipped rulesets
  • PMG-Cluster: full IPv6 support
  • ISO works on Citrix XenServer
  • Documentation available via https://pmg.proxmox.com/pmg-docs
  • Bugfixes

Proxmox Mail Gateway 5.1

Released 05. October 2018

  • Allow to configure TLS policy via GUI
  • New 'helpdesk' role
  • Support SMTPUTF8 protocol feature
  • GUI improvements
  • Update Debian Stretch 9.5
  • Update kernel to 4.15
  • Bugfixes

Proxmox Mail Gateway 5.0

Released 23. January 2018

  • Fully licensed under the open source license AGPL
  • Based on Debian Stretch 9.3 with a 4.13.13 kernel
  • ISO installer supports all ZFS raid levels
  • ExtJS based user interface
  • New API
  • Integrated documentation
  • Subscription-based enterprise support options (similar to the Proxmox VE support subscription model)
  • Bug fixes

Old Releases

  • Proxmox Mail Gateway 4.1
  • Proxmox Mail Gateway 4.0
  • Proxmox Mail Gateway 3.1
  • Proxmox Mail Gateway 3.0
  • Proxmox Mail Gateway 2.6
  • Proxmox Mail Gateway 2.5
  • Proxmox Mail Gateway 2.4
  • Proxmox Mail Gateway 2.3
  • Proxmox Mail Gateway 2.2
  • Proxmox Mail Gateway 2.1
  • Proxmox Mail Gateway 2.0
  • Proxmox Mail Gateway 1.7
  • Proxmox Mail Gateway 1.6
  • Proxmox Mail Gateway 1.5
  • Proxmox Mail Gateway 1.4
  • Proxmox Mail Gateway 1.3
  • Proxmox Mail Gateway 1.2
  • Proxmox Mail Gateway 1.1
  • Proxmox Mail Gateway 1.0 (April 2005)
Retrieved from "https://pmg.proxmox.com/wiki/index.php?title=Roadmap&oldid=80"