I’ve been running my own mail server for well over ten years now. It’s pretty old, so it’s hard to make changes to it, but it’s running in Kubernetes. I was using a mixture of Postfix, OpenDKIM, OpenDMARC, and Amavis for spam filtering with SpamAssasin, but it wasn’t very good at catching spam. Instead, its time move to rspamd. It’s much newer and encapsulates DKIM, DMARC, DNS based blacklisting, bayesian filtering, etc. all in one single tool.

Here’s my notes on migrating, what it took to get it going and some tweaks I made.

Setup Redis/Valkey

Rspamd uses Redis to store runtime stats and config. Deploy it using Helm, or manually. I’m going to assume it’s deployed in the mail namespace with a service named called rspamd-redis.

Rspamd Configuration

I started with a basic configuration defined below. Starting with this meant I could skip the quick start guide that mandated that I run the rspamadm configwizard command, though you might have some slightly different needs

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
apiVersion: v1
data:
  classifier-bayes.conf: |
    servers = "rspamd-redis.mail.svc.cluster.local.";
    backend = "redis";
  redis.conf: |
    read_servers = "rspamd-redis.mail.svc.cluster.local.";
    write_servers = "rspamd-redis.mail.svc.cluster.local.";
  worker-normal.inc: |
    bind_socket = "0.0.0.0:11333";
  worker-proxy.inc: |
    milter = yes; # Enable milter mode
    timeout = 120s; # Needed for Milter usually
    upstream "local" {
      default = yes; # Self-scan upstreams are always default
      self_scan = yes; # Enable self-scan
    }
kind: ConfigMap
metadata:
  name: rspamd-config
  namespace: mail

Then deployed rspamd like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: rspamd
  namespace: mail
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 256Mi
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: rspamd
  namespace: mail
spec:
  replicas: 1
  selector:
    matchLabels:
	  app: rspamd
	  component: main
  template:
    metadata:
      labels:
        app: rspamd
        component: main
    spec:
      containers:
        - image: rspamd/rspamd
          imagePullPolicy: IfNotPresent
          name: rspamd
          volumeMounts:
            - mountPath: /etc/rspamd/local.d
              name: config
            - mountPath: /var/lib/rspamd
              name: data
              subPath: data
      volumes:
        - configMap:
            defaultMode: 420
            name: rspamd-config
          name: config
        - name: data
          persistentVolumeClaim:
            claimName: rspamd

Then in postfix, I remove all the existing filters and use only rspamd:

1
2
3
4
5
- non_smtpd_milters = inet:localhost:9999, inet:localhost:9998,
- content_filter = amavisfeed:[127.0.0.1]:10024

+ smtpd_milters = inet:rspamd.mail.svc.cluster.local:11332
+ non_smtpd_milters = inet:rspamd.mail.svc.cluster.local:11332

DNS Blocklisting

Upon starting rspamd, I get a bunch of warnings like this:

1
rspamd_monitored_dns_cb: DNS reply returned 'no error' for zen.spamhaus.org while 'no records with this name' was expected when querying for '1.0.0.127.zen.spamhaus.org'(likely DNS spoofing or BL internal issues)

It filters email, but still lets through a lot of obvious spam. This happens because rspamd queries several DNS-based block-list providers, like Spamhaus, to identify which IP addresses commonly send spam and reject them. But providers, like Spamhaus, have a free-tier and paid tiers and chose to block public DNS servers because public DNS servers anonymize the source for queries and Spamhaus is unable to limit traffic. I’m a very low-volume mail receiver, so the free tier is fine.

My server was just using the default DNS server provided by my dedicated server provider, OVH, which they considered to be a public resolver. Thus every query was being rejected.

A diagram showing a query to a public resolver which then goes to Spamhaus’ server, but they reject the query

To fix this, I’m going to have to change my resolver to not use a public resolver and instead use a resolver that does not use a public resolver. Instead, I’ll use a resolving DNS server, one that sends queries directly to Spamhaus, i.e. a recursive resolving DNS server. I just needed a simple containerized server. CoreDNS, which I normally use, did not support recursive DNS queries without an external plugin. Instead, I came across unbound which natively did.

Configuration

The following is the configuration I used. The server is configured to forward queries for .cluster.local. to the Kubernetes internal kube-dns instance. This is required because rspamd needs to connect to other Kubernetes services, such as Redis.

Additionally, we must allow all DNS blocklist domains to return internal IP addresses via the private-domain configuration option. This is critical because DNS-based block lists commonly use A DNS queries that return IP addresses to signal what action a mail server should take. For example, Spamhaus returns 127.0.0.2 when a request is on a specific filter list (other responses.) Without this configuration, unbound would discard all responses.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
apiVersion: v1
data:
  unbound.conf: |
    server:
        chroot: ""
        
        # Skip IPv6 since my cluster is not dual-stack yet
        do-ip6: no

        private-domain: cluster.local.
        # Add all DNS blocklists below
        private-domain: dnsbl.manitu.net.
        private-domain: surbl.org.
        private-domain: spamhaus.org.
        private-domain: spameatingmonkey.net.

    # Forwards all queries for internal cluster queries to kube-dns
    forward-zone:
        name: "cluster.local."
        forward-addr: 10.43.0.10@53
kind: ConfigMap
metadata:
  name: rspamd-dns-resolver
  namespace: mail

Switch to unbound

Next, we need to start the DNS server. I added it as a side-car container with the existing Rspamd service and configured the pod’s dnsConfig to unbound.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
apiVersion: apps/v1
kind: Deployment
metadata:
  name: rspamd
  namespace: mail
spec:
  template:
    spec:
      containers:
        - name: rspamd
          # ... elided
        - image: mvance/unbound:latest
          imagePullPolicy: IfNotPresent
          name: dns
          resources:
            limits:
              memory: 128Mi
            requests:
              cpu: 8m
              memory: 64Mi
          volumeMounts:
            - mountPath: /opt/unbound/etc/unbound/unbound.conf
              name: dns-config
              subPath: unbound.conf
      dnsConfig:
        nameservers:
          - 127.0.0.1
        options:
          - name: ndots
            value: '1'
      dnsPolicy: None
      volumes:
        # ...
        - configMap:
            defaultMode: 420
            name: rspamd-dns-resolver
          name: dns-config

After restarting Rspamd, it started using the DNS block lists and was able to flag even more obvious spam.

A screenshot from Rspamd showing results from an email that contained URIs blocked via DNS block listing

Conclusion

Migrating to rspamd from my previous solution was a great idea. It consolidated and reduced the services I had to run and it actually helped cut down on the spam that made it through with better rules. The UI was very easy to investigate what emails were being marked and why.

A pie chart from the Rspamd UI showing 29% of emails being rejected as spam

My previous solution based on Amavis and SpamAssassin was a black-box and it was difficult to understand what’s going on without digging through logs.

Comments

Comments are currently unavailable while I move to this new blog platform. To give feedback, send an email to adam [at] this website url.