In this answer I do not dive into MongoDB technology and how to mitigate issues with duplicates manually. I explain what was referenced in the article (Debezium), suggest to use out of the box solutions and mention other patterns than transactional outbox.
Debezium
If you are willing to use Kafka as your message broker then there is a project called Debezium which is based on Kafka Connect.
Small note here - there is a feature called Debezium Server which doesn't require Kafka nor Kafka Connect, but it is currently (1.9) in incubating state.
It has a MongoDB connector, so you can just point it at your MongoDB instance. With MongoDB Outbox Event Router the implementation of the transactional outbox pattern is (almost) just a matter of proper configuration. Personally I've used Debezium and it worked, but it may be challenging to get the configuration right first time.
Duplicates
About duplicates. Debezium does the work for you and really tries to not send duplicate messages, but keep in mind that your consumers should always be prepared to receive duplicate messages. From the docs:
When everything is operating nominally, Kafka consumers will actually see every message exactly once. However, when things go wrong Kafka can only guarantee consumers will see every message at least once. Therefore, your consumers need to anticipate seeing messages more than once.
It's near to impossible to achieve exactly-once delivery in a distributed system, although there is some magic trickery you can do with Kafka like enabling exactly-once semantics.
Still, in real-world applications your consumers should be idempotent and here is a good article why.
Scalability
The transactional outbox using change streams or any other form of CDC - Change Data Capture mechanism is currently the most scalable option if you are writing to the DB first and then later want to send a message - addressing dual writes problem. It can use low-level database features, so with proper DB technology the messages are published almost in real-time with minimal performance hit on the DB itself.
However, there is also a listen to yourself pattern, where you write to the message broker instead of writing to the DB.
If you don't want the message to be visible immediately to "others" (parallel is not an option) then you can have an internal consumer that listens to those messages, writes to the DB and maybe after that you can publish the message further for "others" - notice that you need to record the fact that you processed the message after publishing it further (commit/ack to the message broker) and your own internal consumer must be idempotent - it can receive duplicates when something fails after writing to the DB.
This way you are not tied to the specific DB technology, so you do not really care if it is a relational or document databse. It can scale really well, but it has its own drawbacks too - the state of the system is not so obvious, should you backup now your message broker data too? and so on. The solution really depends on what you are trying to achieve.
There are even some ideas of using Event Sourcing and Event Store itself as a message broker... This solution and others are described in this red hat article.
Horizontal scaling with Debezium
Usually you want your messages / events to be processed in order only within some boundaries. If you can identify areas in your Domain that can be processed in parallel then you can create separate event streams and you could in theory configure multiple Debezium connectors per area / Bounded Context or maybe even per Aggregate. It really depends on your Domain and required throughput. However, more connectors means higher load on the DB, so there is a tradeoff. Keep in mind that single Debezium connector can actually handle a lot and you can still process things in parallel on the other side (sink) of Debezium - in Kafka realm you do this by setting proper partition keys:
The MongoDB connector does not make any explicit determination about
how to partition topics for events. Instead, it allows Kafka to
determine how to partition topics based on event keys. You can change
Kafka’s partitioning logic by defining the name of the Partitioner
implementation in the Kafka Connect worker configuration.
Kafka maintains total order only for events written to a single topic
partition. Partitioning the events by key does mean that all events
with the same key always go to the same partition. This ensures that
all events for a specific document are always totally ordered.
Debezium connector doesn't do a lot of CPU work, it's mostly I/O.
So if you don't have a particular reason to jump into multi-connector setup then don't do it. Start with single connector and distribute your events with proper partition keys, so you can horizontally scale your consumers. What is interesting, by default the partition key field in MongoDB Outbox Event Router is called aggregateid.
Final notes
If you can use Debezium then go for it. There is a lot of stuff in the docs about how it actually works with MongoDB, what are the requirements etc:
...operations on the actual business collection(s) and the insert into the outbox collection must be done as part of a multi-document transaction, which have been being supported since MongoDB 4.0, to prevent potential data inconsistencies between business collection(s) and outbox collection. For future update, to enable updating existing data and inserting outbox event in an ACID transaction without multi-document transactions, we have planned to support additional configurations for storing outbox events in a form of a sub-document of the existing collection, rather than an independent outbox collection.
...With change streams, the MongoDB server exposes changes to
collections as an event stream. The Debezium connector watches the
stream and delivers the changes downstream. And, when the connector
sees a replica set for the first time, it looks at the oplog to get
the last recorded transaction and then performs a snapshot of the
primary’s databases and collections. When all the data is copied, the
connector then creates a change stream from the position it read
earlier from the oplog... The MongoDB connector is also quite tolerant
of changes in membership and leadership of the replica sets, of
additions or removals of shards within a sharded cluster, and network
problems that might cause communication failures. The connector always
uses the replica set’s primary node to stream changes, so when the
replica set undergoes an election and a different node becomes
primary, the connector will immediately stop streaming changes,
connect to the new primary, and start streaming changes using the new
primary node. Likewise, if connector experiences any problems
communicating with the replica set primary, it will try to reconnect
(using exponential backoff so as to not overwhelm the network or
replica set) and continue streaming changes from where it last left
off. In this way the connector is able to dynamically adjust to
changes in replica set membership and to automatically handle
communication failures.
