Your phone can hold contacts from iCloud, Google, Fastmail and your own Nextcloud server at the same time. All four appear in the contacts app, all four update correctly when you edit them anywhere, and the same person appears four times — and never once gets reconciled.
People assume this is a misconfiguration. It is the most carefully designed behaviour in the stack.
The reason is in the protocol, not in your phone
Contacts move between a phone and a server using CardDAV. It is the contacts half of CalDAV, and it was built for one job: letting one client keep one server's address book in step with itself.
The shape of it is one server, one client, a mirror. Your contacts app is the client, and it can hold any number of them at once — which is exactly why the situation looks so much like a solved problem. It is not. Clients do not reconcile each other, and no setting in any phone will change that.
Each connection on its own works. If you edit a contact on the Nextcloud server, it changes on your phone. What does not exist — and this is the part that surprises people — is any connection between the four servers.
The community that has thought about this hardest puts it plainly:
You can add different accounts from different servers/providers to your devices, and all those calendars and contact lists in the different providers will appear in your iOS/macOS calendar and contacts apps. However, there's no cross-account or cross-server sync happening. Server-side they remain with their respective providers in their respective accounts.
Which means the duplicate you are looking at is not a mistake anybody made. It is what you get when a human being — you — saves the same person into a second address book, because that is the only tool available for doing it.
Why the duplicates pile up rather than settle
Two properties of this arrangement compound each other.
There is no identity layer. CardDAV moves fields between records. It has no concept of "this record and that record are the same person" — matching is not part of the protocol, and no amount of CardDAV configuration adds it. The record has a name and some phone numbers, and nothing in it says which other record it belongs with.
Every server is authoritative over its own copy. Contacts created on the Nextcloud server exist there because something wrote them there, and a contact added in Google exists in Google. Neither is a cache of the other. There is no master record anywhere for the two to be compared against — because in this architecture, there is not supposed to be one.
So the answer to "why do I have four copies of my accountant" is not that something went wrong four times. It is that you met your accountant once, and there are four places a human being could have saved them, and nothing in the protocol connects them.
What people do about it
Do nothing and treat the duplicates as a search problem. Search all four accounts every time. Free, permanent, and it works until the day you are in a hurry and pick the number that was disconnected eight months ago.
Make one address book canonical and stop writing anywhere else. Pick one — your own server is the usual choice if you have one, since it is the one you control. Add everyone there, let the phone pull from it, and treat the others as read-only history. This is the correct answer for a lot of self-hosters and it costs nothing. The failure mode is the familiar one: you save someone from the wrong place once, and the duplicate is back.
Sync server to server with a tool built for it. vdirsyncer is the one the self-hosted community reaches for, and it genuinely does this — it will sync two CardDAV servers to each other, unattended, on a schedule. If your problem is exactly "two of my four address books should be the same address book", it is the right tool and it is free. What it is not: a reconciler. It copies, it does not understand that two records are one person, and getting it to write into a live address book without producing a fourth copy of everyone involved is real work. It is a power tool for someone who enjoys configuring syncs. That is a legitimate thing to be, and this paragraph is not an objection to it.
Use a reconciler that understands identity. Something outside the ecosystems that reads each address book, works out that the same person appears more than once, and writes the resolved record back to all of them. This is the option that fixes the symptom above rather than the duplication underneath it.
The part worth sitting with
There is a reason this has stayed broken for twenty years, and it is not laziness.
Every company in this stack is fine with it. Google is not losing customers because you cannot merge two Google accounts. Apple is not losing iPhone sales. Fastmail is not losing mailbox subscriptions because your contacts are split up. Nextcloud is not losing installs because CardDAV does one thing extremely well.
The people the gap hurts are people who work across more than one account — which, now that most working lives involve a personal address and a work address, is most people. There is simply no company for whom fixing it is worth anything, which is the same reason there is no native solution and never has been.
Somebody has to build it anyway. That is a small, unglamorous, genuinely useful thing to build, and if you have decided that somebody should be you, the rest is mechanics.
The short version
Your phone holds four address books and does not merge them because CardDAV is a server-to-client protocol and no client cross-syncs two servers. That is by design, not by accident, and no configuration fixes it. Making one server canonical solves it for nothing. Syncing two servers with vdirsyncer solves it for one pairing. Reconciling all of them, and keeping them reconciled, is what does not exist yet.