I Stopped Treating Staff Sync as a Background Job

The first version of a staff synchronization feature sounded like a scheduled-data problem: fetch a central feed, update profiles, and run it regularly. Once I worked through real editorial cases, I stopped thinking of it as a background job. Staff profiles contain public identity, biographies, locations, images, and site-specific visibility decisions. Creating, updating, or removing one is a publishing action. The reliable design became a controlled workflow with preview, explicit enrollment, review actions, idempotent media handling, and a clear boundary between central authority and local choice.

The feed was authoritative, but not for everything

The central source knew who belonged to each brand and supplied shared profile fields. The destination site still owned local publication state, selected overrides, and the decision to expose a profile. Treating the feed as total authority would have overwritten editorial work; treating it as optional reference would have allowed stale staff information to persist.

I documented ownership field by field. The sync could update feed-owned values while preserving local fields outside that contract. This made disagreements visible instead of resolving them by whichever process wrote last.

Preview became the main safety feature

Before any write, the operator can see proposed creates, updates, unchanged records, skipped rows, and possible removals. That preview is not a cosmetic summary. It uses the same matching and normalization rules as execution, so it is a rehearsal of the real decision.

I learned to include reasons, not only counts. “Five skipped” is weak. “Five skipped because their required photos are incomplete” tells the operator what can be fixed and confirms that the system did not silently discard them.

Enrollment made automation intentional. Existing local profiles were not automatically handed to the feed. I added a manual enrollment workflow that links a reviewed local record to its central identity. Feed-only staff could also be created through a controlled action.

This step prevented name similarity from becoming authority. Slugs and stable source identifiers were prioritized, compatibility values were normalized, and uncertain matches stayed in review rather than being forced.

Images needed their own idempotency

Downloading a profile photo on every sync would create duplicate attachments and unnecessary processing. I recorded the remote source identity and reused an existing attachment when the image had not changed. A repeat sync therefore produced the same media relationship instead of another file.

I also kept image failure separate from profile identity. A temporary media problem should not make the system invent a new person or lose the durable link it already knows.

Responsive actions matter during long operations. Once the workflow grew beyond a tiny feed, a single opaque request was not enough. Review and manual sync actions needed clear progress, bounded work, and stable results if the browser waited or retried.

My acceptance checklist became: preview equals execution logic; reruns are safe; remote identity persists; images deduplicate; local-only fields survive; removals require an explicit policy; and every outcome is explainable to an editor.

The business value is confidence in public identity

A fast sync that occasionally publishes the wrong person or removes a valid profile is not efficient. It shifts cost to support, editorial review, and reputation. A controlled workflow may add one approval step, but it removes hours of detective work after a silent mistake.

The final system still supports scheduling. The difference is that automation now operates inside a documented ownership model rather than treating a feed response as permission to rewrite the site.

Removal was the most sensitive operation. Creating and updating profiles received most of the early attention, but removal had the larger consequence. A person missing from today’s feed might have left the organization, been temporarily excluded by a source error, or moved between brand assignments. Automatically trashing the local profile would turn a transient feed problem into a public content incident.

I separated “not present,” “not eligible for this site,” and “explicitly inactive” where the source provided enough information, and surfaced proposed removals in review. I also preserved attribution needs: an author who leaves may still need to remain available as the historical byline on published content even when their directory profile is no longer promoted. That led to a broader rule I now reuse: synchronization state and historical identity are not the same field. Removal policy must say what disappears from listings, what remains resolvable, what happens to authored content, and who approves the transition.

Auditability completed the workflow. Each run records when it occurred, which source it used, what actions were proposed, and what the operator approved. I avoid storing unnecessary personal data in logs, but I keep enough identifiers and outcome codes to reconstruct why a public profile changed. That history is especially valuable when source teams and site editors are different people.

What I now decide before scheduling a sync

For the next synchronization project, I would define authority, identity, and removal policy before writing the fetcher. Then I would build preview from the same service used by execution. If the operator cannot explain what the next run will do, the job is not ready to be automatic.

Photo by Walls.io on Pexels.