Propose a shared event-ingestion service
Fictional worked example: On 14 May 2025, Northstar Services' platform team proposed a shared event-ingestion service to reduce duplicated integrations. The proposed scope is ingestion and consistent validation while existing producer interfaces remain stable during a gradual transition. Decision owner: Platform Engineering Lead. Recommendation: run a bounded pilot before deciding on adoption, subject to resolving ownership and observability questions below.
Problem and constraints
What the proposal is intended to solve
In this fictional example, product teams maintain separate pipelines for similar event sources. Event formats vary, so validation and integration work is repeated across teams. The proposal addresses that duplication; it does not assume that every downstream processing function should move to one central service.
Constraints reviewers should test
- Producer compatibility: keep current producer interfaces stable during a gradual transition.
- Team ownership: consuming teams retain responsibility for their event definitions and migration decisions.
- Consistent validation: define how shared validation behaves when incoming events do not match an agreed schema.
- Bounded service scope: distinguish ingestion responsibilities from downstream processing and analytics.
Compare the options
Qualitative tradeoffs in the fictional Northstar example; these are not measured results.
- Strength:
- Least migration work initially
- Tradeoff:
- Duplication and validation differences continue
- Strength:
- Consistent validation with producer interfaces kept stable
- Tradeoff:
- Requires platform work and agreement on service ownership
- Strength:
- Broadest opportunity for shared processing
- Tradeoff:
- Largest coordination change and wider operational coupling
Proposed decision and rationale
Decision recorded 14 May 2025 in the fictional example
Proposed decision: pilot a shared event-ingestion service with two producers before deciding on wider adoption, keeping producer interfaces stable.
This option targets duplicated integration and inconsistent validation without making the wider change of centralizing all event processing. The rationale is a design judgment for this example, not evidence of measured cost or performance improvement.
Revisit the choice if the producer-interface constraint changes, the platform and consuming teams cannot agree on service ownership, or review evidence shows that shared validation does not address the duplication described here.
Open questions for review
Discussion prompts—not an approval or task-tracking workflow.
Who owns schema changes?
How will teams observe rejected events?
What evidence supports wider adoption?
Proposed pilot and return path
Fictional planning record, 14 May 2025. The dates and effort are proposals, not approved commitments or measured results.
| Stage | Evidence to bring | Decision boundary |
|---|---|---|
| Prepare by 16 May | Platform and the two producer leads name schema owners, rejection diagnostics and a shared test-event set. Reserve an estimated five platform engineer-days and two days per producer team. | Do not start if ownership or the effort allocation is unresolved. |
| Shadow pilot, 19–30 May | Replay the agreed test events through existing and proposed validators without sending duplicate production events downstream. Compare accepted/rejected outcomes and investigate every unexplained difference. | Existing producer paths remain authoritative; the pilot does not replace them. |
| Review on 2 June | Bring the comparison log, unresolved differences, rejection-diagnostic examples and updated effort estimate to the Platform Engineering Lead and producer reviewers. | Only propose a controlled cutover once differences are explained and producer owners agree. Otherwise continue current paths and revise or stop the pilot. |
| Before any later cutover | Document replay/duplicate handling, monitoring, the role authorized to stop rollout and a tested return to the existing path. | A separate cutover review is required; this proposal does not authorize production routing changes. |