middleware-integration-patterns

# Middleware Integration Patterns

This skill activates when a practitioner needs to determine whether middleware is required for a Salesforce integration scenario, or when comparing iPaaS platforms (MuleSoft, Dell Boomi, Workato, Informatica) across vendor-agnostic selection criteria. It provides pattern-level guidance for multi-system orchestration, protocol conversion, message transformation, and transactional guarantees. It does not cover the internals of any single vendor's connector configuration.

---

## Before Starting

Gather this context before working on anything in this domain:

- **Is middleware genuinely required?** Salesforce's native capabilities handle many integration scenarios without a separate platform. Platform Events, Change Data Capture, Pub/Sub API, Flow HTTP Callout, Apex callouts, and External Services collectively address real-time event push, incremental change capture, outbound REST, and inbound webhook reception. Middleware adds operational overhead; justify it with a concrete requirement it meets that the native platform cannot.
- **What are the orchestration requirements?** Middleware is mandatory per Salesforce Architects guidance when the integration requires cross-system orchestration (fan-out to multiple backends, aggregation across systems), protocol conversion (SOAP to REST, flat-file to JSON), persistent message queuing with guaranteed delivery, or distributed transaction coordination. Apex callouts provide none of these natively at scale.
- **What is the team's existing tooling and license position?** Enterprises with an active MuleSoft Anypoint license are rarely justified in adopting a second iPaaS. Evaluate additional vendors only when the existing platform demonstrably cannot address the requirement.

---

## Core Concepts

### When Middleware Is Mandatory

According to Salesforce Architects integration guidance, middleware is the right choice when any of the following conditions apply:

1. **Cross-system orchestration:** The integration must fan out to, or aggregate from, more than two backend systems in a single business transaction. Apex orchestration hitting governor limits (100 callouts per transaction, 120-second timeout) is a technical forcing function.
2. **Protocol conversion:** Source or target systems communicate over protocols Salesforce cannot natively consume — AS2, MQ, JMS, SFTP, proprietary binary formats, SOAP with WS-Security, JDBC. Apex HTTP callouts reach only HTTP/HTTPS endpoints.
3. **Reliable message queuing:** The integration must guarantee delivery even when either system is temporarily unavailable. Salesforce has no native durable queue for outbound messages to arbitrary systems. Platform Events provide replay up to 72 hours within Salesforce's event bus, but delivering to an external system that may be down still requires an intermediary queue.
4. **Transactional guarantees:** The integration must coordinate a unit of work across two or more systems that each have their own transaction boundaries. Salesforce's SOAP Outbound Messages provide at-least-once delivery, but two-phase commit across external systems requires a middleware transaction coordinator.
5. **Complex message transformation:** Source data requires structural transformation beyond field mapping — aggregation, splitting, enrichment from a third source, canonical data model normalization. Apex can handle simple transformations but is not designed for pipeline-style transformation orchestration.

### iPaaS Vendor Comparison: Selection Criteria

All four major iPaaS vendors connect to Salesforce through certified connectors or APIs. Selection criteria that distinguish them for Salesforce integrations:

| Criterion | MuleSoft Anypoint | Dell Boomi | Workato | Informatica IICS |
|---|---|---|---|---|
| **Primary strength** | API-led architecture, developer control, enterprise governance | Rapid connector deployment, low-code process orchestration | Business-user automation, recipe-driven, speed to value | Enterprise data integration, data quality, MDM convergence |
| **Salesforce connector** | Anypoint Salesforce Connector (certified, deep API coverage including Streaming/Pub-Sub) | Boomi Salesforce connector (SOAP/REST/Bulk, real-time listener) | Salesforce trigger/action with real-time event subscription | Salesforce connector with CDC support and data quality transforms |
| **Orchestration model** | Java-based Mule flows; complex CEP and routing via DataWeave and Mule runtime | Boomi Process canvas; low-code but extensible with scripting steps | Recipe-based, trigger-action model; conditional logic via conditional steps | PowerCenter/IICS mappings; strong for ETL-style batch and CDC |
| **Protocol coverage** | Broadest: HTTP, JMS, AMQP, SFTP, AS2, JDBC, SAP, legacy | Broad: HTTP, JMS, SFTP, EDI, SAP, legacy via Boomi AtomSphere | Narrower: primarily HTTP/webhook; limited legacy protocol support | Broad for data: JDBC, flat-file, S3, SAP, HTTP; limited event streaming |
| **Transformation language** | DataWeave (purpose-built, expressive) | Groovy scripting or Boomi map function | Workato formulas (Ruby-based, limited) | Mapplet / Expression Transformation (SQL-like) |
| **Real-time event support** | Streaming API, Pub/Sub gRPC, CDC via connector | Real-time via HTTP listener + Salesforce listener operations | Salesforce real-time trigger (polling every 5 minutes minimum) | CDC-based via Informatica Intelligent Data Management |
| **Governance and API management** | Anypoint API Manager, Exchange; full API lifecycle | Limited native API management; relies on external gateways | Minimal API governance out-of-the-box | Axon governance for data assets; not API-lifecycle focused |
| **Target buyer** | Enterprise IT / integration CoE | Mid-market and enterprise, business-technology teams | Business users, citizen integrators, SMB/mid-market | Enterprise data teams, data governance programs |

### Native Salesforce vs. Middleware Decision Boundary

Before selecting any middleware vendor, confirm that native capabilities cannot address the requirement:

| Requirement | Native Salesforce Capability | Middleware Required? |
|---|---|---|
| Push Salesforce record changes to external HTTP endpoint | Flow HTTP Callout Action or Apex callout | No, if endpoint is HTTP and volume is within governor limits |
| Subscribe to Salesforce Platform Events from external system | Pub/Sub API (gRPC streaming), CometD | No (external system consumes directly) |
| Ingest webhook events from external system into Salesforce | Salesforce REST API from external caller | No |
| Nightly bulk load from ERP (100K+ records) | External ETL tool or Salesforce Bulk API | Middleware or ETL if transformation is required |
| Fanout: one Salesforce event triggers updates to 3+ backends | Middleware required | Yes — governor limits and reliability |
| Transform SFTP flat-file from legacy system into Salesforce records | Apex cannot read SFTP; transformation complex | Yes — middleware handles SFTP, parse, transform, load |
| Guaranteed delivery when target system is offline | No native durable external queue | Yes — middleware queue provides store-and-forward |

---

## Common Patterns

### Pattern 1: Event-Driven Fanout via Middleware

**When to use:** A Salesforce record change must trigger coordinated updates to multiple external systems (ERP, data warehouse, email platform) as a logical unit.

**How it works:**
1. Salesforce publishes a Platform Event or Pub/Sub API event on record change.
2. Middleware subscribes to the event channel (Pub/Sub gRPC or CometD).
3. Middleware routes the event to a fanout pattern: one copy per downstream system.
4. Each downstream leg has independent error handling and retry logic; failures in one leg do not roll back others.
5. A correlation ID (e.g., Salesforce record Id) is propagated to all legs for end-to-end traceability.

**Why not Apex:** Apex triggers hitting multiple callout endpoints face the 100-callout-per-transaction limit and the 10-second synchronous timeout. Failures in one callout leg can roll back the entire transaction if not carefully isolated. Middleware removes Salesforce from the synchronous path entirely.

### Pattern 2: Protocol Conversion for Legacy System Integration

**When to use:** A legacy backend (mainframe, AS400, SAP RFC, EDI trading partner) communicates over a protocol Salesforce cannot natively speak.

**How it works:**
1. Middleware exposes a protocol adapter for the legacy system (JMS, MQ, AS2, JDBC, RFC).
2. Middleware transforms the legacy message format into a canonical JSON/XML model using DataWeave, Boomi Maps, or equivalent.
3. The canonical model is loaded into Salesforce via REST API composite requests or Bulk API, depending on volume.
4. Return data (e.g., ERP order confirmation) is mapped back from Salesforce to the legacy protocol format for acknowledgment.

**Why not Apex:** Apex has no JDBC, JMS, MQ, or SFTP connectivity. Any connectivity to these protocols from Salesforce requires an intermediary.

### Pattern 3: Store-and-Forward for Resilient Delivery

**When to use:** The target system has planned maintenance windows or is frequently unavailable, and lost messages are unacceptable.

**How it works:**
1. Middleware receives the Salesforce-originated event or record and places it on an internal durable queue (MuleSoft JMS, Boomi Atom Queue, Workato Job Queue).
2. A downstream consumer reads from the queue and attempts delivery to the target.
3. On failure, the message is retained and retried with exponential backoff.
4. A dead-letter queue captures messages that exhaust retries, with alerting and manual reprocessing tooling.

**Why not Platform Events:** Salesforce Platform Events have a 72-hour replay window for Salesforce-internal consumers. External systems accessing the Pub/Sub API bear the responsibility for maintaining their own replay position. If the external system is offline for more than 72 hours, events are lost. Middleware's durable queue is not subject to Salesforce's retention window.

---

## Decision Guidance

| Situation | Recommended Approach | Reason |
|---|---|---|
| Simple Salesforce-to-external-HTTP callout, low volume | Apex callout or Flow HTTP Callout — no middleware | Native capability is sufficient; middleware adds operational overhead |
| Complex orchestration across 3+ systems | Middleware required; MuleSoft preferred for enterprise governance needs | Governor limits and reliability make Apex inadequate; MuleSoft for API-led governance |
| Business-user automation, SMB context, Salesforce + SaaS apps | Workato | Low-code recipe model reduces IT dependency; strong SaaS connector library |
| ETL-style bulk data sync with data quality and MDM requirements | Informatica IICS | Best-in-class data transformation, profiling, and master data management |
| Mid-market process automation, moderate complexity | Dell Boomi | Low-code, fast deployment, competitive total cost vs MuleSoft at mid-market scale |
| Enterprise API governance + multi-consumer integration + Agentforce | MuleSoft Anypoint | API-led architecture, Anypoint Exchange catalog, Agent Fabric for Agentforce integration |
| Legacy protocol conversion (AS2, MQ, JMS, SFTP, RFC) | MuleSoft or Boomi | Both have broad protocol support; Workato and Informatica weaker here |
| Team with no iPaaS and wanting speed-to-value | Workato or Boomi | Lower learning curve than MuleSoft; functional in days, not weeks |

---

## Recommended Workflow

Step-by-step instructions for an AI agent or practitioner working on middleware selection for Salesforce:

1. **Confirm that middleware is genuinely required.** Map the integration requirements against the native capability table above. If Platform Events, Pub/Sub API, Flow HTTP Callout, or Apex callouts satisfy the requirement within governor limits, document that decision and stop here.
2. **Identify the integration drivers.** Determine which specific requirements mandate middleware: protocol conversion, orchestration fanout, message queuing, transactional guarantees, or complex transformation. Each driver points toward different vendor strengths.
3. **Audit existing platform investments.** Check whether the organization already holds an active MuleSoft, Boomi, Workato, or Informatica license. Introducing a second iPaaS is almost never justified; optimize the existing platform first.
4. **Score vendors against the selection criteria.** Use the comparison table (protocol coverage, transformation capability, real-time event support, governance, buyer profile) to score the top two candidate platforms against the specific integration requirements.
5. **Validate Salesforce connector coverage.** For the selected vendor, confirm the Salesforce connector supports the required API (Bulk, Streaming/Pub-Sub, REST, SOAP) and the Salesforce org edition. Workato's real-time polling interval (5-minute minimum) is a disqualifying constraint for sub-minute latency requirements.
6. **Design error handling and observability.** Define the dead-letter queue strategy, retry policy, and alerting thresholds before implementation. Per Salesforce Architects guidance, every middleware integration must have a documented error handling strategy and a path for failed-message recovery.
7. **Review against the checklist before committing.** Confirm all architecture decisions are documented, error handling is designed, and the middleware necessity decision is recorded.

---

## Review Checklist

Run through these before marking work in this area complete:

- [ ] Middleware necessity confirmed: native Salesforce capabilities evaluated and found insufficient for this scenario
- [ ] Primary integration drivers documented: which of orchestration / protocol conversion / queuing / transactionality / transformation mandates middleware
- [ ] Existing iPaaS license inventory checked before evaluating new vendors
- [ ] Vendor selected against specific requirements, not default preference or brand familiarity
- [ ] Salesforce connector capability confirmed for the selected vendor (API type, edition compatibility, real-time event latency)
- [ ] Error handling strategy designed: dead-letter queue, retry policy, alerting, and manual recovery path
- [ ] Correlation IDs or idempotency keys planned for end-to-end message traceability
- [ ] Related skills consulted: mulesoft-salesforce-connector (if MuleSoft selected), api-led-connectivity (if multi-layer governance needed), error-handling-in-integrations

---

## Salesforce-Specific Gotchas

Non-obvious platform behaviors that cause real production problems:

1. **Platform Events 72-hour replay window is not middleware-level durability** — Salesforce Platform Events support replay for up to 72 hours for Pub/Sub API consumers. Teams that use Platform Events as the integration backbone without a middleware durable queue discover that external subscribers offline for more than 72 hours permanently lose events with no error or notification. Middleware with a persistent queue removes this exposure.
2. **Workato's Salesforce real-time trigger is polling, not push** — Despite being labeled "real-time," Workato's Salesforce trigger polls for new/changed records at a minimum interval of 5 minutes. Integrations requiring sub-5-minute latency for record changes cannot use Workato's standard trigger; a separate Pub/Sub API listener flow must be designed.
3. **Governor limits do not pause — they fail hard** — When an Apex callout integration approaches the 100-callout-per-transaction or 10-second synchronous limit, the transaction is not queued or paused; it fails with an unhandled exception. Teams that incrementally add integration logic to Apex triggers hit this cliff unexpectedly. Plan for middleware before the limit is reached, not after it is first hit in production.

---

## Output Artifacts

| Artifact | Description |
|---|---|
| Middleware necessity decision record | Documented yes/no decision on whether middleware is required, with rationale citing specific requirements |
| iPaaS vendor selection scorecard | Scored comparison of candidate vendors against integration-specific criteria |
| Integration pattern recommendation | Selected pattern (event-driven fanout, protocol conversion, store-and-forward, etc.) with justification |
| Error handling design | Dead-letter queue strategy, retry policy, alerting thresholds, and recovery runbook outline |

---

## Related Skills

- integration/mulesoft-salesforce-connector — Use when MuleSoft is selected: covers Anypoint Salesforce Connector configuration, watermark, batch processing, and auth
- integration/api-led-connectivity — Use when MuleSoft is selected and multi-layer API governance is needed: System/Process/Experience API layer design
- integration/integration-pattern-selection — Use to determine synchronous vs asynchronous vs event-driven pattern before selecting middleware
- integration/error-handling-in-integrations — Use when designing the error handling, retry, and dead-letter strategy for the middleware integration
- integration/retry-and-backoff-patterns — Use for configuring middleware retry logic with Salesforce-specific backoff constraints