Brief Summary
This video compares event-driven architectures (EDA) powered by Apache Kafka with request-response architectures, highlighting six key factors: reactivity, coupling, consistency, historical state, architectural flexibility, and data access/reuse. EDA uses asynchronous event exchange for loosely coupled, reactive systems, while request-response involves direct, synchronous communication between services. EDA excels in flexibility, historical data access, and data reuse, making it suitable for complex data ecosystems. Request-response is better for real-time operations but can be challenging for data access across multiple services.
- Event-driven architectures (EDA) are loosely coupled and asynchronous, using event streams for communication.
- Request-response architectures involve direct, synchronous communication between services.
- Key factors for comparison include reactivity, coupling, consistency, historical state, architectural flexibility, and data access/reuse.
Intro
The video introduces event-driven architectures (EDA) powered by Apache Kafka, where systems communicate by exchanging events. It contrasts EDA with request-response architectures, such as REST and RPC, where services communicate directly. The comparison is based on six factors to highlight the strengths and weaknesses of each architecture. An example scenario involves a storefront where customers place orders via their cell phones, illustrating both request-response and event-driven approaches.
Reactivity
In an event-driven architecture, the storefront writes an order record to an "orders" topic upon receiving it. The Fulfillment service then consumes this record, stores it, and processes it for shipping. This demonstrates reactivity, where actions are triggered by events. In contrast, a request-response service requires the storefront's business logic to initiate the next step. The storefront either instructs the Fulfillment service to process the order or the Fulfillment service requests work from the storefront. This direct communication defines the reactive behavior in request-response systems.
Coupling in Space and Time
Event-driven architectures are inherently asynchronous, decoupling services in space and time. By writing a record to a topic, the storefront is decoupled from the Fulfillment service. If the Fulfillment service is offline, it can resume processing orders when it comes back online. The consumer doesn't need to know the producer's identity, and they are only coupled via the Kafka topic. In request-response, services need to know each other's API details and locations at runtime, creating tight coupling. Both services must be online simultaneously, and if one goes offline, the service must retry, highlighting the synchronous nature and tight dependencies.
Consistency
The asynchronous nature of event-driven architectures leads to eventual consistency. The Fulfillment service will eventually be consistent with the orders placed in the storefront, but the time this takes depends on event volume and consumer processing rate. Request-response services can achieve easier consistency, especially if all data is kept within a single service. However, if the Fulfillment service requests work from the storefront, the order data will be eventually consistent, which is typical when data is copied between services.
Historical State
In an event-driven architecture, if a customer modifies an order, the storefront updates the data and writes a new record to the topic. The Fulfillment service consumes this update and adjusts accordingly. The Kafka topic maintains a complete history of state, allowing new services, like an inventory service, to consume data from any point in time. This provides flexibility for reprocessing data in case of failures. Request-response systems typically overwrite data in place. The storefront must communicate modifications to the Fulfillment service. Historical state is not commonly stored alongside current state in typical database patterns, though it is possible.
Architectural Flexibility
Event-driven architectures offer significant architectural flexibility. New services can be easily built, and data from across the company can be integrated. For example, incoming inventory data from warehouses can be added to the inventory service to create a complete picture. Order data can be reused for multiple use cases. In contrast, request-response systems rely on service APIs as building blocks. Creating a new inventory service requires communication with multiple existing services, such as Fulfillment and storefront, through their APIs. This can be challenging if service boundaries don't align well, requiring multiple service calls to complete tasks, especially when data is spread across systems.
Data Access and Data Reuse
Event-driven architectures are powerful for data reuse. Existing event streams can power various applications, including data lakes, warehouses, machine learning models, and generative AI vector databases. Tools like Kafka Connect facilitate creating connectors to ingest data, and native consumers offer flexibility. The work done to define and populate events into streams can be reused extensively. Request-response architectures primarily serve real-time operational data. Populating data lakes requires setting up ETL jobs, which can be challenging. The curated APIs for business use cases don't necessarily transfer over when trying to format data for analytics, machine learning, and generative AI.
Summary
Event-driven architectures, like those powered by Apache Kafka, provide a loosely coupled and asynchronous framework for building reactive applications. Event streams act as reusable building blocks for powering applications and data-intensive use cases. Request-response services provide business functions for other services to call, but uptime and liveness are critical, as failures can affect dependent services. While request-response services can power distributed applications, accessing important business data for various use cases remains complex and challenging.
