Tag: middleware
All the articles with the tag "middleware".
Kafka (Part 7): Integrating Apache Avro and Apicurio Schema Registry to Ensure Message Compatibility Between Producers and Consumers
To address deserialization failures caused by Kafka message format changes, this article walks through migrating from JSON to Avro and integrating Apicurio Schema Registry. It covers schema definition, Java class generation, producer and consumer configuration, and version compatibility checks, reducing the risk of poison-pill messages.
Kafka (Part 5): Consumer Callbacks, Scheduled Retries, and Rebalancing
This largely completes a messaging system built with Kafka. The key code has been explained across the articles; see the source repository for the complete design and implementation. The next Kafka article is planned to cover synchronizing Oracle data to PostgreSQL with Kafka Connect.
Kafka (Part 4): Consuming JSON, Sharing a Deserializer, and Improving Throughput
This article implements Kafka consumers for JSON messages, including shared deserialization, deduplication, retries, and offset commits. Load tests inform partition counts and consumer settings, and the template method pattern reuses the consumption flow for emails of different priorities.
Kafka (Part 3): Sending JSON with a Shared Serializer and Improving Producer Throughput
Implements a Kafka producer that sends JSON messages, using a shared serializer for different objects and configuring asynchronous sending, failure recording, and retries. Based on actual email-message sizes, it discusses how compression, batch size, request size, and waiting time affect throughput.
Kafka (Part 2): Designing a Messaging System to Decouple Email Delivery
Uses email delivery to explain a design that decouples a business system from an email system through Kafka. It discusses responsibility boundaries, a unified message format, sending and callback flows, and a fallback when switching between old and new systems, providing background for subsequent producer and consumer implementations.
Analyzing the ShardingJDBC Execution Flow
Uses the MyBatis call flow to analyze how ShardingJDBC 4.1.1 integrates with JDBC through custom DataSource, Connection, and PreparedStatement implementations. It traces parameter buffering, physical connection acquisition, and result-set handling, then simulates the proxy workflow with simplified code.
Using ShardingSphere Database Middleware: ShardingProxy (Part 1), Data Sharding and Read/Write Splitting
Uses ShardingProxy 4.1.1 to configure data sharding and read/write splitting across multiple MySQL instances, then verifies routing through query and insert logs. It covers starting the proxy service, connecting an application, and MySQL driver and version compatibility considerations.
Using ShardingSphere Database Middleware: ShardingJDBC (Part 2), Read/Write Splitting
Builds on a data-sharding example to configure ShardingJDBC read/write splitting for a MySQL primary-replica setup. It introduces primary and replica data sources and replica load-balancing strategies, observes queries alternating between replicas, and records the unresolved issue of a slow first query.