Comparison
Messaging Patterns Comparison
Service | Pattern | Description | Architecture |
---|---|---|---|
Storage Queue | Simple Queue | Basic FIFO message queue | Producer → [Queue] → Consumer |
Service Bus Queue | Point-to-Point | Enterprise messaging queue | Producer → [Queue] → Consumer |
Service Bus Topic | Publish-Subscribe | Fan-out to multiple subscribers | Producer → [Topic] → Multiple Subscriptions |
Event Hub | Event Streaming | High-throughput event ingestion | Producer → [Event Hub] → Multiple Consumer Groups |
Throughput & Scale Comparison
Feature | Storage Queue | Service Bus Queue | Service Bus Topic | Event Hub |
---|---|---|---|---|
Max Throughput | 20,000 msgs/sec | 2,000 msgs/sec (Standard) 100,000 msgs/sec (Premium) | 2,000 msgs/sec (Standard) 100,000 msgs/sec (Premium) | Millions of events/sec |
Message Size | 64KB | 256KB (Standard) 1MB (Premium) | 256KB (Standard) 1MB (Premium) | 1MB per event |
Queue/Topic Limit | Unlimited queues | 10,000 queues/namespace | 10,000 topics/namespace | 32 partitions/hub |
Retention | Until consumed (7 days max) | Until consumed (max 14 days) | Until consumed (max 14 days) | 1-7 days (configurable) |
Storage Limit | 500TB per account | 80GB (Standard) 1TB (Premium) | 80GB (Standard) 1TB (Premium) | Unlimited |
Delivery Guarantees & Features
Feature | Storage Queue | Service Bus Queue | Service Bus Topic | Event Hub |
---|---|---|---|---|
Ordering | Best-effort FIFO | FIFO guaranteed (When using sessions) | FIFO guaranteed (per-subscription when using sessions) | Per-partition ordering |
Delivery | At-least-once | At-least-once | At-least-once | At-least-once |
Transactions | No | Full ACID support | Full ACID support | No |
Dead Letter Queue | Manual implementation | Built-in | Built-in | Manual implementation |
Duplicate Detection | No | Yes | Yes | No |
Message TTL | 7 days max | 14 days max | 14 days max | 7 days max |
Batching | Yes (32 messages) | Yes | Yes | Yes (up to 1MB) |
Consumer Patterns
Feature | Storage Queue | Service Bus Queue | Service Bus Topic | Event Hub |
---|---|---|---|---|
Consumers | Multiple competing | Single active consumer | Multiple independent subscriptions | Multiple consumer groups |
Message Consumption | Destructive | Destructive | Destructive | Non-destructive (replay) |
Load Balancing | Competing consumers | Competing consumers | Per-subscription competing | Partition-based |
Filtering | Client-side only | Basic | Advanced SQL-like filters | Client-side only |
Peek Support | Yes | Yes | Yes | No |
Session Support | No | Yes | Yes | No |
Security & Authentication
Feature | Storage Queue | Service Bus Queue | Service Bus Topic | Event Hub |
---|---|---|---|---|
Authentication | Storage Account Key SAS Token Azure AD | Connection String SAS Token Azure AD | Connection String SAS Token Azure AD | Connection String SAS Token Azure AD |
Network Security | Private Endpoints Firewall Rules | Private Endpoints Firewall Rules | Private Endpoints Firewall Rules | Private Endpoints Firewall Rules |
Encryption | At rest & in transit | At rest & in transit | At rest & in transit | At rest & in transit |
RBAC Support | Yes | Yes | Yes | Yes |
Cost Structure
Feature | Storage Queue | Service Bus Queue | Service Bus Topic | Event Hub |
---|---|---|---|---|
Pricing Model | Per operation ($0.0036 per 10K ops) | Per operation + base cost ($0.05 per million ops) | Per operation + base cost ($0.05 per million ops) | Throughput Units ($22.50/TU/month) |
Base Cost | None | 677/month (Premium) | 677/month (Premium) | 1 TU included |
Relative Cost | $ (Cheapest) | $ (Medium) | $$ (Higher) | $ (High volume) $$ (Low volume) |
Free Tier | Yes (12 months) | No | No | No |
Management & Operations
Feature | Storage Queue | Service Bus Queue | Service Bus Topic | Event Hub |
---|---|---|---|---|
Setup Complexity | Very Simple | Medium | High | Medium |
Monitoring | Azure Monitor Storage Analytics | Azure Monitor Service Bus Metrics | Azure Monitor Service Bus Metrics | Azure Monitor Event Hub Metrics |
SDK Support | All major languages | All major languages | All major languages | All major languages |
REST API | Yes | Yes | Yes | Yes |
Management Tools | Storage Explorer Azure Portal | Service Bus Explorer Azure Portal | Service Bus Explorer Azure Portal | Event Hub Explorer Azure Portal |
Use Case Recommendations for VM Lifecycle management: [azure-event-grid-and-storage-queue-for-vm-events]
Service | Best For | Avoid When | Your Use Case Fit |
---|---|---|---|
Storage Queue | • Simple message processing • Cost-sensitive scenarios • Basic queuing needs • High-volume simple tasks | • Need guaranteed ordering • Complex enterprise features • Transactional requirements | ✅ Perfect for VM lifecycle events |
Service Bus Queue | • Enterprise integration • Guaranteed FIFO ordering • Transactional processing • Complex business logic | • High-volume scenarios • Cost-sensitive applications • Simple use cases | ⚠️ Overkill for your current needs |
Service Bus Topic | • Multiple consumer teams • Fan-out messaging • Advanced filtering needs • Microservices coordination | • Single consumer • High-volume streaming • Simple point-to-point | ⚠️ Complex for single consumer |
Event Hub | • Real-time analytics • IoT telemetry • High-throughput streaming • Event replay requirements | • Low-volume scenarios • Simple message processing • Cost-sensitive applications | ❌ Overkill for your volume |
Performance Characteristics
Metric | Storage Queue | Service Bus Queue | Service Bus Topic | Event Hub |
---|---|---|---|---|
Latency | Low (< 10ms) | Medium (10-50ms) | Medium (10-50ms) | Very Low (< 5ms) |
Availability SLA | 99.9% | 99.9% | 99.9% | 99.9% |
Geo-replication | Yes (GRS, RA-GRS) | No (Premium: Geo-DR) | No (Premium: Geo-DR) | Yes (Geo-DR) |
Auto-scaling | Automatic | Manual (Premium: Auto) | Manual (Premium: Auto) | Manual/Auto |
For the concept of ‘Subscription’ for ‘service bus’
Subscriber pattern always means: fully duplicated message across multiple subscribers.
- subscribers are NOT for loadbalancing.
- queue is for loadbalancing.
So whenever you see ‘subscription’, it means, ‘duplicated message delivery’, and it is for ALL messages.
For kafka:
- Consumer Groups == Service Bus Subscriptions
- Consumers within a grop == Compete for messages (load balancing)
- Different consumer groups means: each get all messages (duplication)
For Kafka: parition vs consumer group
- Message Placement (Producer Side) Messages are distributed to partitions based on:
- specific partition key (if provided)
- partition hash of the message key
- round-robin assignment if no key is provided
- Consumer Assignment (Consumer Side) Within each consumer group, each partition is assigned to exactly one consumer:
Key Insights About Partitions:
- Ordering Guarantee
- Within a partition: Messages are strictly ordered
- Across partitions: No ordering guarantee
- Scaling Limitations
- Maximum consumers per group = Number of partitions
- If you have 3 partitions, you can have at most 3 consumers in a group
- Consumer Rebalancing
- When consumers join/leave a group, partitions are reassigned