February 28, 2025
Table of Contents
This white paper details the migration of Electronic Data Interchange (EDI) functionality from a Service-Oriented Architecture (SOA) to Azure to modernize infrastructure, improve scalability, and enhance transaction efficiency. It outlines key challenges, implementation strategies, infrastructure configurations, and the impact of the migration on business operations.
- The EDI Project sought to migrate existing Electronic Data Interchange (EDI) processes from a legacy SOA-based infrastructure to Azure. The project aimed to:
- Improve performance and scalability.
- Enhance exception handling mechanisms.
- Streamline integration with Kafka and OTM.
The EDI Project was initiated to transition EDI processes from a legacy SOA-based system to Azure. This migration aimed to optimize performance, ensure seamless integration with Kafka and Oracle Transportation Management (OTM), and establish a robust exception-handling framework. The project covered two primary use cases: NBL (North America) and EDM (Mexico).
The Objective
The primary objectives of this migration included:
Improving system scalability and operational efficiency.
Enhancing exception-handling mechanisms to ensure transaction integrity.
Streamlining integration between Azure, Kafka, and OTM.
Implementing security best practices, including OAuth authentication.
Legacy SOA-based EDI infrastructure faced several limitations, including scalability constraints, inefficient business logic handling, and complex exception management. Additionally, Kafka integration challenges and real-time shipment tracking issues required a modernized approach to support business needs effectively
The objective of this modernization is to shift the legacy EDI functionality from SOA to Azure to modernize the infrastructure and enhance efficiency. The project catered to two primary scenarios:
NBL Use Case (America):
- Outbound Transactions: Process 204 requests coming from OTM, store them in a database, and send them to Kafka.
- Inbound Transactions: Handle 214, 210, and 990 requests coming from Kafka, process them, and send them to OTM.
EDM Use Case (Mexico):
- Integrate and process EDI transactions specific to the Mexico use case.
Implementation Strategy
The infrastructure structured using Azure services to support seamless data processing and transaction management. Key components included:
- Azure Cosmos DB: Used for data persistence and partitioned for optimized transaction processing.
- Azure Function Apps: Implemented to handle inbound and outbound EDI transactions asynchronously.
- Azure API Management (APIM): Enabled OAuth 2.0 authentication and secured API endpoints.
- Azure Key Vault: Managed API keys and database connection strings securely.
- Application Insights: Integrated for proactive monitoring and logging.
Development Phases
Outbound Transactions (204 Requests):
- Processing XML payloads, storing transactions in Cosmos DB, transforming for Kafka, and updating transaction status.
Inbound Transactions (214, 210, 990 Requests):
- Inbound Transactions (214, 210, 990 Requests): Validating responses, integrating with live tracking APIs, and ensuring transaction consistency.
Secure OAuth communication and API integration for Mexico-specific transactions.
Enhanced logging mechanisms for transaction flow tracking.
Azure Configuration:
- Create a new resource group
- Apply necessary tags for cost allocation and management
- Set up access control (IAM) permissions for team members
- Configure Cosmos DB account with SQL API.
- Define structured collections with partition key “/transactionId” for optimized retrieval.
- Enable TTL for data retention.
- Base Setup:
1.Create Function App on consumption plan
2.Select Node.js runtime stack
3.Enable system-assigned managed identity
4.Set up custom domains with SSL certificates
5.Implement Azure Sink Connector for Kafka integration
- Application Settings:
1.Configure environment variables for different environments
2.Setup connection strings for Cosmos DB
3.Add API keys for external services
4.Configure CORS policies
- Service Setup:
1.Created APIM instance
2.Enabled Application Insights integration
- API Configuration:
1.Created separate API for EDI services
2.Set up OAuth 2.0 authentication
- Policies Implementation:
1.Set up request/response transformation
2.Added IP filtering
3.Configured cross-origin resource sharing
- Basic Configuration:
1.Created Application Insights resource
2.Integrated with Function Apps
3.Set up custom events and metrics
4.Configured sampling rates
- Monitoring Setup:
1.Created custom dashboards
2.Set up availability tests
3.Implemented custom alerts
4.Configured log analytics workspace
- Alert Configuration:
1.Set up error rate thresholds
2.Configured performance alerts
3.Implemented availability monitoring
4.Set up action groups for notifications
- Initial Setup:
1.Created Key Vault instance
2.Enabled soft-delete and purge protection
3.Set up access policies
4.Configured network security
- Secret Management:
1.Stored connection strings
2.Managed API keys
3.Stored SSL certificates
4.Implemented secret rotation policy
Challenges and Resolutions
| Challenge | Resolution | |
|---|---|---|
| Incorrect Purpose Code Mapping in Kafka | Kafka could not derive the purpose codes accurately for 204 requests. | Migrate the logic to Azure, enabling precise and efficient processing. |
| Regional Mismatch Between Function Apps & Kafka | Function App and Kafka cluster regions mismatch, causing integration failures. | Redeploy the infrastructure in the correct region to ensure successful integration. |
| Exception Handling in Inbound Transactions | Inbound scenarios required rolling back transactions in case of API failures on OTM’s side. | Build robust error-handling mechanisms to revert transactions and notify Kafka. |
| SOA 214 Logic Migration for Shipment Tracking | Ensure real-time shipment updates by integrating the live tracking API. | Shift 214 processing logic to Azure for better real-time data updates. |
Key Takeaways
Improved Scalability and Adaptability
Migrating to Azure provided a highly scalable and flexible infrastructure, capable of supporting complex EDI transactions.
Optimized Regional Deployments
Aligning Kafka and Azure Function Apps within the same region significantly reduced latency and improved integration performance.
Enhanced Exception Handling
With a structured rollback mechanism, the new system ensured data integrity and reduced transaction failures.
Business Agility
The transition allowed faster adaptation to evolving business requirements while ensuring security and compliance.
Final Words
The successful migration of EDI processes from SOA to Azure will significantly improve system performance, reliability, and scalability. The integration of Kafka, OTM, and modern Azure services streamlines business operations while ensuring robust security and monitoring mechanisms. Future enhancements will include AI-driven transaction validation and automated anomaly detection to further optimize transaction handling.
Author
Zeeshan Mukhtar
Websites used to be something you built once and basically forgot about. That doesn’t work …Read More »- Websites used to be something you built once and basically forgot about. That doesn’t work …Read More »
- Learn how to plan an Optimizely CMS 13 upgrade with .NET 10, Optimizely Graph, Visual …Read More »
