What are your insights and guidelines for "Batch Modernization"? What are the target future architecture for Batch Workloads? Such as Data Streaming, Events, Data Mesh, AWS Batch? What are the criteria to chose each one of the future architecture?
Software Engineer in Software, 51 - 200 employees
Business Objectives and Use CasesAnalyze the volume and velocity of data that needs to be processed
onsider integrating with data lakes, databases, external APIs, etc
Event-driven architectures (e.g., AWS Lambda, Apache Kafka) are suitable for real-time processing
IT Analyst in Media, 1,001 - 5,000 employees
Look out for single report creating platform across the enterprises. To save data and logs for years to compare the growth and control the value of data. To build on single platform.Senior IT Analyst - data engineering in Real Estate, 1,001 - 5,000 employees
Batch modernization involves transitioning from traditional batch processing to more modern and efficient architectures. The target future architecture for batch workloads often includes elements like data streaming, event-driven processing, and technologies like AWS Batch or data mesh. When choosing between these, consider factors like workload characteristics (e.g., latency requirements, data volume), scalability needs, cost considerations, and the existing skill set of your team. Data streaming is suitable for real-time, low-latency processing, while event-driven architectures enable efficient handling of event-triggered batch jobs. AWS Batch offers scalability and managed infrastructure for batch workloads, and data mesh is beneficial for handling complex data pipelines. The choice depends on aligning these factors with your specific batch processing requirements.Content you might like
Improving the developer experience (DX)39%
Improving user/customer experience65%
Solutions to measure and report on code/application quality54%
Consolidating tools to reduce context-switching for your developers28%
Improve Application Performance Monitoring (APM) Capabilities8%
142 PARTICIPANTS
The future of retail belongs to automated and cloud-based solutions58%
Merchants are not ready34%
Maybe, but not by 20258%
160 PARTICIPANTS
Engineer in Services (non-Government), Self-employed
In one of the BUs, the GPT capability of a third-party chatbot was activated for HR Knowledge Base. Senior Systems Analyst / Team Leader in Government, 10,001+ employees
Tackling data flow challenges from mammoth sources like SAP to your Data Platform demands a strategic touch. In this age of exponential tech, it's about harmonizing innovation with practicality. I believe the following ...read more
1. What do we want to solve here ?
2. What are options do we have?
3. How is the economical impact if we deploy one of that solution ?
4. Do we have the resource to do it?
Once you have those defined, it should be easier to set the goal and boundaries for batch workloads you’re aiming