Unlocking the real value of data by modernizing MS SQL workloads on AWS Cloud

By | 5 minute read | September 15, 2021

Data is the new oil in this digital era, and most organizations are vouching for the same as a direct enabler for their business growth. Capturing maximum data and mining holds the key for all businesses especially for retail, banking, telecommunication and entertainment streaming as it allows them to be faster when it comes to launching new products and enables a more efficient customer service experience. This in turn leads to the explosion of data, and the challenges due to this are many-fold and specific to infrastructure that can support this exponential growth. The key challenges are data storage, scalability, usability of the same for analytics through various integrations, and more. Handling such a scale for on-premises, even by using prominent databases like MS SQL, can be a big challenge for any customer. We will discuss how Cloud can handle this with its scalable, elastic and cloud-native architecture and also bring out the real value, which Microsoft SQL offers as a database. Being the pioneer cloud platform, AWS offers all capabilities that is needed to host, migrate, run MS SQL and integrate with different, advance services to reap the real benefits.

AWS offers a long list of services and tools on their platform to support migration of Microsoft SQL workloads from on-premises to cloud. Depending on the type of workloads, data size, business requirements and the move strategy finalized during the assessment phase, we will decide the right target landing zone, whether it will be IaaS/PaaS/container-based and use the respective migration tool. There are multiple migration paths defined for the SQL database, and the same is portrayed in a workflow as below:

Microsoft SQL Modernization Path on AWS

As you can see in Figure 1, AWS provides a plethora of tools to rehost SQL engines to run on Windows or Linux platforms and enhance flexibility to reap the benefits of cloud. Organizations can choose a pay as you go model or move into an open source model that will reduce the licensing cost and enhance performance per core. Replatforming SQL into RDS using AWS DMS/SCT reduce management overhead of patching, taking backups, etc., will reduce FTE cost for the customer, and teams can focus on innovations. AWS also offers ECS and EKS for running SQL on containers that will bring in scalability, elasticity and optimum resource usage. The product development lifecycles are shortened with faster deployment along with orchestration capabilities on AWS native Kubernetes platform, which will enhance business agility. Overall, multiple options are provided for customers to modernize their Microsoft SQL workloads on AWS, making it easy for organizations to decide which migration path they should embrace.

We have discussed moving workloads into AWS as a first step of modernization. But, unless the hyperscaler provides the capability of running them in cloud native mode, we cannot reap the real benefits. From this perspective there are two critical aspects of running SQL workloads on AWS:

  1. How do we run the workloads on AWS with maximum automation, governance and guardrails in place?
  2. How do we consume the data effectively and efficiently using different cloud-native services and the integrations it offers?

This is portrayed below using a reference architecture on how differently SQL workloads can run on AWS. AWS offers rich automation capability using its Infra as a code in automating cloud foundation, application lifecycle management through its native DevOps pipeline, serverless functions and API’s to integrate between services to form data lakes, perform analytics through data mining services and use the insights for better business growth and innovation.

Reference Architecture – Integrated Ecosystem on AWS for Data Modernization

The ecosystem shown above in the architecture in Figure 2 can be explained effectively through the modernization use cases (foundation, migration & operations) and how AWS helps customers in achieving this with its platform, services and tools.

Use Cases (SQL Modernize)

 

Services

 

AWS Capabilities

 

Authentication AWS Managed AD Windows applications on AWS can be integrated with On-Prem AD using AWS Managed AD offerings. This will ensure continuity with existing Single Sign on authentications for users to connect to SQL Databases
Cloud Foundation CFN, Lambda, API Fully automated cloud foundation setup using AWS Infra as a code capability for hosting apps and DBs and integrate them with other services for various business functions.
Reliability Multi Availability Zones Robust Infrastructure provided by geographically separated AZ’s offers best in class reliability for Apps on Windows. Better RTO/RPO
HA & DR Auto scaling, Snapshots AWS auto scaler across AZ’s along with Database snapshots that can moved across regions ensure business to restore in case of any catastrophe. Active-Passive & Active–Active replication models ensures high availability.
Performance SQL on Linux, Containers Utilization of CPU cores based on the SQL workloads by moving them into containerized architecture supported by AWS. This will give better performance as well by reducing CPU cycle wastage and optimize licensing cost as well. Better capacity planning and overall cost reduction.
Migrate AWS SMS, DMS, CloudEndure, Backup & Restore, Storage Gateway, DataSync, FSx, S3, EBS AWS offers multiple migration paths for SQL workloads depending on the right ‘R’ strategy. Depending on the workload, SQL EOL support it can be hosted on EC2, RDS, Aurora, etc.
Agile Operations CloudWatch, Lambda, Systems Manager, AWS Backup AWS can bring in the right governance for the MS SQL workloads by enabling automated monitoring using CloudWatch, CloudTrail and Lambda. Be compliant with AWS Systems manager patching and monitor the patching activities by eliminating vulnerabilities. Vigorous backup capability through different backup plans by AWS Backup.
Integrations S3, Glue, Kinesis, QuickSight Use cloud native big data capabilities with MS SQL using advanced AWS services by creating data lakes for Analytics, Dashboarding and Reporting. AWS brings in best in class integrations to inject data and mine the same for business insights.
Service Management

 

Lambda, API Gateway, Service Now

 

AWS offers API programming and serverless capabilities to integrate with service management platform to monitor, remediate DB issues proactively. Creates source of records for all incident, problems and changes that occur and update CMDB.

In conclusion, we are unlocking maximum value of data modernized on AWS using the migration ecosystem it offers. This is realized through a robust infrastructure governed by having inbuilt automation and guardrails in place. Best-in-class integrations through API programming using the cloud-native architecture enables migrated Microsoft SQL data on AWS cloud to create an AWS Lake formation. This can help in creating blueprints and meaningful business workflows for real-time analytics to apply machine learning modules for better decision making and business insights. So why wait? Get set and embrace AWS Cloud for your Windows SQL workloads!