Arizona State Hospital Application Migrations
Arizona Department of Health Services
The Arizona Department of Health Services (ADHS) is an agency within the State of Arizona with a mission to promote, protect, and improve the health and wellness of individuals and communities in Arizona. The Arizona State Hospital (ASH) is a division under ADHS. ASH provides long-term inpatient psychiatric care to Arizonans with mental illnesses who are under court order for treatment. The hospital operates programs within a 260-bed funded facility, is accredited by The Joint Commission, and the Civil Hospital is certified to receive reimbursement from the Centers for Medicare and Medicaid Services (CMS).
Arizona State Hospital (ASH) had several web applications hosted in the State DataCenter. The apps had to be relocated out of the Datacenter due to ageing infrastructure and end of life hardware at the Datacenter.
AWS was picked to be the new home for the apps for reasons not limited to, better Return on Investment (ROI), reduced capital expenditure, reliable and state of the art infrastructure, On-demand availability of compute resources, regulatory and compliance needs specially HIPAA, PHI and PII, elasticity and scalability, less to manage with managed services.
In addition to failing infrastructure at the DataCenter there were also issues with the security of the apps with respect to limiting the exposure and the threat landscape of the apps. A lot of the servers, running public facing web applications were placed in the public subnets exposing them to security threats from the internet.
ADHS required a solution that would:
- Result in cost savings via the consolidation of systems.
- Decrease the threat landscape and improve the overall security posture for its applications.
- Improve visibility into the health of various components by collecting and monitoring, logs and metrics.
- Allow for the procurement of the latest hardware and server instances for reliable operations.
- Include the automated patching of servers using AWS Systems Manager.
Proposed Solution & Architecture
The solution proposed included the migration of Virtual Machines (VMs) from the DataCenter to AWS using AWS Server Migration Service. Leveraging this would Migrate VM images and convert them into Amazon Machine Images (AMIs) in AWS. Then, EC2 instances were created from the migrated images in private subnets and Load Balancers were placed in the public subnet to receive external traffic for the instance.
In order to improve performance efficiency, Zuggand also proposed that DHS provision Amazon WorkSpaces Windows desktops. This allowed ADHS to run Cloud-based tools to access its RDS database over AWS’s low latency network.
The environment is currently utilizing AWS Systems Manager to keep Windows Server instances up-to-date on patches. For reliability, we consolidated SFTP onto one solution and enabled auto-healing. And finally, we configured the environment to collect logs and metrics from all EC2 Instances and AWS Services into CloudWatch to improve visibility into the health of various AWS components.
Below is the architecture diagram for the overall solution in AWS and a list of AWS services used along with their definitions.
List of AWS Services Used
The following AWS Services were implemented as part of the solution:
- AWS Directory Service: Also known as AWS Managed Microsoft AD, enables your directory-aware workloads and AWS resources to use managed Active Directory in the AWS Cloud.
- AWS Systems Manager: AWS Systems Manager helps maintain security and compliance by scanning your instances against your patch, configuration, and custom policies. You can define patch baselines, maintain up-to-date anti-virus definitions, and enforce firewall policies. You can also remotely manage your servers at scale without manually logging in to each server.
- Amazon EC2: A web service that provides secure, resizable compute capacity in the cloud. It is designed to make web-scale cloud computing easier for developers.
- Amazon Elastic File System: Provides simple, scalable, elastic file storage for use with AWS Cloud services and on-premises resources.
- AWS CloudFormation: A service which gives developers and systems administrators an easy way to create and manage a collection of related AWS resources, provisioning and updating them in an orderly and predictable fashion.
- Amazon EBS: Amazon Elastic Block Store (EBS) is an easy to use, high performance block storage service designed for use with Amazon Elastic Compute Cloud (EC2) for both throughput and transaction intensive workloads at any scale.
- Amazon CloudFront: A global content delivery network (CDN) service that securely delivers data, videos, applications, and APIs to your viewers with low latency and high transfer speeds.
- Amazon RDS: A managed relational database service that provides six familiar database engines to choose from, including Amazon Aurora, MySQL, MariaDB, Oracle, Microsoft SQL Server, and PostgreSQL.
- Amazon CloudWatch: A monitoring service for AWS cloud resources and the applications that run on AWS.
- AWS WAF: A web application firewall that helps protect web applications from common web exploits that could affect application availability, compromise security, or consume excessive resources.
- AWS VPN: AWS Virtual Private Network (AWS VPN) lets you establish a secure and private tunnel from your network or device to the AWS global network. AWS VPN is comprised of two services: AWS Site-to-Site VPN and AWS Client VPN. AWS Site-to-Site VPN enables you to securely connect your on-premises network or branch office site to your Amazon Virtual Private Cloud (Amazon VPC). AWS Client VPN enables you to securely connect users to AWS or on-premises networks.
- AWS Server Migration Service: AWS Server Migration Service (SMS) is an agentless service which makes it easier and faster for you to migrate thousands of on-premises workloads to AWS. AWS SMS allows you to automate, schedule, and track incremental replications of live server volumes, making it easier for you to coordinate large-scale server migrations.
- Amazon WorkSpaces: Amazon WorkSpaces is a managed, secure cloud desktop service. You can use Amazon WorkSpaces to provision either Windows or Linux desktops in just a few minutes and quickly scale to provide thousands of desktops to workers across the globe.
- AWS Config: AWS Config is a service that enables you to assess, audit, and evaluate the configurations of your AWS resources. Config continuously monitors and records your AWS resource configurations and allows you to automate the evaluation of recorded configurations against desired configurations.
- AWS Trusted Advisor: AWS Trusted Advisor is an online tool that provides you real time guidance to help you provision your resources following AWS best practices.
- AWS Key Management Service (KMS): A managed service that makes it easy for to create and control the encryption keys used to encrypt data.
- AWS Certificate Manager: AWS Certificate Manager is a service that lets you easily provision, manage, and deploy public and private Secure Sockets Layer/Transport Layer Security (SSL/TLS) certificates for use with AWS services and your internal connected resources.
List of Third Party Services Used
Third Party Services that were implemented as part of the solution
- GitLab: A service that enables teams to collaborate and work from a single conversation, instead of managing multiple threads across disparate tools.
- MSP360 (Formerly Cloudberry Lab) Backup and Recovery:: Backup-as-a-Service platform with centralized management and monitoring. File and folder backup for Windows workstations and servers.
- Jenkins: A service that is used to automate all sorts of tasks related to building, testing, and delivering or deploying software.
- LoadImpact: An on-demand service that provides online load and performance testing service that lets you test your website, web-app, mobile app or API over the Internet.
The entire process took a couple of months and involved the following steps:
Consultation and diagnosis:
- Discussions with key stakeholders to determine the suitability of an AWS solution for the ADHS website.
- Check application inter-dependencies with other on-premise applications and shared services.
- Develop a migration strategy for databases and files.
- Verify how the application is classified in the business. Business critical and LOB applications demand high availability
Architecture & Delivery:
- Re-engineer and Re-architect ADHS’s internally hosted infrastructure to the AWS platform using DevSecOps automation.
- Deliver a solution that included high availability, multiple availability zones and elastic load balancers.
- Leverage managed services from AWS as much as possible – Aurora MySQL RDS, CDN, WAF, etc.
Training and Knowledge Transfer:
- Ensure self-sufficiency of staff for future management of the environments
- Transition developers to DevOps (an amalgamation of two roles spanning development and operations)
- Build relationships with key ADHS stakeholders to bring them onboard with the project and encourage adoption of the AWS solution.
By migrating its critical workloads to AWS, ADHS now has a stronger security posture for the applications related to the Arizona State Hospital. In addition, by leveraging Managed Services by AWS, ADHS was able to recognize significant cost savings and increased reliability of its systems.
In particular, the agency now can automatically:
- Downscale during troughs and thereby lower costs (pay only for what you use).
- Add capacity as needed to ensure high reliability of its systems.
- Patch its critical infrastructure to maintain its security posture.
- Aggregate logs in a centralized system for the purpose of improving security and performance over time.
The solution has also improved agility, time to market and innovation by:
- Leveraging AWS’s proven infrastructure and builds around available solutions thereby reducing investment and maintenance cost.
- Providing a robust, creative open environment to work.
- Implementing round-the-clock monitoring mechanism that sends all necessary alerts.
Alignment to Well Architected Framework (WAF)
By optimizing the architecture around the five (5) WAF pillars, ADHS was able to gain the benefits of a well-architected design in the cloud:
- Optimize costs by using Managed services like RDS, Route 53, Certificate Manager, AWS Directory Services etc.
- Monitor AWS Spend using billing alarms.
- Right size Instances by using recommendations from Trusted Advisor for optimizing resources, instance types and sizes.
- Consolidate services into fewer number of EC2 Instances.
- Made architectural choices based on cost/budget, business needs and benchmarking.
- Selection of compute resources, instance family, type and size based on characteristics, cost and business needs.
- Selection of storage solution bases on access patterns, characteristics and requirements.
- Proactively record and monitor performance related metrics and generate alarm based notifications.
- Utilize latest EC2 Instance types for better performance at lower cost.
- Determine priorities by extensively evaluating customer and compliance needs.
- Ensure operational readiness by training personnel to support production workloads.
- Identify key performance indicators and define workload metrics.
- Monitor the performance of resources through metrics collected in CloudWatch.
- Enabled AWS WAF to protect application assets from external attacks.
- Control human access by granting least privileges.
- Tighten firewall rules to protect compute resources by reducing the blast radius.
- Regularly scan for vulnerabilities and patching using AWS Systems Manager.
- Protect all data at rest using KMS keys for encrypting EBS volumes, Snapshots and S3 buckets.
- Protect all data in transit by enforcing SSL encryption and managing SSL Certificates inside AWS Certificate Manager.
- Regularly monitor and manage service limits with AWS Trusted Advisor.
- Perform data backups automatically and encrypt it using KMS keys.
- Define recovery objectives for downtime and data loss use recovery strategies to meet the recovery objectives.
Lessons Learned / Outcomes
The applications used by the Arizona State Hospital was another set of mission-critical workloads successfully migrated to AWS by DHS and Zuggand. Due to the nature of their applications and data, it was critical to meet the high levels of regulatory and compliance requirements as part of the overall architecture.
The tight timeline requirement to have all systems out of the State Data Center by the end of the year meant the right teams needed to be involved to ensure success. Zuggand recommended DHS create a Cloud Center of Excellence team to drive the project. The team was made up of technology, business owners, and executive leadership to ensure the entire organization was prepared for the transformation.