Arm Versus x86 for database server comparison review

AWS Graviton VS Intel to power Amazon RDS for MySQL

At CarsGuide we decided for migrate some of our common services hosted in AWS from a traditional x86 instance to the Graviton2-based instances, aka Arm instance. We want to explore the new processor and see how it can helps optimize our cloud performance and cost. This review is a real case comparison of performance before and after the migration of the database service running on AWS RDS for the application powering carsguide.com.au website.

A lot of comparison tests that I have read out-there run scenario where CPU utilization is maxed out. In this comparison I have instead measured the performance of real server load, hence usually low for an online services (because you want to keep available resources to handle surge in requests). It’s not a labs experiment but more a long term review for web application.

I first experimented with arm based cloud server in 2018 (see Review of an ARM-based cloud server). At that time it was just a test. In 4 years Arm servers has been made available at most cloud providers and have found many useful applications. But how do they perform to power a common database service?

The guinea pig application

The review is done on the (real) website carsguide.com.au before and after migration of the database from a x86 to an arm server. CarsGuide is a news, reviews and advice Australian website about automotive. The site handles 10+ Millions monthly page views, the backend server receives 300 requests per minutes under normal load and up to 1000 rpm under heavy load.

It is powered by a Drupal 7 engine. If you are not familiar with Drupal CMS, just keep in mind that database service is a key component: the CMS makes extensive use of tables to store content and structure data. Even more for this one because of several components present on every page, each one triggering multiple queries during page generation to retrieve related data from the database. On average 1 request to the back-end generates 150 databases queries here. One last metrics to show the importance of the DB is when considering the average server response time of 500ms, 270 ms is spend on Database (ie more than 50%) when only 144ms is spend on running the php code.

Database is MySQL 5.7, it was migrated to 8.0 during this migration.

The hardware

Before

The DB was powered by a db.m5.xlarge Amazon RDS Instance: that is 4 vCPU, 16GiB memory, priced at 0.326 USD per hour (Sydney region, reserved instance, 1 year)

According instances.vantage.sh the underlying CPU is Intel Xeon Platinum 8175, ie a 64-bit 24 cores x86 server microprocessor introduced by Intel in mid-2017.

After

The DB is now powered by a db.m6g.xlarge Amazon RDS instance: that is 4 vCPU, 16GiB memory (same), priced at 0.29 USD per hour (Sydney region, reserved instance, 1 year)

The commercial name of the underlying CPU is AWS Graviton2, it is a custom built by AWS first launched late 2019. Under the hood this is 64-bit 64 cores Arm Neoverse N1.

The performance

The migration was done on the 2nd of June-2022.

The traffic of the website, and hence the load on the server and the database is not predictable and that makes it harder to compare the performance of the 2 databases. But there are patterns and using our telemetry data I was able to compare similar period before and after the migration. It’s not perfect, hence there are a significant error margin in the end results, but it’s done with the available data and the goal is to discover some obvious overall impacts on the application behavior.

There is a daily pattern of traffic: the website is busier during the daytime than during night time. And there is a weekly pattern: weekend are generally busier than week days. In the comparison below I have always compared similar period eg: a 48 hours from Saturday 12pm, to Sunday 11:59pm in May with a similar period in June, or 6 hours on Thursday 3am to 9am in May with similar period in June.

I have organized 2 different scenario to push the test further and gain more insights: normal load and heavy load. Normal load is the real site traffic. There is a CDN setup to handle popular requests and flatten the load on the backend, the result is a nice an steady “normal load” of the server just below 300 rpm. Heavy load is a scenario where I have artificially increased significantly the load (by generating extra requests, based on real traffic, directly to the backend) to push the throughput to 1000rpm. I did that repetitively before and after migration to collect metrics.

I ended comparing the average of 3 normal load periods of 48h each before with the average of 3 normal load period of the same duration after. And the same for heavy load periods.

The comparisons metrics are:

• Application throughput (request per minute): shouldn’t change. This is a metric to make sure that the before and after are aligned (see precautions described above), to make the comparison valid.
• Database CPU Utilization: maybe the most important comparison, to measure the expected benefit of the arm architecture.
• Application response times: Total server response time (including mysql, php and memcache respond time: the 3 parts of the response time)

Below is the summary of results: comparing 3 periods of normal load on a x86 versus 3 periods of normal load on arm. Followed by 3 periods of heavy load on x86 versus 3 periods of heavy load on arm. Then comparing the average, in percentage:

Throughput is constant (+0.58% and +0.22%, within the error margin): good. The compared load are very similar.

Results

• Under heavy load (defined here as 40%-50% CPU Utilization for the database) there is same CPU utilization (-2.99% within error margin). But it is consistent with other comparisons out read where the benefit of arm starts to show only above 70% CPU utilization.
• Under normal load (15% CPU utilization), this is the odd finding: CPU utilization on arm is ~17% higher than on x86. It doesn’t have much impact because it is precisely small load but still…
• The MySQL average time spent is in par with the database CPU utilization: on heavy load no significant change: -0.79%. On normal load the arm is a little bit slower than the x86: +6.94%
• On the compatibility side of things: no problems: the MySQL engine works exactly the same, the change is totally transparent for the application.

They are some other odd results coming from that study (memcache, php spent time) but i think the most significant findings are the ones above.

Conclusions

Running a website database on arm architecture doesn’t bring performance benefits, most of the time. It does bring cost benefit, the arm server chosen is 11% cheaper than the x86 used before. It has no compatibility impact of the application code, ie it is a smooth transition in both directions.

We learned a lot and gained confidence: we know now that we can transition service to arm-based server with very little effort without impacts on the code.

I’m looking forward for the next steps where we could run web applications on arm, because, having already all of our apps on containers we can try to maxed out the usage of underlying server to some level where arm is truly more performant and rip more cost benefits.