How Transparent Huge Pages negatively influence MySQL Server Performance

Introduction

Transparent Huge Pages (THP) are a feature in modern Linux kernels intended to improve performance by reducing the overhead of memory management for large processes. However, THP can negatively impact the performance of MySQL Server, particularly InnoDB, MySQL's default storage engine. Here’s why.

Transparent Huge Page impact on MySQL Server Performance

1. Memory Allocation Overhead

  • Fragmentation: THP can lead to memory fragmentation. While it attempts to allocate large pages (typically 2MB), under memory pressure, it might fail and fall back to smaller pages, causing fragmentation.
  • Inefficient Memory Usage: MySQL’s InnoDB engine manages its memory buffers (like the buffer pool) in chunks. THP's larger page sizes can lead to inefficient memory usage, where memory is allocated but not effectively utilized.

2. Jitter and Latency Issues

  • Increased Swap Tendency: Systems with THP enabled are more prone to swap, even when there is available memory. Swapping can cause significant performance degradation for database systems.
  • Performance Inconsistency: THP can cause unpredictable performance, especially in high-throughput, write-intensive workloads. This is often due to the way THP handles memory under different load conditions.

3. Interference with InnoDB’s Own Optimization

  • Page Management: InnoDB is optimized for standard 4KB pages. THP, by changing the page size, can interfere with these optimizations, reducing the effectiveness of InnoDB’s internal mechanisms for memory management.
  • CPU Overhead: Managing huge pages can incur additional CPU overhead, affecting the overall performance of the MySQL server.

4. Difficulty in Performance Tuning

  • Tuning Complexity: THP adds another layer of complexity to the performance tuning of MySQL. Diagnosing issues caused by THP can be challenging, as its effects may not be immediately apparent.
  • Inconsistent Behavior: The performance impact of THP can vary depending on the workload, making it difficult to create a consistent and reliable performance tuning strategy.

Disabling Transparent Huge Pages in MySQL

Due to these potential issues, it is often recommended to disable THP on servers running MySQL. This can be done by setting the transparent_hugepage kernel parameter to never:

These commands should be run as root and added to the system's startup scripts (like /etc/rc.local) to persist across reboots.

Conclusion

While Transparent Huge Pages can benefit certain applications, they can negatively impact MySQL performance by causing memory fragmentation, increased swap tendency, and additional overheads. Disabling THP is commonly recommended for MySQL servers, especially those with high workloads, to maintain consistent and optimized performance. It’s important to test and monitor the impact of such configurations changes in your specific environment.

About Shiv Iyer 485 Articles
Open Source Database Systems Engineer with a deep understanding of Optimizer Internals, Performance Engineering, Scalability and Data SRE. Shiv currently is the Founder, Investor, Board Member and CEO of multiple Database Systems Infrastructure Operations companies in the Transaction Processing Computing and ColumnStores ecosystem. He is also a frequent speaker in open source software conferences globally.