3 knobs you should know in redhat transparent hugepage

There are 3 knobs  in redhat_transparent_hugepage/ in the order of effectiveness

• enabled   – this is the main knob. Setting as disable will disable THP entirely.
• defrag – THP is on. If disabled, no defrag by application thread. See note (1) below.
• khugepaged/defrag – THP is on. If disabled, no defrag by kernel thread khugepaged. See note (2) below

 Note that defrag can happen in 2 scenarios:

1. When application thread needs memory, it traps into kernel and tries to allocate huge page. If defrag is on, the process will try to coalescent small pages into large pages. (This is what Espresso ran into and the Hadoop blog describes)
2. THP also has a khugepaged that tries to defrag in the background (so that more huge pages are avail and application thread defrags less frequently).

Thus echo “disable” to “enabled” is sufficient. 

Performance impact of dirty pages in Linux

Memory (RAM) is divided into pages of equal size. A page is the smallest unit of memory that can be manipulated.

My Tech Project Notes describes Dirty Pages as

Dirty pages are the pages in memory (page cache) that have been updated and therefore have changed from what is currently stored on disk. This usually happens when an existing file on the disk is modified or appended.

Page cache on a Linux system is the area where filesystem based I/O is cached and that these settings affect how the cache is utilized by the kernel.

We can tune pdflush to determine how much RAM based cache to use for (dirty pages) data targeted to disk and how frequently to flush that cached data by writing the pages back to disk.

There are two types of flush to disk in Linux and parameters / ratios controlling how often they are triggered

1. Stop the world – Controlled by dirty_ratio.

This is the Big ass flush, that will lock the memory during the flush, the ditry_ratio determines the threshold based on the total size of the Memory/Dirty page Cache  ( Dirty page cache size is not the same as total RAM size, its usually less than total RAM size)

For example, if the dirty_ratio is set to 20%, or 0.2, a “stop the world” flush is invoked when the dirty page cache is  20% full.

2. Background Flush – controlled by dirty_background_ratio

This is a intrusive flush, similar to CMS in JVM, which happens in parallel and does not block the memory for reads and writes.

The default value for dirty_background_ratio is 50% of dirty_ratio, or any value lower than 50%  of the dirty_ratio set by the admin.

for example, if you set the dirty_page ratio to be 20% and the dirty_background_ratio to be 20% as well, then the kernel will set the dirty_background_ratio to 10%.

the logic in the kernel would look something similar to this.

If ( dirty_background_ratio >= dirty_ratio )

{

dirty_background_ratio = dirty_ratio / 2

{