[RHEL] Understanding Linux Hugepages

Understanding Linux Hugepages

By unixadminschool com

Before we discuss about Huge pages, we should revise some of the basic concepts about Operating system memory management.

Operating systems basically deals with two types of memory for its operations one is Physical memory ( actual Physical RAM) and the second is Virtual memory ( combination of Physical and Swap). Virtual Memory is used to store entire program that is currently running and the physical memory is to store only specific part of memory that is important at this moment. And both physical memory and virtual memory will be segmented as pages, and the process of copying the pages of the program from to/from the virtual memory to/from the physical memory is called paging.

Operating system manages a page table to map virtual memory pages to physical memory pages. So when ever operating system needs to access memory for some information it will look at the paging table first and then look at the corresponding physical memory, if it doesn’t find information in physical memory then go back to virtual memory to get the required information.

Terminology That we use when referring to hugepages:

TLB (Transalation Lookaside Buffer) : It is a buffer ( or cache) in a CPU that contains parts of the page table. This is a fixed size buffer being used to do virtual address translation faster.

hugetlb: This is an entry in the TLB that points to a HugePage (a large/big page larger than regular 4K and predefined in size). HugePages are implemented via hugetlb entries, i.e. we can say that a HugePage is handled by a “hugetlb page entry”.

hugetlbfs: This is a new in-memory filesystem like tmpfs and is presented by 2.6 kernel. Pages allocated on hugetlbfs type filesystem are allocated in HugePages.

How operating system deals with process when there was no concept of Hugepages?

In unix environment every operation/task runs in the form of a process and every process has its own page table which is having entries that references to system wide page table. System wide page table will have complete information about memory pages that are required for the execution of system wide process.

During the process execution, each user process looks at its own local page table, and goes to system wide page table to find the location of the memory that it wants to access. And also sometimes it is possible that some related process will access same memory page in shared mode.

How Hugepages makes difference to regular Memory Management?

HugePages is a feature integrated into the Linux kernel with release 2.6. This feature basically provides the alternative to the 4K page size (16K for IA64) providing bigger pages. There are some basic differences between Huge pages and regular pages, and they are :

a. Huge pages can be allocated on-demand but they must be reserved during the system startup.Otherwise the allocation fails because the memory by default segmented into 4K pages.

b. HuHugePages are not swappable. Therefore there is no page-in/page-out mechanism overhead.HugePages are universally regarded as pinned.

c. Decreased Page Table Overhead: Each entry in page table occupies 64bytes memory , and is we assume we have 256GB of memory which is segmented into 4K memory pages, the pagetable alone takes around 800MB in size and managing such large table is an overhead to the operating system. In other case, if the huge pages are activated then we can reserve major part ( around 90%) of the memory to be accessed as 256MB hugepages so that the number of entries in the system page table reduces and the size of the page table also reduces to 40 to 50MB.

d. Overall memory performance will be improved : On virtual memory systems each memory operation is actually two abstract memory operations. Since there are less number of pages to work on, the possible bottleneck on page table access is clearly avoided

e. Size of the hugepages can vary from 2MB to 256MB depending on the kernel version.

How to find the current size of hugepage?

$ grep Huge /proc/meminfo

HugePages_Total: ...

HugePages_Free: ...

HugePages_Rsvd: ...

Hugepagesize: ..... kB

$