HugePage Implementation

- Single database instance 

- Mem: 528265812k (528 GB) 

Recommended values for SGA, PGA, SHARED_POOL, DB_CACHE and hugepages

SGA_TARGET=150G

SGA_MAX_SIZE=150G

PGA_AGGREGATE_TARGET=100G

SHARED_POOL_SIZE=50G

DB_CACHE_SIZE=70G

vm.nr_hugepages=77000

STEPS

• Log into DB as SYS user.
• create pfile from spfile;
• Take the backup of pfile;
• Change parameters 

              ALTER SYSTEM SET USE_LARGE_PAGES=ONLY SCOPE=SPFILE; 
                  ALTER SYSTEM SET SGA_TARGET=150G SCOPE=SPFILE; 
                  ALTER SYSTEM SET SGA_MAX_SIZE=150G SCOPE=SPFILE; 
                  ALTER SYSTEM SET PGA_AGGREGATE_TARGET=100G SCOPE=SPFILE; 
                  ALTER SYSTEM SET SHARED_POOL_SIZE=50G SCOPE=SPFILE; 
                  ALTER SYSTEM SET DB_CACHE_SIZE=70G SCOPE=SPFILE; 
                  ALTER SYSTEM SET MEMORY_MAX_TARGET=0 SCOPE=SPFILE; 
                  ALTER SYSTEM SET MEMORY_TARGET=0 SCOPE=SPFILE; 
                  ALTER SYSTEM SET SORT_AREA_SIZE=0 SCOPE=SPFILE;

• create pfile from spfile
• shut immediate;
• As root add below valules in sysctl.conf

              vm.nr_hugepages=77000

                  vm.hugetlb_shm_group=501 (To know the group use fgrep dba /etc/group)  

• As a root sysctl -w vm.max_map_count=262144
• As a root sysctl -p
• reboot server
• Edit pfile and Remove memory_max_target, memory_target and sort_area_size from pfile
• start database with pfile

                startup pfile=/u01/app/oracle/product/11.3.0/db_1/dbs/initXXX.ora

• create spfile from pfile
• shut immediate
• startup

Large Pages used by this instance (#pages) = HugePages_Total – HugePages_Free + HugePages_Rsvd

Ref.

• Huge Page Implementation Note 361468.1
• Large Pages Information in the Alert Log Note: 1392543.1
• Oracle Not Utilizing Hugepages (Doc ID 803238.1)
• Shell Script to Calculate Values Recommended Linux HugePages / HugeTLB Configuration (Doc ID 401749.1)
• USE_LARGE_PAGES To Enable HugePages (Doc ID 1392497.1)
• Maximum SHMMAX values for Linux x86 and x86-64 (Doc ID 567506.1)
• Upon startup of Linux database get ORA-27102: out of memory Linux-X86_64 Error: 28: No space left on device (Doc ID 301830.1)

Why HugePages?

HugePages is a feature integrated into the Linux kernel with release 2.6. This feature basically provides the alternative to the 4K page size (16K for IA64) providing bigger pages.

Using HugePages, the page size is increased to 2MB (configurable to 1G if supported by the hardware), thereby reducing the total number of pages to be managed by the kernel and therefore reducing the amount of memory required to hold the page table in memory.

• HugePages can be allocated on-the-fly but they must be reserved during system startup. Otherwise the allocation might fail as the memory is already paged in 4K mostly.

• HugePage sizes vary from 2MB to 256MB based on kernel version and HW architecture (See related section below.)

• HugePages are not subject to reservation /  release after the system startup unless there is system administrator intervention, basically changing the hugepages configuration (i.e. number of pages available or pool size)

The Size of a HugePage

The size of a single HugePage varies according to:

• Kernel version/linux distribution
• HW Platform

The actual size of the HugePage on a specific system can be checked by:
      $ grep Hugepagesize /proc/meminfo

-->Following command to determine the current HugePage usage. The default HugePage size is 2MB on Oracle Linux 5.x and as you can see from the output below, by default no HugePages are defined.

        $ grep Huge /proc/meminfo

1. Large Pages used by this instance (#pages) = HugePages_Total – HugePages_Free + HugePages_Rsvd
2. (HugePages_Total- HugePages_Free)*2MB will be the approximate size of SGA
3. Disable AMM (Automatic Memory Management)
4. Check the MEMORY_TARGET parameters are not set for the database and SGA_TARGET and PGA_AGGREGATE_TARGET parameters are being used instead.
5. If you are running Oracle 11.2.0.2 or later, you can set the USE_LARGE_PAGES initialization parameter to "only" so the database fails to start if it is not backed by HugePages.

Ref.

• Huge Page Implementation Note 361468.1
• Large Pages Information in the Alert Log Note: 1392543.1
• Oracle Not Utilizing Hugepages (Doc ID 803238.1)
• Shell Script to Calculate Values Recommended Linux HugePages / HugeTLB Configuration (Doc ID 401749.1)
• USE_LARGE_PAGES To Enable HugePages (Doc ID 1392497.1)
• Maximum SHMMAX values for Linux x86 and x86-64 (Doc ID 567506.1)
• Upon startup of Linux database get ORA-27102: out of memory Linux-X86_64 Error: 28: No space left on device (Doc ID 301830.1)