Oracle & Linux Hugepages settings

Question:

Is hugepages still recommended for 64bit Linux + Oracle?

Answer:

Yes!

Why:

Without hugepages, each page is 4kB by default, to maintain a pagetable of 32GB memory addressing, we need 32*1024*1024/4*8/1024/1024=64MB (each page reference point takes 4 bytes in 32bit OS and 8bytes in 64bit OS). For example, we have 100 such processes; we need 6400MB memory to maintain the pagetable only. (But normally we will not hit this worst situation, because of lazy allocation memory algorism)

With hugepages, each page is 2MB on EM64 architecture. To maintain a pagetable of 32GB memory addressing, we only need 32*1024/2*8/1024=128KB. For example, we have 100 such processes, we only need 12.5MB memory for pagetable.

Another reason to configure hugepages is memory associated hugepages are not swappable, a.k.a, always in memory.

One more benefit, hugepages increases the efficiency of TBL.

Another one, "Cached" memory with hugepages does not include SGA any more.

How to set?

[root@VMXDB01 ~]# cat /etc/sysctl.conf
... ...
vm.nr_hugepages = 16640
... ...

Method of calculating vm.nr_hugepages:

Get Hugepagesize from /proc/meminfo, for our case, it's 2MB.

vm.nr_hugepages = sum (all sga_max_target size in MB) / Hugepagesize + min(2,value_you_feel_conformtable)

[root@VMXDB01 ~]# cat /etc/security/limits.conf
... ...
oracle              soft    memlock 33564672   <-- larger than max SGA, in KB
oracle              hard    memlock 33564672   <-- larger than max SGA, in KB
... ...

Verify:

[root@VMXDB01 ~]# cat /proc/meminfo
MemTotal:     64951768 kB
MemFree:      29433020 kB
Buffers:         62304 kB
Cached:         701376 kB
SwapCached:          0 kB
Active:         726660 kB
Inactive:       575368 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     64951768 kB
LowFree:      29433020 kB
SwapTotal:    33551744 kB
SwapFree:     33551744 kB
Dirty:             376 kB
Writeback:           0 kB
AnonPages:      549344 kB
Mapped:         114368 kB
Slab:            63620 kB
PageTables:      23268 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  48988268 kB
Committed_AS:  2529688 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    283056 kB
VmallocChunk: 34359455303 kB
HugePages_Total: 16640
HugePages_Free:  13887
HugePages_Rsvd:  13695
Hugepagesize:     2048 kB