Background Introduction
An Oracle instance running on a cluster in RHEL is started using the systemd service (to facilitate cluster switching operations). During testing, it was found that standard huge pages were not being utilized. The specific situation is as follows:
$ grep HugePages /proc/meminfo
AnonHugePages: 0 kB
ShmemHugePages: 0 kB
FileHugePages: 0 kB
HugePages_Total: 2034
HugePages_Free: 2034
HugePages_Rsvd: 0
HugePages_Surp: 0
Cause Analysis
As shown above, the value of HugePages_Free is 2034, and the value of HugePages_Total is also 2034, which means that standard huge pages are not being used at all.
The basic information of the Linux server is as follows:
$ more /etc/redhat-release
Red Hat Enterprise Linux release 8.10 (Ootpa)
$ free -m
total used free shared buff/cache available
Mem: 11697 4929 5986 17 780 6612
Swap: 16383 0 16383
The database parameters are checked as follows, which fully meet the conditions:
SQL> select banner from v$version;
BANNER
--------------------------------------------------------------------------------
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
SQL> col name for a20;
SQL> col value for a32;
SQL> select name, value from v$parameter
2 where name in ('memory_target','sga_target','use_large_pages');
NAME VALUE
-------------------- --------------------------------
use_large_pages TRUE
sga_target 4261412864
memory_target 0
SQL>
The kernel parameter vm.nr_hugepages is also correctly set, as shown below:
$ grep vm.nr_hugepages /etc/sysctl.conf
vm.nr_hugepages = 2034
$ ./hugepages_settings.sh
This script is provided by Doc ID 401749.1 from My Oracle Support
(http://support.oracle.com) where it is intended to compute values for
the recommended HugePages/HugeTLB configuration for the current shared
memory segments on Oracle Linux. Before proceeding with the execution please note following:
* For ASM instance, it needs to configure ASMM instead of AMM.
* The 'pga_aggregate_target' is outside the SGA and
you should accommodate this while calculating the overall size.
* In case you changes the DB SGA size,
as the new SGA will not fit in the previous HugePages configuration,
it had better disable the whole HugePages,
start the DB with new SGA size and run the script again.
And make sure that:
* Oracle Database instance(s) are up and running
* Oracle Database 11g Automatic Memory Management (AMM) is not setup
(See Doc ID 749851.1)
* The shared memory segments can be listed by command:
# ipcs -m
Press Enter to proceed...
Recommended setting: vm.nr_hugepages = 2034
The resource limit configuration file limits.conf also has the correct setting for memlock, as shown below:
# grep memlock /etc/security/limits.conf
# - memlock - max locked-in-memory address space (KB)
oracle soft memlock 10485760
oracle hard memlock 10485760
# su - oracle
Last login: Fri Aug 8 13:54:36 CST 2025 on pts/0
$ ulimit -l
10485760
$ grep memlock /etc/security/limits.conf
# - memlock - max locked-in-memory address space (KB)
oracle soft memlock 10485760
oracle hard memlock 10485760
It is puzzling why the configuration is correct, but Oracle does not use standard huge pages. After restarting the Oracle instance, a clue was found in the alert log, as follows:
**********************************************************************
2025-08-08T13:50:16.662256+08:00
Dump of system resources acquired for SHARED GLOBAL AREA (SGA)
2025-08-08T13:50:16.662285+08:00
Domain name: system.slice/bpsdbsvr.service
2025-08-08T13:50:16.662302+08:00
Per process system memlock (soft) limit = 64K
2025-08-08T13:50:16.662318+08:00
Expected per process system memlock (soft) limit to lock
instance MAX SHARED GLOBAL AREA (SGA) into memory: 4066M
2025-08-08T13:50:16.662356+08:00
Available system pagesizes:
4K, 2048K
2025-08-08T13:50:16.662387+08:00
Supported system pagesize(s):
2025-08-08T13:50:16.662404+08:00
PAGESIZE AVAILABLE_PAGES EXPECTED_PAGES ALLOCATED_PAGES ERROR(s)
2025-08-08T13:50:16.662421+08:00
4K Configured 11 1040395 NONE
2025-08-08T13:50:16.662450+08:00
2048K 2034 2033 0 NONE
2025-08-08T13:50:16.662466+08:00
RECOMMENDATION:
2025-08-08T13:50:16.662483+08:00
1. Increase per process memlock (soft) limit to at least 4066MB
to lock 100% of SHARED GLOBAL AREA (SGA) pages into physical memory
2025-08-08T13:50:16.662514+08:00
**********************************************************************
Checking the limits of the Oracle process, it was found that the Max locked memory for the process was 65536, which is 64K.
# oracle_pid=$(pgrep -f "_pmon_")
# cat /proc/$oracle_pid/limits
Limit Soft Limit Hard Limit Units
Max cpu time unlimited unlimited seconds
Max file size unlimited unlimited bytes
Max data size unlimited unlimited bytes
Max stack size 33554432 unlimited bytes
Max core file size 0 unlimited bytes
Max resident set unlimited unlimited bytes
Max processes 46635 46635 processes
Max open files 262144 262144 files
Max locked memory 65536 65536 bytes
Max address space unlimited unlimited bytes
Max file locks unlimited unlimited locks
Max pending signals 46635 46635 signals
Max msgqueue size 819200 819200 bytes
Max nice priority 0 0
Max realtime priority 0 0
Max realtime timeout unlimited unlimited us
This means that when the Oracle instance is started by the systemd service, for some reason, memlock is still 64K, as indicated in the log “Per process system memlock (soft) limit = 64K”
Therefore, starting the database instance manually was used, and it was found that standard huge pages were utilized by Oracle. However, when the Oracle instance is started by the systemd service, the above situation of standard huge pages not being used occurs.
$ grep HugePages /proc/meminfo
AnonHugePages: 0 kB
ShmemHugePages: 0 kB
FileHugePages: 0 kB
HugePages_Total: 2034
HugePages_Free: 4
HugePages_Rsvd: 3
HugePages_Surp: 0
Later, after checking with colleagues, it was found that services started by systemctl do not read the resource limit configuration file (limits.conf) by default. The limits in limits.conf are for user session-level resource control, which takes effect when the user logs in via the PAM module. However, systemd services are started directly by the systemd process, which belongs to a non-login session and does not trigger the PAM’s pam_limits.so module by default. Therefore, all resource limits set for the oracle user in /etc/security/limits.conf will not automatically apply to processes started by the systemd service.
Solution
If you want the memlock limit to take effect when starting the Oracle instance with the systemd service, you can directly configure the memlock limit in the oracle.service or let the service read limits.conf through PAM. Online resources recommend directly configuring in the systemd service file (this is the recommended way by systemd, which is more reliable than relying on limits.conf), as follows:
Original configuration of oracle.service
[Unit]
Description=Oracle Database Service
After=network.target
[Service]
Type=forking
User=oracle
Group=oinstall
ExecStart=/home/oracle/xxxx/ora19c.sh start
ExecStop=/home/oracle/xxxx/ora19c.sh shutdown
StandardOutput=append:/var/log/rhcs_resource_logs/xxx/xxx.log
RemainAfterExit=yes
KillMode=none
[Install]
WantedBy=multi-user.target
Note: The configuration of oracle.service has a bit of confusion, which does not affect everyone’s understanding.
Modified configuration of oracle.service
[Unit]
Description=Oracle Database Service
After=network.target
[Service]
Type=forking
User=oracle
Group=oinstall
ExecStart=/home/oracle/xxxx/ora19c.sh start
ExecStop=/home/oracle/xxxx/ora19c.sh shutdown
StandardOutput=append:/var/log/rhcs_resource_logs/xxx/xxx.log
RemainAfterExit=yes
KillMode=none
# oracle /etc/security/limits.conf
LimitNPROC=16384
LimitNOFILE=65536
LimitSTACK=10485760
LimitMEMLOCK=10737418240
[Install]
WantedBy=multi-user.target
With this configuration, starting the Oracle instance through the systemd service can now properly utilize standard huge pages. The troubleshooting of the issue is perfectly resolved. The root cause is still due to insufficient understanding of Linux’s systemd services.