今天数据库后台报错:

Thu Dec 26 02:00:00 2013
Closing scheduler window
Closing Resource Manager plan via scheduler window
Clearing Resource Manager plan via parameter
Thu Dec 26 02:06:36 2013
Process J000 died, see its trace file
kkjcre1p: unable to spawn jobq slave process
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl/trace/orcl_cjq0_21975.trc:
Process J000 died, see its trace file
kkjcre1p: unable to spawn jobq slave process
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl/trace/orcl_cjq0_21975.trc:
Process J000 died, see its trace file
kkjcre1p: unable to spawn jobq slave process
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl/trace/orcl_cjq0_21975.trc:
Process J000 died, see its trace file
kkjcre1p: unable to spawn jobq slave process
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl/trace/orcl_cjq0_21975.trc:
Thu Dec 26 02:06:46 2013
Process J000 died, see its trace file
kkjcre1p: unable to spawn jobq slave process
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl/trace/orcl_cjq0_21975.trc:
Process J000 died, see its trace file
kkjcre1p: unable to spawn jobq slave process
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl/trace/orcl_cjq0_21975.trc:
Process J000 died, see its trace file

 

 

查了下官方文档,说系统资源耗尽完:

 

CAUSE

This is caused by lack of OS resources or incorrect configuration, where more memory is required and Operating system reaches the limits set.

SOLUTION

* For Unix Platforms:

– run the HCVE script found in NOTE.250262.1 which is responsible to check for the OS resources and perform the suggested changes.

* For Windows platforms, 32 bit architecture:

– enable the /3GB switch in the boot.ini file, following the steps mentioned in Note 225349.1 – Address Windowing Extensions (AWE) or VLM on Windows Platforms

 

果然top一看,有一个占用将近8个cpu的进程,该机器一共就8个cpu;然后直接kill掉后,观察一切正常了。

top - 10:12:11 up 23 days, 16:39,  3 users,  load average: 7.99, 7.87, 7.89
Tasks: 296 total,   1 running, 295 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.1%us,  0.1%sy, 99.8%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   8110496k total,  5220400k used,  2890096k free,   135824k buffers
Swap:  8191992k total,    17660k used,  8174332k free,  4289168k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                                   
31308 root      20   0  760m  11m  888 S 798.1  0.1   3801:42 hald-daemon -B -c /etc/hald-daemon.conf                                                                  
24170 oracle    -2   0 3829m  16m  14m S  1.0  0.2   0:09.45 ora_vktm_orcl                                                                                            
24974 oracle    20   0 15168 1356  908 R  0.7  0.0   0:00.23 top