Oracle E-Business Suite R12 | SGA vs PGA vs NOT being a DBA!

May 7, 2018

Hi guys

I know the subject title is not very professional this time. Yet, I want to make a claim that I figured out something, for which I spent more than couple of years time and have followed up few Oracle community threads (without much interesting results)

We had to retired a hardware that was recommended by the Oracle EBS implementation partner, within 2 years once after we went online with the R12 instance. We had 10g 10.2.0.3 with the instance, things were getting messy and slow & the new support partner recommended for a better hardware.

I always had eyes on this retired server. It had Linux, hence we couldn’t come up with a practical requirement to integrate the Linux server with our Windows domain environment and it was kept switched off until the virtualization project came online.

We needed “something” to hold a copy of the EBS instance while it was being virtualized.

So, I cloned this machine & before continuing let me describe what this is hardware is like:

Processor:


Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 8
On-line CPU(s) list: 0-7
Thread(s) per core: 1
Core(s) per socket: 4
Socket(s): 2
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 23
Model name: Intel(R) Xeon(R) CPU E5420 @ 2.50GHz
Stepping: 6
CPU MHz: 1992.000
BogoMIPS: 4987.29
Virtualization: VT-x
L1d cache: 32K
L1i cache: 32K
L2 cache: 6144K
NUMA node0 CPU(s): 0-7

Storage space


[root@erp-bak ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sdj2 59G 12G 44G 22% /
tmpfs 9.7G 3.9M 9.7G 1% /dev/shm
/dev/sdj1 2.0G 325M 1.5G 18% /boot
/dev/sdj5 738G 531G 170G 76% /u01
/dev/mapper/mpathbp1 591G 320G 242G 57% /u02
/dev/mapper/mpathcp1 197G 38G 150G 20% /u03
/dev/sdf1 591G 70M 561G 1% /u04
/dev/sdi1 269G 59M 256G 1% /u05

in addition to the local disks this server have partitions mounted from a IBM SAN.

Once the clone was done, I realized that the instance was extremely slow & our part time DBA started making excuses like “See that’s why we are changing the hardware” (He had 2G SGA and 1G PGA with 20 job_queue_processes against nearly 1TB database)

I opened few discussions with Oracle communities and was pointed towards a tone of documents suggesting me how to fine tune the hardware and database for better performances. Actually nothing were applicable as I didn’t have much hands on experiences with a database & I couldn’t find a person who could really HELP me.

Then I started taking interest about database technology, which I should have years back & came across SGA/PGA and JVM etc & as I had an idle instance, started trying out whatever I have “learned” against it.

While doing the 11g R2 the hard way I realized that I can use AMM and forget about tuning different parameters for memory optimization. Well, still the goddamn instance lagged like hell & I was almost done with it!.

Few of the persistent issues were:

After a cold boot

  • The login form would load at client end after waiting almost 3-4 minutes, which gets faster during consecutive attempts.
  • It takes ages for to open the concurrent programs window
  • Our custom forms & LOVs lag to extremes and so on..

Even shutting down the instance for anything was turning into a nightmare as the database always took more than 15-20 minutes and I had to kill multiple processes manually in order to bring it offline!

Then on a different note, while trying to learn SQL learning I landed against an ask Tom thread, where the asker says “I have setup both SGA and PGA 3GB”, still the SQL runs slow…

I did a fresh clone. Our database was upgraded to 11g almost year back. The default clone had 1G for both SGA and PGA. I altered them with 3G and 3G & bullseye

I went back and altered the SGA and PGA with 4G which was 40% of the total physical memory available for the hardware. I did three shutdowns and restarts of the physical server, did a dozen application and database startup to confirm that what I am experiencing is NOT a once in bluemoon phenomena. Each of my attempt to shutdown the database gracefully were completed within few seconds, not a single time I had to kill the Linux processes to bring it down!

I modified one of the main forms for a custom application and changed few VIEW calls with better logic & I can’t be happier!

Now, said that, don’t rush to me saying “I also did 4G for SGA and PGA and moron I still have a slow instance”. There are many factors affecting the performance of your database and application & most important few are:

Age of your hardware, especially the spinning disks. The aged they are, the worse the performance is going to be as there is hell loads of I/O happens when you are accessing/processing the data from a database.

Recently I was going through a MS SQL discussion about Multi-Tenant architecture and one of the contributors were discussing about a hosting firm that keeps on changing their hardware once in 6 months. I think he was just BLUFFING! ;)

I hope someone gets benefitted by the minor finding I have made YESTERDAY (6th May 2018)!

regards,

rajesh

 

 

Advertisements

Oracle Application R12(12.0.x) | JRE | UNKNOWN Publisher

January 19, 2018

Hi guys

This time I am going to post a solution towards one of the nagging annoyances with Oracle application R12 (12.0.6 in our specific case) & JRE securities those were implemented by JRE 7 (1.7.x)

Our instance was upgraded to use JRE 6 almost 5 years back during 12.0.4 to 12.0.6 upgrade and ever since there were dozens of JRE updates & the current JRE 1.8.0_161 (AS on 19th January 2018) & with each update, whatever the runtime parameters we add specifically for Oracle Application used to get removed, forcing us to renter it for each user.

I’ve been searching for a proper solution, not only towards the runtime parameter fix, for the security prompts those started annoying our end users whenever they tried to launch the forms, once after we forcefully updated many users who were sticking to 1.6 JRE

So after referring many documents, using some of the past experiences dealing with security certificates, I believe, have figured out how to get rid of those annoyances, minimum for a while.

Solution 1

Runtime parameters being removed after each JRE update

jre

Add a new environment JAVA_TOOL_OPTIONS with a value -Djava.vendor=”Sun Microsystems Inc.”

You may add it under your specific user or system wide, which I usually prefer so that I don’t have to worry about the next user who will be using the same system (for some reasons)

ENV

Solution 2

Addressing JAVA security prompts

Trying to open the forms with latest JRE without adding exceptions will produce the following JAVA security error

1.png

Now we need to add the application URL with complete port details to the security tab of JRE

2

Now, we will be presented with the “toughest” situation to resolve, which will be explained later.

3

Clicking the More Information, JAVA presents more details about the digital certificate issues, stating the publisher of the digital certificate as UNKNOWN, hence this warning appears for the end user each time whenever the user tries to open a Forms based module.

4

You may require Domain Administrator/Administrator privileges to continue the below fixes

1st we have to retrieve the certificate from Oracle application server. Login as application manager user to LINUX environment (We’re setting up the entire stuffs for R12 application that is running on 64Bit Linux)

download $APPL_TOP/admin/appltop.cer file to local computer (If your implementation team has cloned the instances across, the appltop.cer will remain the same across different servers, so the single certificate integration with the user computer should deal with all different instances of the Oracle application (Test, Development & Production)

Once the certificate is downloaded or copied to the local computer, you need to install it. We’ll install the certificate twice, in two different locations.

Follow the images below for more details

5

We will install the certificate for the “Local Machine”, so that all users using the same computer will get benefitted

6

For the 1st instance, We MUST install this certificate to “Trusted Root Certification Authorities” Container.

For 78

 

9

Once the certificate is imported, We should install the certificate once again, Select “Local System” and follow the prompts. We don’t have to specify the container this time.

10

Now, trying to open a forms based module will present you another warning

11

Checking “Do not show this again for apps from the publisher and location above” will suppress this dialog until the certificate remains the same for the application server.

Hope this helps few out there!

regards,

rajesh

 

 


Oracle Application R12 | HTML Page load issues with Internet Explorer 11 | Windows 10

January 7, 2018

Hi guys

Our Oracle application R12 (12.0.6) instance has JDK 6 environment & We don’t have any plans to migrate to any recent versions of Oracle Application those are supported by later Java environments.

Almost all users those use Oracle applications are using JRE for many other engineering programs (Calculations programs by Vendors) & frequent updates of the JRE happens automatically at the user ends. We have a significant crowd that is still using JRE 1.6_20 as base JRE, who seldom have the following issue(s) discussed below.

Over Windows 10, Internet Explorer 11 fails to load HTML pages properly & many times after the logon to the application, the HTML menu page is loaded in the explorer, leaving a 4-5 white space lines. Refer the image below

halfpage

Basically this issue is observed with boxes those have latest JRE (x86) installed and used with parameter -Djava.vendor=”Sun Microsystems Inc.”

image

Oracle blogs strictly says, Compatibility view for Oracle applications R12 shouldn’t be enabled for Internet Explorer 11. I have tried enabling the Compatibility view for the Oracle applications, without any positive changes.

As a last resort, I reset the Internet Explorer 11

image

and after restarting the computer, Oracle Application HTML pages started loading correctly, without leaving white spaces between the IE Window and application top frame.

Hope this helps few folks out there!

rajesh


Oracle EBS R12(12.0.x) | Disable concurrent programs executing immediately after a clone

November 28, 2017

Hi guys

One of the annoyances suffered by business is the alerts and concurrent programs those start sending highly confusing messages to users immediately after a cloning completion. Application tier cloning finishes by trying to start all related services after a successful cloning. Hence the best method to stop the concurrent manager/alerts from firing up is NOT to let the cloning process start application tier level services, so that you can disable alerts/concurrent programs those are NOT required by the cloned instance.

First step is to alter the $COMMON_TOP/clone/bin/adcfgclone.pl (perl script, please refer the image)

image

Change the portion in the highlighted block as shown below. This will stop the clone script from kick starting the application services after a successful clone.

elsif (!($PLATFORM eq "win32")) {
print "\n Not Starting application Services for $s_dbSid:\n";
#bugfix:5838814 subgovin
#runPipedCmd("$s_config_home/admin/scripts/adstrtal.$scrExt -nopromptmsg", "$s_apps_user", "$gConfig{'appspass'}");
}

Now, you may proceed to disable all or selected alerts/concurrent programs using backend before starting the Application services.

Please note, this workaround is ONLY tested on 12.0.x releases of Oracle Applications. Please adjust to your environments.

References:

http://onlineappsdba.blogspot.com/2008/06/prevent-concurrent-requests-executing.html

http://oracle-latest-technology.blogspot.com/2013/11/how-to-disable-all-scheduled-concurrent.html

regards,

rajesh


Oracle Applications R12 | error while loading shared libraries: libig.so.0…

December 11, 2016

 

applprod@erp-prodbak.my.home:/u01/applprod/PROD/apps/apps_st/appl/au/12.0.0/forms/US>frmcmp_batch module=$AU_TOP/forms/US/XXITMDTLS.fmb userid=apps/apps output_file=$ONT_TOP/forms/US/XXITMDTLS.fmx Module_Type=FORM compile_all=Special
frmcmp_batch: error while loading shared libraries: libig.so.0: cannot open shared object file: No such file or directory
applprod@erp-prodbak.my.home:/u01/applprod/PROD/apps/apps_st/appl/au/12.0.0/forms/US>

Recently I cloned our Production instance to check the feasibility of OEL 6.7 & as usual securely copied most of the bash profile for the Oracle and Application manager users from the production server.

I came across the “cannot open shared object file: No such file or directory” error while trying to compile an old form module & hadn’t have a clue for how to deal with this error, that happened to me first time in last 8 years with Oracle applications.

I crosschecked the folder permissions against production instance and everything was fine. All I could remember which could have gone wrong was narrowed down to the bash profile for the application user and after validating I realized that I was sourcing a wrong environment file!

Instead of sourcing the “/apps/apps_st/appl/APPSPRODBAK_hostname.env” file, I was sourcing “/apps/apps_st/appl/PRODBAK_hostname.env”. I realized it while trying to echo the $ORACLE_HOME variable as application manager user.

So, if you come across “error while loading shared libraries” error while trying to compile, before start dismantling your instance, try to see whether you have sourced the environment variables correctly.

regards,

rajesh


Java runtime 1.8.0.91 | Oracle EBS R12 (12.0.6)

May 2, 2016

Hi guys

You could run into a situation after the latest Java client runtime update, your R12 instance failing to load forms, complaining about the lower version of JRE. This is mainly due to the fact that, if your Windows OS is 64Bit, the latest java runtime update installs both 64Bit & 32Bit and most of the R12 environments run against 32Bit JRE.

Internet explorer (the only one browser that runs the JRE properly for EBS R12) loads the 64Bit java runtime by default from a 64Bit Windows OS & eventually the forms will fail to load.

At present the only possible solution to address this situation is to uninstall the 64bit version of java runtime, using “Programs and features” or “Add remove programs” based on the version of Windows you are using.

Hope this helps few out there :)

regards,

rajesh

 

 


R12 | Clone response file | adcfgclone automation

December 8, 2015

Hi guys

If you are familiar with R12 cloning, the entire process requires loads of input from a DBA before the cloning really starts kicking in. Now, there could be hundreds of reasons for the cloning process to fail and keeping on entering the responses for the cloning prompts are NOT at all fun, minimum for me.

So I started searching for “R12 clone response file” & google landed me in this post

https://community.oracle.com/thread/2142127?start=0&tstart=0

Excellent way Hussein Sawwan-Oracle has explained how to create the response files for both db, apps tier.

 

clone

First of all, context file to response file creation has a validation flag, which confirms the responses provided are valid throughout the response collection. This validate has a downside with the port pool selection. You will receive a port is not free error while the Oracle application is online, for which a system reboot is most recommended (rather than selecting “No” for the validation process”) after graceful shutdown of both application and database tiers.

I’m using .bash_profile files to source the environment for both my Oracle (oraprod) and Application Manager (applprod) accounts, hence I was able to shorten the efforts like following

As Oracle user (oraprod)

# cd $ORACLE_HOME/appsutil/clone/bin
# perl adclonectx.pl contextfile=$CONTEXT_FILE outfile=/u02/db.xml

Provided the responses and the db.xml was successfully created under /u02 and the context file creation log was created under /u02/log folder.

Please make sure you do “echo $CONTEXT_FILE” to check whether you $CONTEXT_FILE returns a meaningful file name!

As Application Manager user (applprod)

# cd $COMMON_TOP/clone/bin
# perl adclonectx.pl contextfile=$CONTEXT_FILE outfile=/u02/appl.xml

Now you should able to run the cloning processes using the newly cloned context files, for example

# perl adcfgclone.pl dbTier /u02/db.xml
# perl adcfgclone.pl appsTier /u02/appl.xml

That’s all folks, it works like a charm & you are NOT anymore typing in whole those lengthy paths and other variables :)

regards,

rajesh