Scalability configuration in OTM

[pritam]

Hi,
Has any one ventured onto the scalability configuration in OTM. as per the brief documentation (in admin guide) it looks to be very advanced feature (e.g. having ability to redirect even at function / domain level etc...)

I am trying to segregate the bulk planning process to a separate server, so that it does not become bottleneck for other transactions. So far no luck. Does any one know, which all "application functions" are involved in moving the bulk planning business process?

Any other pointers / experiences in OTM scalability are welcome.

Thanks,
- Pritam.

[chrisplough]

Re: Scalability configuration in OTM

Pritam,

The SCA (scalability) feature in OTM is very advanced. It's been in the product since around v4.0, but for various reasons, has not really been production tested until recently. Currently, I'm only aware of a couple production clients using it, primarily in domain-split SCA mode. Currently no one is utilizing the functional-split SCA to my knowledge, but I do know of a few companies that are considering it. The biggest problem you'll it (as you identified) is splitting out logical functions into the those defined within the Admin UI, so that you can assign them. Currently, there's no definitive resource for this -- it all comes down to experience and understanding how the underlying functional architecture fits together. As for the full active/active clustering -- it currently doesn't perform well, but it expected to improve in v6.0.

If you're experimenting with SCA, then make sure you're at least at the OTM V5.5 CU4 patch level -- as there were significant improvements within this patch. This is also the base patch that Oracle will require in order to support your efforts.

As for the documentation -- you're right, it's currently poor and hasn't really been updated since v4.0. There is a workshop in April that Oracle is holding, in order to discuss all of the features of SCA, gather real-world experience and improve their documentation. I don't know how long it will take to get deliverables out of this workshop, but do know that we will be involved in the process.

I can share more details as they become available. In the meantime, if you have specific questions, post them and we can help. I've setup SCA a lot of times, both in QA environments at G-Log and for customers experimenting with it since.

--Chris

[pritam]

Re: Scalability configuration in OTM

Thanks Chris,
We are testing this with 5.5 CU4. There are few issues that I am facing in configuration.
My configuration is - 1 web server process (i.e. 1 apache and 1 tomcat) 2 app server processes (server 1 and server 2)
In application servcers, I have configured 2 clusters each having 1 app server. When I start this configuration, everything starts well, but I get JMS erros in the logs of one application server (seen in console logs of server which was started later) as below -
2008-04-01 16:13:27.172 0 Error Exception weblogic.jms.common.JMSException: Connection not found

weblogic.jms.common.JMSException: Connection not found
at weblogic.jms.dispatcher.InvocableManager.invocable Find(InvocableManager.java:136)
at weblogic.jms.dispatcher.Request.wrappedFiniteState Machine(Request.java:630)
at weblogic.jms.dispatcher.DispatcherImpl.dispatchSyn c(DispatcherImpl.java:179)
at weblogic.jms.client.JMSConnection.createSessionInt ernal(JMSConnection.java:410)
at weblogic.jms.client.JMSConnection.createTopicSessi on(JMSConnection.java:369)
at glog.util.jms.GC3TopicConnection.createTopicSessio n(GC3TopicConnection.java:13)
at glog.util.message.TopicSubscriptions.subscribeForA pp(TopicSubscriptions.java:161)
at glog.util.message.TopicSubscriptions.loadForApp(To picSubscriptions.java:13
at glog.util.remote.NamingDirectory.reloadAppServerMa p(NamingDirectory.java:110
at glog.util.remote.NamingDirectory.getAppServerMapEx (NamingDirectory.java:1075)
at glog.util.remote.NamingDirectory.routeServerByDoma in(NamingDirectory.java:792)
at glog.util.remote.NamingDirectory.<init>(NamingDire ctory.java:706)
at glog.util.remote.NamingDirectory.get(NamingDirecto ry.java:110)
at glog.util.remote.NamingDirectory.get(NamingDirecto ry.java:121)
at glog.util.remote.SessionBeanHomeInvoker.<init>(Ses sionBeanHomeInvoker.java:26)
at glog.util.remote.NamingDirectory.get(NamingDirecto ry.java:165)
at glog.util.remote.NamingDirectory.get(NamingDirecto ry.java:220)
at glog.util.remote.NamingDirectory.get(NamingDirecto ry.java:236)
at glog.util.remote.NamingDirectory.impersonateSecuri ty(NamingDirectory.java:360)
at glog.server.workflow.WorkflowManager.execute(Workf lowManager.java:323)
at glog.server.workflow.Trigger.trigger(Trigger.java: 123)
at glog.util.event.EventHandler.triggerEvent(EventHan dler.java:22
at glog.server.workflow.TopicManager.execute(TopicMan ager.java:203)
at glog.server.workflow.ServerPublisher.execute(Serve rPublisher.java:152)
at glog.server.workflow.ServerPublisher.execute(Serve rPublisher.java:159)
at glog.server.workflow.Topic.execute(Topic.java:169)
at glog.server.workflow.RecalcDataObjectSync.execute( RecalcDataObjectSync.java:36)
at glog.server.workflow.RecalcDataObjectSync.afterCom pletion(RecalcDataObjectSync.java:43)
at glog.util.transaction.GLTransactionHelper$GLSync.a fterCompletion(GLTransactionHelper.java:231)
at weblogic.transaction.internal.ServerSCInfo.callAft erCompletions(ServerSCInfo.java:853)
at weblogic.transaction.internal.ServerTransactionImp l.callAfterCompletions(ServerTransactionImpl.java: 2789)
at weblogic.transaction.internal.ServerTransactionImp l.afterCommittedStateHousekeeping(ServerTransactio nImpl.java:2687)

I have followed the steps as per given in admin guide. Any idea if I am missing particular step?

Thanks,
- Pritam.

[chrisplough]

Re: Scalability configuration in OTM

Pritam,

The error is a bit cryptic, but I would guess that it stems from a configuration issue with the glog.properties files. Have you edited these on all 3 servers and made appropriate changes? Have you installed each of the app servers from scratch, so that they each have unique server names (rather than copying)? If you'd like to post the glog.properties files from each of your servers, I can review and provide feedback.

Thanks,
Chris

[pritam]

Re: Scalability configuration in OTM

Thanks Chris,
I have been able to resolve this error. This issue happened because I installed 2 instances of OTM app server on the same machine. While installing OTM generates the weblogic server name and it's 'gc3-<machine host name>' - this was the issue.

I manually changed the weblogic server name in all required places (config.xml, weblogic.conf etc...) and now it's working properly. (i mean only about the exception error)

I am still not able to offload the bulk plan functions properly.

Thanks,
- Pritam.

[chrisplough]

Re: Scalability configuration in OTM

Pritam,

Glad to hear you got past it. In the future, it can help to setup separate virtual IP addresses for each OTM instance on a server, and then assign each a unique hostname, even as simple as servername-a and servername-b. This will avoid such issues.

--Chris

[pritam]

Re: Scalability configuration in OTM

Thanks Chris.
I had thought about it but decided not to try that as our Weblogic licenses are bound to CPU and the specific ip address. I don't know if it will work with new virtual-ip address.

Thanks,
- Pritam.

[chrisplough]

Re: Scalability configuration in OTM

Pritam,

You can use the development copy of WebLogic (freely available from their website) to run test versions of OTM. So, I'd do this to compare the performance of WebLogic vs OAS and get around the IP specific issue.

--Chris

[pritam]

Re: Scalability configuration in OTM

We have finally got the scalability (what we required) working. We wanted to offload bulk plans to a separate dedicated app server. It seems there was a bug in OTM causing locking issues. (Check in metalink, bug id - 7168537 )
Our environment - OTM 5.5 CU04
Things look pretty good after installing the OTM patch. Hope this helps if you are trying for scalability.

Thanks,
- Pritam.