Announcement

**nick.polites** · March 28, 2008, 13:59

Re: OTM 5.5 Hardware

To get the best bang for the $$, you'll need to use the Intel platform which means your choice will be either Windows or Linux.

On a split stack I would recommend 8GB for app, web/rpt servers
Databse should have 16GB ram.

If you are going to use all in one for Test/Dev make sure that these servers have 16GB of ram.

You will find that OTM is a memory consuming application. JVMs start at 1.5 GB and can be configured to 2.4 GB which can grow up to 4 GB +

Running Solaris is an option but you will find it slow especially in a production environment unless you get the latest and greatest but it will be slower than an Intel server.

**GSW** · March 28, 2008, 18:03

Re: OTM 5.5 Hardware

Hi Nick, Many thanks for the update, we will be using the new CoolThreads servers from Sun, very fast, lots of memory and cheap... I'm glad that we can use a single server for DB/reports for all 3 dev environments and another server for all 3 dev apps/web instances. There will be 16Gb on each of the servers. We will be using a separate server for each of the 4 Prod services. Thanks for the JVM advice, I always create another JVM whenever the number of connected users > 100 or active users > 33 (Does this still apply for OTM?) Thanks Again Gary.

**nick.polites** · March 28, 2008, 18:13

Re: OTM 5.5 Hardware

Just a warning on having the database and reports on the same server. You're going to have conflict of ports when installing reports after the database is installed. You are going to have to change a few configuration ports as the installer doesn't know what ports are open. I would personally put leave the database as a standalone and put the reports either with the app or web tier.

The JVM is initialized as one user and does not need to be modified once the application is up and running. If you tweak any parameters, it will require a restart of the application.

**chrisplough** · March 28, 2008, 18:18

Re: OTM 5.5 Hardware

Not to be the bearer of bad news, but we've seen customers use the Sun CoolThreads servers and each had performance issues (Sam actually used to work for one of them). While the servers tend to do well at low-computational, high thread applications, OTM is highly intensive and benefits from strong, number-crunching CPU cores.

I'd recommend running the VolanoMark and DaCapo benchmarks using the Sun 1.4.2_12 JDK to better understand the performance limitations of this platform:

--Chris

**GSW** · March 29, 2008, 09:56

Re: OTM 5.5 Hardware

Hi Chris, Many thanks for the warning, we have a T5220 Coolthreads server that we use as a combined database and apps/web server for another system and the performance has been great. It's especially good that you can create multiple logical domains for Dev/Test etc. Looking at all the official benchmarks, these servers seem to really excel (esp for multi-threaded applications such as Java and Oracle). We plan to use T5220 servers. I am aware that large single threaded batch jobs are slower, but I have been told that OTM doesn't really have large sequential jobs, instead, it is multi-threaded which should especially suit the CoolThreads servers. The manuals state each production server (DB/Reports/Apps/Web) needs 2*1.35Ghz UltraSparc IV processors. We have been told by Sun that a single T5220 core (1.2Ghz) is equal to 1.5*Ultrasparc IV 1.35Ghz CPU's. So if we assign 2 Cores to each server (apps/web/reports) then each will be getting the equivalent of 3*1.35Ghz US CPU's. Googling real life experiences of coollthreads performance, they look great and blow everything else away:- please see :- milek's blog: T2000 real web performance All Published SPEC web2005 Results /~colmmacc/ » Blog Archive » Niagara vs ftp.heanet.ie Showdown https://www.oracle.com/apps_benchmar...On_Solaris.pdf I see that the Siebel CRM benchmark recently performed by Oracle (using T5220/T5120 servers) managed to serve 10,000 concurrent users. Oracle state, "..provides the highest performing and the most cost-effective business solution." I would be very interested to learn about specific bad experiences regarding the coolthreads servers (servers used and actual problems) as we are serious about using these servers (Guess we can get them on the free 60 day trial and actually try them within a OTM environment!). Thanks Again for the comments.

**chrisplough** · March 29, 2008, 14:21

Re: OTM 5.5 Hardware

Nice - thanks for the great information and benchmarks!

Please don't get me wrong, I'm not attacking the hardware choice (always a good way to start out an email - ha!); and I actually enjoy the back and forth!

There may be a flaw in one of the primary assumptions, "I have been told that OTM doesn't really have large sequential jobs, instead, it is multi-threaded which should especially suit the CoolThreads servers". I say may, because it depends on your usage of OTM. It is true that it is highly multi-threaded (very highly - it starts out with 150+ threads on the app tier). The problem is that most of the processes that businesses rely upon are actually very intensive single-threaded processes -- such as the planning logic (especially bulk plans), saved queries, many Agent actions, reports, etc. So OTM actually presents the hardware with a tough mix of both many, many threads and very intensive single-threaded algorithmic calculations. In fact, this per-transaction latency (i.e. how fast a single-threaded process completes) is the biggest cause for OTM performance issues and complaints. A higher number of cores simply provides scalability, which is also very important, but fundamentally different.

Also all current OTM versions utilize the 1.4.2 JDK - which doesn't scale well on multi cpu/core hardware; with the exception of JRockit on the Intel platforms. OTM v6.0 will support the 1.5.x JDK, but that won't be generally available for another year.

I'm currently working with 3 of the OTM clients who have been using the application for years and who now have (or will soon have - depending on the client) the app's highest volumes. On one of these projects, we worked with Sun and were able to test against the servers you've mentioned, as well as a M9000 server with the new soon-to-be-released Jupiter-class CPUs. I'm not at liberty to give actual results, but I can say the they were disappointing. I have higher hopes for the Jupiter+ CPUs on the far horizon (as they scale further into the GHz range); but I believe the root cause is the Sun JDK -- which isn't nearly as efficient as JRockit. F

Finally, as part of this analysis, we found that the per-core performance on all tiers was severely lacking. Financially, this doesn't matter on the OTM Web and App tiers (no per-core licensing), but was a killer on the DB server, with a final Oracle license cost several times the comparative Intel/Linux combination.

Now - this may not be an issue, depending on your projected volumes and usage. That's one part of your solution that I don't have any insight into.

The reason I sharing this information is just so you're better prepared. I highly recommend the VolanoMark benchmark mentioned earlier, as it compares well to general OTM performance (highly multi-threaded) and the DaCapo benchmark, as it compares well to bulk plans and other high-calculation, low thread processes, and finally the Hammerora benchmark, which replicates TPC-C and gives a great indication of DB performance.

Run the tests for yourself and then compare to the results we've posted. As a note, we'll have a new server coming in about 1-2 weeks in preparation for a Hosting client that has a total of 16 Intel cores. I'll post the results when I have them, but from years of experience, I'm expecting VolanoMark results between 394232 and 417889, DaCapo results around 12632.00 and Hammerora results around 205.00.

Please let me know your thoughts and your results if you run the benchmarks!

BTW - looking at your volume numbers, I would consider you a high-volume client. Again, get the hardware (you can't beat a 60-day trial) and benchmark away. Please share your results, as I'm always interested in other experiences -- it helps me stay up-to-date.

--Chris

**GSW** · March 30, 2008, 18:40

Re: OTM 5.5 Hardware

Hi Chris, Many thanks for the great information. I think we'll have to do the Sun free 60 day server trial and try it out.

In regard to the system, it is for 847,000 consigment ID's/year with an average of 2.5 jobs/consignment. The No of transactions (inbound and outbound) is 4.1M/year. The number of truck plans in OTM is 352,000

Would you consider this to be a high-transactional system ?

Regarding Oracle licenses, the T5 gets a discounted factor which actually makes it more beneficial than typical SMP servers.

Thanks again for your advice.

**GSW** · March 31, 2008, 09:15

Re: OTM 5.5 Hardware

Hi Chris,

Sorry to keep hassling you, but looking back, you said you had disappointing results from a M9000 (this is a huge beast), If performance problems were encountered running on this, then I think we have no chance...!

The OTM Admin Guide states that the recommended hardware for production is only 2*1.35Ghz UltraSPARC IV CPU's per each of the 4 servers).
Is this realistic only for small or medium systems ?

Doesn't the JVM's on the Web and Apps tiers always use multi-threading so should be able to take advantage of the many threads available.

Thanks Again...

**chrisplough** · March 31, 2008, 09:38

Re: OTM 5.5 Hardware

No hassle at all. We often get called into clients who have implemented OTM and are having performance issues. I'd rather tackle the problem up front by giving advice, than to come into a heated environment where hundreds of thousands (or millions, in the case of the M9000) have been spent.

First, on the OTM Admin guide -- these specs haven't been updated in several OTM versions (since possibly v4.5) and represent the minimum specs to run OTM. Also, keep in mind that the performance per platform differs greatly, as it does with any application. In the case of Java applications, Intel/Linux is currently the best performing platform and offers the benefit of being one of the most cost effective.

The largest problem that your hitting up against is the poor scalability and performance of any 1.4.x JDK/JVM, except for JRockit -- and JRockit is only supported on Intel platforms. I choose Linux because it's easier to manage and it allows me to allocate a larger Java memory heap than Win32 does. As for the M9000, with current CPUs it doesn't perform very well. With the soon-to-be-released Jupiter class CPUs, it performs well (but not as well as Intel/Linux) up to 4 cores -- after that the Sun JVM fails to scale and 64-CPUs doesn't provide much better performance.

Also, to an earlier comment about the pricing discount for the CoolThreads CPUs -- while this does help, it is unfortunately overcome by the poor PPC (performance per core) that this architecture provides. Now - if I was running an NFS server or a traditional web server (many low weight threads) then I'd pick the T5. I'm not against this architecture, just in choosing the right tools for the job at hand.

With OTM, it is highly multi-threaded, but each of those threads are of significant weight (i.e. highly computational in nature), so it needs a platform that both provides incredible number crunching, and the ability to handle multiple threads at once. Currently, the best platform for this is Intel/Linux utilizing the new high-GHz, multi-core CPUs. Also - realize that a lot of the performance advantage is due to the JRockit JDK, which is much faster and scales better than any other current JDK (currently proven to scale well up to at least 16 cpus/cores).

I have a ton of experience with this (I've been performance testing OTM since the beginning of the product), but I don't ask you to take my word for it. Run the benchmarks and come to your own conclusion

--Chris

**GSW** · March 31, 2008, 10:59

Re: OTM 5.5 Hardware

Hi Chris,
Thanks again, I guess we have the choice of using the 60 day trial for the T5220
or consider a number of Sun Intel servers. Can we not run Solaris 10 on Intel servers as opposed to Linux (our own experience has shown Solaris 10 to be faster and more stable/secure than Linux)?

We normally use Sun hardware, so would therefore be looking at something like the SunFire X4600 (4 AMD Opteron Model 8218 Processors OR
8 * AMD Opteron Model 8220 processors).

Without server virtualisation, we would need something like :-
Server-1 DEV/TEST/TRG DB/Reports (Opteron 4 * dual-core ,16Gb)
Server-2 DEV/TEST/TRG Apps/Web (Opteron 4 * dual-core, 16Gb)

Server-3 PROD DB (Opteron 4 or 8 * dual-core ,16Gb)
Server-4 PROD Reports (Opteron 4 or 8 * dual-core ,16Gb)
Server-5 PROD Apps (Opteron 4 or 8 * dual-core ,16Gb)
Server-6 PROD Web (Opteron 4 or 8 * dual-core ,16Gb)

Do you think this is a better proposition ?

Thanks again for your help...

**nick.polites** · March 31, 2008, 11:25

Re: OTM 5.5 Hardware

One thing I will chime in with is that I was never a fan of the Sun/AMD x4200 servers. Oracle On-Demand was using these and I found on random servers that they held a load of 3 for no reason. The best that I could find on the web (as I didn't have root access) was that it had to do something with Redhat 4 and Sun's Virtual drivers in the Bios. I would recommend anything but these servers, besides the new Intel Xeons blow away anything that AMD has.

Intel Xeon and AMD Opteron Battle Head to Head | Tom's Hardware

**GSW** · March 31, 2008, 12:13

Re: OTM 5.5 Hardware

Chris, We will be looking at the Sun Fire X4600 (8 Chip's 16 cores). We really need to use a Sun servers so we can use Solaris 10 (for the reasons stated). Solaris10 will allow us to use containers, so we can segregate DEV/TEST and PROD etc.

Does the server config in the last post look OK ?

**nick.polites** · March 31, 2008, 12:39

Re: OTM 5.5 Hardware

Just so you are aware OTM does not work on Solaris x86.

**chrisplough** · March 31, 2008, 12:42

Re: OTM 5.5 Hardware

Just want to consolidate responses to the last couple of posts (I'm also in the airport, so please excuse the brevity):

I like Solaris 10 on Intel as an OS. The container technology is quite good and works well. Unfortunately, you won't be able to leverage it on Intel for 2 reasons:
- OTM isn't supported on it.
- You won't be able to leverage JRockit - which is a large part of the Intel/Linux performance advantage.
Should you choose to use virtualization of any type, I would recommend it's use in DEV and TEST, but not in PROD. With high volumes, you'll want as much direct access to the hardware as possible (this applies on all platforms).
I haven't had any personal experience with the Sun AMD hardware, but trust Nick's opinion. I've worked with him for many years back at G-Log and he leads up our Hosting team. He experienced many issues (as he alluded to) while working at Oracle, and in-fact they've begun moving away from it.
From my standpoint, it doesn't matter if you leverage Dell, HP or IBM hardware - it's all roughly the same and more dependent on support contracts, existing relationships, etc.

I realize a hardware and/or OS change may be difficult, if not impossible for your organization. In the end, the choice is yours -- I just want to ensure you have the best information to make your decision.

For TEST/DEV, hardware comparable to what you've spec'd is fine. However, for PROD hardware sizing, there is a lot that goes into it (we have a whole worksheet and Q&A process that we utilize with our clients). A lot depends not only on volumes, but on what OTM functionality you plan to utilize and details like the data retention period, etc. I wouldn't be able provide you with an accurate sizing without going through it -- sorry.

Hope this helps!
--Chris

Announcement

OTM 5.5 Hardware

OTM 5.5 Hardware

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment