• 2 replies
  • Latest Post - ‏2010-01-22T19:19:35Z by SystemAdmin
238 Posts

Pinned topic CA Virtual Appliance 1500 with large files, long orchestration times

‏2010-01-08T22:15:37Z |

This goes along with my previous post about inspecting a running orchestration

I'm running a devel Virtual Appliance with 2gb Ram and 2x5gz CPU's. I have a web service that CastIron calls with values taken from a CSV File. During normal run time, with a relatively small CSV, my service runs at about 1 row per second.

I have a client's file that is ~19000 lines so I thought I'd give it a try, expecting it to finish in about 5 hours. It's now been running for over 24 hours and I'm seeing major spikes on my server in CPU usage. The CastIron web interface dashboard shows memory usage occasionally hitting 50%, but never any more, and GC Activity occasionally hits 100% (I have no idea what GC Activity means)

My web service is certainly not the bottle neck as it just happily receives a row from CI when it gets it, and returns, so there's something seriously wrong with my appliance

I'm wondering if anyone has experience with something similar to this. It seems a bit ludicrous that a 2gb VM with 2x5ghz CPU cores can't process a 5mb CSV file

I know this set up is for dev use only, which is what i'm doing, but i thought it could at least handle this much load. It's fairly typical of the data we're getting

One thing I've noticed is that if I run top on the server itself, the VMware-vmx process only ever uses about 100mb physical memory and the other 1900mb is virtual. I figured by assigning 2gb to the VM that this would be all physical but that doesn't seem to be the case, unless I'm looking at this wrong

Any tips or hints as to how I can inspect better what's going on and how to debug this activity would be great

Updated on 2010-01-22T19:19:35Z at 2010-01-22T19:19:35Z by SystemAdmin
  • SystemAdmin
    238 Posts

    It may be a design pattern issue

    The only way for us to get to the bottom of this is to look at the design pattern you are using for proessing the file. There are certain patterns that can lead to this behavior, but without seeing the project I can't advise you.
    If you still need help on this, please email your project to Cast Iron support, and I'll have the team pass it on to me.
    Director of Solutions Engineering and Cast Iron guru.
  • SystemAdmin
    238 Posts

    will do


    Thanks for the response, my CastIron duties have been put on hold for the moment but I'll post back with my project when I get a chance.