Topic
  • 9 replies
  • Latest Post - ‏2011-01-25T14:08:02Z by geekarticles.blogspot.com
SystemAdmin
SystemAdmin
2105 Posts

Pinned topic Fatal Error: Fork failed: Not enough space

‏2005-08-02T03:10:57Z |
Hi All,

We are using DataStage EE v7.5.1a in HP-UX 11i (11.23) with Oracle 10g (10.1.0.4) on Itanium Environment for a Datawarehousing project. The machine rx7620 has 4 cpu, 4gb RAM, 4gb swap space.

We have a SEQUENCER calling two datastage jobs in parallel. Both these job reads same sequential file and transforms according to business rules.

During execution of this SEQUENCER, we are getting the following error:

"cs}}},0: Fatal Error: Fork failed: Not enough space
Failure during execution of operator logic.
node_node2: Player 15 terminated unexpectedly."

and either one of the configured parallel job (in the SEQUENCER) is aborted.

The config file "default.apt" has two nodes, with "resource diskk" (/opt/app/Ascential/DataStage/Datasets) and "scratch disk" (/opt/app/Ascential/DataStage/Scratch) configured (same location for both the nodes). 2.9GB free space available in these resource and scratch disk locations.

Ulimit for the unix user, in which we installed ds is as below:

$ ulimit -a
time(seconds) unlimited
file(blocks) unlimited
data(kbytes) 4292870144
stack(kbytes) 131072
memory(kbytes) unlimited
coredump(blocks) 4194303

If we change the SEQUENCER to run the ds jobs one after the other (SEQUENTIAL), then, we dont get such error and the job completes successfully.

But, it would be better if we can run parallel to reduce the execution time to half (as both the jobs are reading the same source file and does transformation).

Please share with us, if you have faced any such issue, any resource to be increased like tuning kernel parameters ....

Thanks,
Srini
Updated on 2011-01-25T14:08:02Z at 2011-01-25T14:08:02Z by geekarticles.blogspot.com
  • SystemAdmin
    SystemAdmin
    2105 Posts

    Re: Fatal Error: Fork failed: Not enough space

    ‏2005-08-02T20:39:28Z  
    How large is your source file? I'd guess this is related to disk capacity on your resource/scratch space. Can you cut/paste the contents of your config file on developernet (your description is hard to picture)? Have you used a single-node config file successfully on the sequencer?
  • SystemAdmin
    SystemAdmin
    2105 Posts

    Re: Fatal Error: Fork failed: Not enough space

    ‏2005-08-03T21:27:56Z  
    This might be related to running out of virtual memory to instantiate all of the processes. Work with your SysAdmin to include/exclude this potential source of the error.

    hth,
    D
    ************************************************************************
    • Danny Owen * E-Mail: powen@us.ibm.com *
    • 5817 Southwind Dr * Title: Advanced Consultant *
    • NLR, AR 72118 * WWW: www.ascentialsoftware.com *
    • * Phone: (248) 346-8867 (Mobile) *
    ************************************************************************
    #include<stdio.h>#define WQ fprintf(ptr,"%s",a) main(){int **ptr;char a[
    100]={109,97,105,110,40,41,123,99,104,97,114,32,42,99,61,34,109,97,105,1
    10,40,41,123,99,104,97,114,32,42,99,61,37,99,37,115,37,99,59,112,114,105
    ,110,116,102,40,99,44,51,52,44,99,44,51,52,41,59,125,34,59,112,114,105,1
    10,116,102,40,99,44,51,52,44,99,44,51,52,41,59,125,10,0};ptr=stderr;WQ;}
    ************************************************************************
  • SystemAdmin
    SystemAdmin
    2105 Posts

    Re: Fatal Error: Fork failed: Not enough space

    ‏2005-09-07T20:36:48Z  
    We had a similar issue. Our system is such that they can dynamically allocate CPUs and we were running with one CPU, adding a second CPU solved the problem.

    Good luck

    Robert Hale
  • SystemAdmin
    SystemAdmin
    2105 Posts

    Re: Fatal Error: Fork failed: Not enough space

    ‏2005-09-12T09:46:56Z  
    On HPUX, you might want to increase maxdsize, maxtsize and maxssize. (kernel parameters).
  • SystemAdmin
    SystemAdmin
    2105 Posts

    Re: Fatal Error: Fork failed: Not enough space

    ‏2006-01-03T05:56:21Z  
    Hello,
    I have the same issue on Solaris.
    What can i do to check if my system configuration is correct?

    Anka
  • SystemAdmin
    SystemAdmin
    2105 Posts

    Re: Fatal Error: Fork failed: Not enough space

    ‏2006-01-03T08:11:27Z  
    I have 8 CPU, 16GB RAM and 7.4 swap...
  • SystemAdmin
    SystemAdmin
    2105 Posts

    Re: Fatal Error: Fork failed: Not enough space

    ‏2008-02-14T00:57:33Z  
    Hi,

    Even we are facing the same issue and are unable to solve. We are running the job on 2 nodes. We have 4 processor and 8GB RAM.

    Any help would be appreciated. Here are the kernel parameter settings.

    maxdsiz 1073741824 - 1073741824
    maxssiz 262144000 - 262144000
    maxtsiz 1073741824 Y 1073741824
    nproc 8212 - (20+8*MAXUSERS)
    nswapdev 10 - 10
    nswapfs 10 - 10

    Appreciate your help.

    Regards
    Nabeel
  • vijay_peruri
    vijay_peruri
    1 Post

    Re: Fatal Error: Fork failed: Not enough space

    ‏2010-09-08T07:26:37Z  
    Hi,
    how can we delete the files which are created in scratch/resource disk location.

    I tried with orchadmin rm -f filename

    No luck.

    Please suggest me

    Thanks,
    Vijay
  • geekarticles.blogspot.com
    1 Post

    Re: Fatal Error: Fork failed: Not enough space

    ‏2011-01-25T14:08:02Z  
    Hi,
    how can we delete the files which are created in scratch/resource disk location.

    I tried with orchadmin rm -f filename

    No luck.

    Please suggest me

    Thanks,
    Vijay
    What are the different stages you are using the job? Are you sroting data or using lookup stage to keep the data in the memory?

    visit http://geekarticles.blogspot.com or www.datastagetips.com for real time DS scenarios.