Topic
  • 5 replies
  • Latest Post - ‏2011-08-25T15:33:07Z by Bertram_Eckstein
Bertram_Eckstein
Bertram_Eckstein
20 Posts

Pinned topic Limits in SPSS

‏2011-08-25T13:58:15Z |
Hi all,

I'm looking for a information about a limit in SPSS. I have an ASCII-File with 400 cases. Every case has round about 57000 columns. And I have a syntax to import this file with "FILE HANDLE" and "DATA LIST FILE". In the command "FILE HANDLE" I use the option /LRECL=57000. This syntax has about 400000 lines.

With three cases I can start the syntax, but only once. After this I can about 42000 variable in the data set

Now I will start the syntax for all 400 cases. Is there any limit in SPSS. Or how can I solve my problem?

Thanks in advance for any answer.

Best regards,
Bertram
Updated on 2011-08-25T15:33:07Z at 2011-08-25T15:33:07Z by Bertram_Eckstein
  • SystemAdmin
    SystemAdmin
    445 Posts

    Re: Limits in SPSS

    ‏2011-08-25T14:15:09Z  
    In general, there are no specific limits, but processing of this syntax and data is going to be very slow. Each case will expand to nearly .5mb. Do you really have 400,000 lines of syntax? I presume that you are incorporating this via the INSERT command. Don't use the syntax window for a file that size.

    I am very puzzled, though, how you could be running 400,000 lines of syntax over a dataset with 400 cases and 57,000 variables. I wonder whether there is some way to condense this greatly.

    Regards,
    Jon
  • Bertram_Eckstein
    Bertram_Eckstein
    20 Posts

    Re: Limits in SPSS

    ‏2011-08-25T14:33:30Z  
    In general, there are no specific limits, but processing of this syntax and data is going to be very slow. Each case will expand to nearly .5mb. Do you really have 400,000 lines of syntax? I presume that you are incorporating this via the INSERT command. Don't use the syntax window for a file that size.

    I am very puzzled, though, how you could be running 400,000 lines of syntax over a dataset with 400 cases and 57,000 variables. I wonder whether there is some way to condense this greatly.

    Regards,
    Jon
    Hi Jon,

    Thank you for your answer.

    You say that I don't use the syntax windows. But how can I start the syntax without this windows?

    Regards,
    Bertram
  • Bertram_Eckstein
    Bertram_Eckstein
    20 Posts

    Re: Limits in SPSS

    ‏2011-08-25T14:33:33Z  
    In general, there are no specific limits, but processing of this syntax and data is going to be very slow. Each case will expand to nearly .5mb. Do you really have 400,000 lines of syntax? I presume that you are incorporating this via the INSERT command. Don't use the syntax window for a file that size.

    I am very puzzled, though, how you could be running 400,000 lines of syntax over a dataset with 400 cases and 57,000 variables. I wonder whether there is some way to condense this greatly.

    Regards,
    Jon
    Hi Jon,

    Thank you for your answer.

    You say that I don't use the syntax windows. But how can I start the syntax without this windows?

    Regards,
    Bertram
  • SystemAdmin
    SystemAdmin
    445 Posts

    Re: Limits in SPSS

    ‏2011-08-25T14:42:35Z  
    Hi Jon,

    Thank you for your answer.

    You say that I don't use the syntax windows. But how can I start the syntax without this windows?

    Regards,
    Bertram
    If your syntax is in a file named, say, bigjob.sps, you can run it with
    INSERT FILE="c:/someplace/bigjob.sps".

    Also, are you using the 64-bit version of Statistics? That will have access to a lot more memory. And if you have the Server version (StatisticsB) or can use external-mode Python programmability, you can probably speed up processing substantially. In those modes, there is no Data Editor window to maintain, which relieves a substantial burden with that many variables.

    Regards,
    Jon Peck
  • Bertram_Eckstein
    Bertram_Eckstein
    20 Posts

    Re: Limits in SPSS

    ‏2011-08-25T15:33:07Z  
    If your syntax is in a file named, say, bigjob.sps, you can run it with
    INSERT FILE="c:/someplace/bigjob.sps".

    Also, are you using the 64-bit version of Statistics? That will have access to a lot more memory. And if you have the Server version (StatisticsB) or can use external-mode Python programmability, you can probably speed up processing substantially. In those modes, there is no Data Editor window to maintain, which relieves a substantial burden with that many variables.

    Regards,
    Jon Peck
    Thanks a lot for the advice with the command INSERT FILE. The syntax started directly and it works.

    I work with Statistics 18.0.3 Network license under Windows 7. I don't know if this is a 64-bit version. In a next step I will try the Python mode.

    Best regards,
    Bertram