Topic
  • 9 replies
  • Latest Post - ‏2014-03-27T12:19:21Z by AleKo_
AleKo_
AleKo_
4 Posts

Pinned topic Buffer handling problems?

‏2014-03-19T07:51:29Z |

Hello everybody,

I'm struggling on a (maybe) simple problem:

the following chunk of code

Buffer b_tmp_3 = null
Array ar_tmp = create(10,1)
Object objEntry = null
Stream some_stream //further open correctly

//...some other stuff...
//like open Module, get object handler objEntry and so on


//put thins in array
int idx
for(idx=0;idx<10;idx++) 
{
    b_tmp_3 = create //allocate next buffer
    b_tmp_3 = (identifier objEntry)";" buf1 ";" buf2 ";" ( identifier someOtherObject) ";" objEntry."some attribute" ";"
    put(ar_tmp,b_tmp_3,idx,0)
}

//again some code


//get things from array
for(idx=0;idx<10;idx++) 
{
    b_tmp_3 = (Buffer get(ar_tmp,idx,0))
    some_stream << b_tmp_3 "\n"
    delete b_tmp_3 //as the buffer not needed any more
}

with this code I have

1. buffer pointer b_tmp_3 points somewhere else as the buffer object in array (I see it on output to the stream

2. after few loops Doors hangs and throw exception and close

 

if I use the string in between this works just fine:

Buffer b_tmp_3 = create
Array ar_tmp = create(10,1)
Object objEntry = null
//...some other stuff...


//put thins in array
int idx
for(idx=0;idx<10;idx++) 
{
    b_tmp_3 = (identifier objEntry)";" buf1 ";" buf2 ";" ( identifier someOtherObject) ";" objEntry."some attribute" ";"
    string temp = ""
    temp = stringOf b_tmp_3
    put(ar_tmp,b_tmp_3,idx,0)
}

//again some code


//get things from array
for(idx=0;idx<10;idx++) 
{
    string tmp_new
    tmp_new = (string get(ar_tmp,idx,0))
    some_stream << tmp_new "\n"
}

so it seems to me I do something wrong with buffers but I can not recognize what.

Thanks in advance for your ideas.

Alex

Updated on 2014-03-19T07:53:32Z at 2014-03-19T07:53:32Z by AleKo_
  • Mathias Mamsch
    Mathias Mamsch
    2160 Posts
    ACCEPTED ANSWER

    Re: Buffer handling problems?

    ‏2014-03-20T15:13:41Z  
    • AleKo_
    • ‏2014-03-20T13:02:38Z

    thanks for you answer, folks, I am studying the omitted code to check if there's something wrong.

    Some maybe stupid questions:

    1. can it be that at line 15 the buffer will not get the content, but will be pointed to some static location, to which then all buffers will point?

    2. what happens, if the initial array size is smaller than the number of loops, does it make difference if it will be dynamically extended?

    1.  Very unlikely. If so, then you would have found a very surprising bug.

    Concatenating a string to a buffer (buf str) is should always be the same as doing ((stringOf buf) str). In fact you can test yourself, by doing:

    b_tmp_3 = (identifier objEntry) ";" (stringOf buf1) ";" (stringOf buf2) ";" ( identifier someOtherObject) ";" objEntry."some attribute" ";"
    

    There might be a problem with your "buf1" and "buf2" variables, e.g. that they are not initialized, but as long as they are valid the above should work the same and should more importantly not cure your problem.

    2. Arrays are automatically resized by a static factor (1.5 I think). So no worry about out of bounds here. If performance is of high importance and you are using very large arrays in time crititcal code, to avoid large memory reallocations, you should initialize the array size to a sufficient size. But apart from that you should be covered.

    By the way, you did not tell us in WHICH loop your interpreter crashes. Maybe you really have a buffer problem with buf1 and buf2.

    Regards, Mathias

  • Adamarla
    Adamarla
    79 Posts

    Re: Buffer handling problems?

    ‏2014-03-19T13:30:33Z  
    Array ar_tmp = create(10,1)
    Object objEntry = null
    Object someOtherObject = null
    Stream some_stream //further open correctly
    
    //put thins in array
    int idx
    for(idx=0;idx<10;idx++) 
    {
        Buffer b_tmp_3 = create() //allocate next buffer
        b_tmp_3 = identifier(objEntry)
        b_tmp_3 += ";"
        b_tmp_3 += buf1
        b_tmp_3 += ";"
        b_tmp_3 += buf2
        b_tmp_3 += ";"
        b_tmp_3 += identifier(someOtherObject)
        b_tmp_3 += ";"
        b_tmp_3 += objEntry."some attribute" ";"
        put(ar_tmp, b_tmp_3, idx, 0)
    }
    
    //again some code
    
    
    //get things from array
    for(idx=0;idx<10;idx++) 
    {
        Buffer b_tmp_3 = (get(ar_tmp, idx, 0)) Buffer
        some_stream << b_tmp_3 "\n"
        delete(b_tmp_3) //as the buffer not needed any more
    }
    

    in the first one you put the same Buffer in all array elements and then try to delete it 10 times.

    In the second one you are not deleting it at all.

    You don't need a global Buffer variable, but you do need 10 Buffers.

    -Adam

    Updated on 2014-03-19T13:32:35Z at 2014-03-19T13:32:35Z by Adamarla
  • Mathias Mamsch
    Mathias Mamsch
    2160 Posts

    Re: Buffer handling problems?

    ‏2014-03-19T14:16:37Z  

    I don't really see a problem with that code. Since he is calling "create" in each loop he is correctly putting a new buffer in the array every time. And he is correctly receiving them from the array and deleting them.

    My guess is, that the problem might be related to either some weird effect or to the code, that was omitted. This shortened code works perfectly as it should:

    Buffer b_tmp_3 = null
    Array ar_tmp = create(10,1)
    
    //put thins in array
    int idx
    for(idx=0;idx<10;idx++) 
    {
        b_tmp_3 = create //allocate next buffer
        b_tmp_3 = "Test ";
        put(ar_tmp,b_tmp_3,idx,0)
    }
    
    //get things from array
    for(idx=0;idx<10;idx++) 
    {
        b_tmp_3 = (Buffer get(ar_tmp,idx,0))
        delete b_tmp_3 //as the buffer not needed any more
    }
    

    Regards, Mathias

  • llandale
    llandale
    3035 Posts

    Re: Buffer handling problems?

    ‏2014-03-19T18:17:19Z  

    I don't really see a problem with that code. Since he is calling "create" in each loop he is correctly putting a new buffer in the array every time. And he is correctly receiving them from the array and deleting them.

    My guess is, that the problem might be related to either some weird effect or to the code, that was omitted. This shortened code works perfectly as it should:

    <pre class="javascript dw" data-editor-lang="js" data-pbcklang="javascript" dir="ltr">Buffer b_tmp_3 = null Array ar_tmp = create(10,1) //put thins in array int idx for(idx=0;idx<10;idx++) { b_tmp_3 = create //allocate next buffer b_tmp_3 = "Test "; put(ar_tmp,b_tmp_3,idx,0) } //get things from array for(idx=0;idx<10;idx++) { b_tmp_3 = (Buffer get(ar_tmp,idx,0)) delete b_tmp_3 //as the buffer not needed any more } </pre>

    Regards, Mathias

    Perhaps line 15, concatenating buffers with strings is the issue?

  • Mathias Mamsch
    Mathias Mamsch
    2160 Posts

    Re: Buffer handling problems?

    ‏2014-03-19T22:00:12Z  
    • llandale
    • ‏2014-03-19T18:17:19Z

    Perhaps line 15, concatenating buffers with strings is the issue?

    If I am not very wrong, concatenating a buffer with a string should be completely valid and will returns a string. So no problem there.

  • AleKo_
    AleKo_
    4 Posts

    Re: Buffer handling problems?

    ‏2014-03-20T13:02:38Z  

    If I am not very wrong, concatenating a buffer with a string should be completely valid and will returns a string. So no problem there.

    thanks for you answer, folks, I am studying the omitted code to check if there's something wrong.

    Some maybe stupid questions:

    1. can it be that at line 15 the buffer will not get the content, but will be pointed to some static location, to which then all buffers will point?

    2. what happens, if the initial array size is smaller than the number of loops, does it make difference if it will be dynamically extended?

  • Mathias Mamsch
    Mathias Mamsch
    2160 Posts

    Re: Buffer handling problems?

    ‏2014-03-20T15:13:41Z  
    • AleKo_
    • ‏2014-03-20T13:02:38Z

    thanks for you answer, folks, I am studying the omitted code to check if there's something wrong.

    Some maybe stupid questions:

    1. can it be that at line 15 the buffer will not get the content, but will be pointed to some static location, to which then all buffers will point?

    2. what happens, if the initial array size is smaller than the number of loops, does it make difference if it will be dynamically extended?

    1.  Very unlikely. If so, then you would have found a very surprising bug.

    Concatenating a string to a buffer (buf str) is should always be the same as doing ((stringOf buf) str). In fact you can test yourself, by doing:

    b_tmp_3 = (identifier objEntry) ";" (stringOf buf1) ";" (stringOf buf2) ";" ( identifier someOtherObject) ";" objEntry."some attribute" ";"
    

    There might be a problem with your "buf1" and "buf2" variables, e.g. that they are not initialized, but as long as they are valid the above should work the same and should more importantly not cure your problem.

    2. Arrays are automatically resized by a static factor (1.5 I think). So no worry about out of bounds here. If performance is of high importance and you are using very large arrays in time crititcal code, to avoid large memory reallocations, you should initialize the array size to a sufficient size. But apart from that you should be covered.

    By the way, you did not tell us in WHICH loop your interpreter crashes. Maybe you really have a buffer problem with buf1 and buf2.

    Regards, Mathias

  • llandale
    llandale
    3035 Posts

    Re: Buffer handling problems?

    ‏2014-03-20T15:25:27Z  

    1.  Very unlikely. If so, then you would have found a very surprising bug.

    Concatenating a string to a buffer (buf str) is should always be the same as doing ((stringOf buf) str). In fact you can test yourself, by doing:

    <pre class="javascript dw" data-editor-lang="js" data-pbcklang="javascript" dir="ltr">b_tmp_3 = (identifier objEntry) ";" (stringOf buf1) ";" (stringOf buf2) ";" ( identifier someOtherObject) ";" objEntry."some attribute" ";" </pre>

    There might be a problem with your "buf1" and "buf2" variables, e.g. that they are not initialized, but as long as they are valid the above should work the same and should more importantly not cure your problem.

    2. Arrays are automatically resized by a static factor (1.5 I think). So no worry about out of bounds here. If performance is of high importance and you are using very large arrays in time crititcal code, to avoid large memory reallocations, you should initialize the array size to a sufficient size. But apart from that you should be covered.

    By the way, you did not tell us in WHICH loop your interpreter crashes. Maybe you really have a buffer problem with buf1 and buf2.

    Regards, Mathias

    I clarify that the structure "Array", with a capital "A" will auto resize.  A generic "array", with a lower case "a", will not and must be declared at the desired size.

    I also clarify that the Array resizing will stay rectangular.  So if most of your rows have 5 columns but one has 1000, then all rows are expanded to have 1000 columns.  I advise that you have a fixed number of Columns in an Array; even if one column is a Skip list with variable number of entries.

    -Louie

  • llandale
    llandale
    3035 Posts

    Re: Buffer handling problems?

    ‏2014-03-20T15:25:50Z  

    1.  Very unlikely. If so, then you would have found a very surprising bug.

    Concatenating a string to a buffer (buf str) is should always be the same as doing ((stringOf buf) str). In fact you can test yourself, by doing:

    <pre class="javascript dw" data-editor-lang="js" data-pbcklang="javascript" dir="ltr">b_tmp_3 = (identifier objEntry) ";" (stringOf buf1) ";" (stringOf buf2) ";" ( identifier someOtherObject) ";" objEntry."some attribute" ";" </pre>

    There might be a problem with your "buf1" and "buf2" variables, e.g. that they are not initialized, but as long as they are valid the above should work the same and should more importantly not cure your problem.

    2. Arrays are automatically resized by a static factor (1.5 I think). So no worry about out of bounds here. If performance is of high importance and you are using very large arrays in time crititcal code, to avoid large memory reallocations, you should initialize the array size to a sufficient size. But apart from that you should be covered.

    By the way, you did not tell us in WHICH loop your interpreter crashes. Maybe you really have a buffer problem with buf1 and buf2.

    Regards, Mathias

    I clarify that the structure "Array", with a capital "A" will auto resize.  A generic "array", with a lower case "a", will not and must be declared at the desired size.

    I also clarify that the Array resizing will stay rectangular.  So if most of your rows have 5 columns but one has 1000, then all rows are expanded to have 1000 columns.  I advise that you have a fixed number of Columns in an Array; even if one column is a Skip list with variable number of entries.

    -Louie

  • AleKo_
    AleKo_
    4 Posts

    Re: Buffer handling problems?

    ‏2014-03-27T12:19:21Z  

    The problem became irrelevant, due to the performance deficiency I have changed all buffers to string, as the latter show better performance against buffers. Strange but true. In other words - I rolled back to the original implementation (second block in the first thread entry).