Topic
  • 14 replies
  • Latest Post - ‏2013-01-17T16:51:51Z by Robop
newuser2010
newuser2010
24 Posts

Pinned topic use flow control to solve a model with several sets of data

‏2010-11-02T01:08:20Z |
Hi, I am a new user of CPLEX, and I want to solve a model with thousands of sets of data. These data are from one Excel file, and one set of data is one column. And I want to get the average of these objective values. How can I do this? Looking for your help, thank you very much.
Updated on 2013-01-17T16:51:51Z at 2013-01-17T16:51:51Z by Robop
  • ArnaudS
    ArnaudS
    12 Posts

    Re: use flow control to solve a model with several sets of data

    ‏2010-11-05T13:08:24Z  
    Hi,

    You could do something like:

    main.mod:
    
    main 
    { var next = 1; var cols = 
    "ABCDE"; var sum = 0;   
    
    while ( next < 6 ) 
    { var source = 
    
    new IloOplModelSource(
    "a.mod"); var def = 
    
    new IloOplModelDefinition(source); var cplex = 
    
    new IloCplex(); var model = 
    
    new IloOplModel(def,cplex);   var c = cols.charAt(next++); var data = 
    
    new IloOplDataElements(); data.col = c+
    "1:"+c+
    "5"; model.addDataSource(data);   var data2 = 
    
    new IloOplDataSource(
    "a.dat"); model.addDataSource(data2); model.generate(); 
    
    if ( cplex.solve() ) 
    { sum += cplex.getObjValue(); 
    } 
    
    else 
    { writeln(
    "No solution!"); 
    
    break; 
    } model.end(); data.end(); data2.end(); def.end(); source.end(); cplex.end(); 
    }   
    }
    


    a.mod
    
    string col =...; 
    {
    
    int
    } intSet =...;   dvar 
    
    int i;   minimize i; subject to 
    { i >= sum(a in intSet)a; 
    }
    


    a.dat
    
    
    // Connection to the data in Excel file. SheetConnection sheet(
    "e:\\sheet.xls"); intSet from SheetRead( sheet, col );
    


    You could also have a look at:
    http://www.ibm.com/developerworks/forums/thread.jspa?threadID=304562&tstart=0

    Hope this helps,

    Arnaud
  • newuser2010
    newuser2010
    24 Posts

    Re: use flow control to solve a model with several sets of data

    ‏2010-11-19T16:27:23Z  
    • ArnaudS
    • ‏2010-11-05T13:08:24Z
    Hi,

    You could do something like:

    main.mod:
    <pre class="jive-pre"> main { var next = 1; var cols = "ABCDE"; var sum = 0; while ( next < 6 ) { var source = new IloOplModelSource( "a.mod"); var def = new IloOplModelDefinition(source); var cplex = new IloCplex(); var model = new IloOplModel(def,cplex); var c = cols.charAt(next++); var data = new IloOplDataElements(); data.col = c+ "1:"+c+ "5"; model.addDataSource(data); var data2 = new IloOplDataSource( "a.dat"); model.addDataSource(data2); model.generate(); if ( cplex.solve() ) { sum += cplex.getObjValue(); } else { writeln( "No solution!"); break; } model.end(); data.end(); data2.end(); def.end(); source.end(); cplex.end(); } } </pre>

    a.mod
    <pre class="jive-pre"> string col =...; { int } intSet =...; dvar int i; minimize i; subject to { i >= sum(a in intSet)a; } </pre>

    a.dat
    <pre class="jive-pre"> // Connection to the data in Excel file. SheetConnection sheet( "e:\\sheet.xls"); intSet from SheetRead( sheet, col ); </pre>

    You could also have a look at:
    http://www.ibm.com/developerworks/forums/thread.jspa?threadID=304562&tstart=0

    Hope this helps,

    Arnaud
    Thank you for your reply.
    In my problem, there are several parameters in the .dat file, for example,

    SheetConnection sheet1("demand.xls");
    demand from SheetRead(sheet1,"A1:D1");

    Take = [0, 1, 1, 1 ,
    0, 1, 1, 1 ,
    0, 1, 1, 1 ,
    0, 0, 0, 0 ,
    0, 0, 0, 0 ];

    suppliers = [ 30, 1200, 60, 3, 1 ,
    34, 1150, 58, 3, 1 ,
    35, 1100, 55, 3, 1 ,
    37, 1000, 50, 3, 1 ,
    40, 350, 18, 1, 1 ];

    Every time I only want to change demand such as from "A1:D1" to "A2:D2" and until "A1000:D1000."

    How can I do this?

    Thank you very much for your help.
  • SystemAdmin
    SystemAdmin
    1883 Posts

    Re: use flow control to solve a model with several sets of data

    ‏2010-11-19T21:33:07Z  
    Thank you for your reply.
    In my problem, there are several parameters in the .dat file, for example,

    SheetConnection sheet1("demand.xls");
    demand from SheetRead(sheet1,"A1:D1");

    Take = [0, 1, 1, 1 ,
    0, 1, 1, 1 ,
    0, 1, 1, 1 ,
    0, 0, 0, 0 ,
    0, 0, 0, 0 ];

    suppliers = [ 30, 1200, 60, 3, 1 ,
    34, 1150, 58, 3, 1 ,
    35, 1100, 55, 3, 1 ,
    37, 1000, 50, 3, 1 ,
    40, 350, 18, 1, 1 ];

    Every time I only want to change demand such as from "A1:D1" to "A2:D2" and until "A1000:D1000."

    How can I do this?

    Thank you very much for your help.
    Hi,

    you can use a similar methodology to the one described earlier except that you would split your dat file so that the data that do not change are not reinitialized each time.
    Here is an example:
    
    main 
    { var src = 
    
    new IloOplModelSource(
    "myModel.mod"); var def = 
    
    new IloOplModelDefinition(src); var fixedData = 
    
    new IloOplDataSource(
    "fixedData.dat"); 
    
    for (var i=1;i<=100;i++) 
    { var opl = 
    
    new IloOplModel(def,cplex); 
    // datasource for the spreadsheet name and range. var dataElt = 
    
    new IloOplDataElements(); dataElt.sheetName = 
    "sheet1.xls"; dataElt.sheetRange=
    "A"+i+
    ":A"+10*i; opl.addDataSource(dataElt); 
    // datasource that defines the spreadsheet connection and 
    // data read in the spreadsheet: demand var dataSource=
    
    new IloOplDataSource(
    "myData.dat"); opl.addDataSource(dataSource); 
    // datasource with elements that do not change from one solve to the other: arrays Take and suppliers opl.addDataSource(fixedData); opl.generate(); 
    
    if (cplex.solve()) 
    { writeln(
    "X = ",cplex.getObjValue()); 
    } 
    
    else 
    { writeln(
    "solve failed"); 
    } opl.end(); 
    } 
    }
    
  • newuser2010
    newuser2010
    24 Posts

    Re: use flow control to solve a model with several sets of data

    ‏2010-11-23T21:52:17Z  
    Hi,

    you can use a similar methodology to the one described earlier except that you would split your dat file so that the data that do not change are not reinitialized each time.
    Here is an example:
    <pre class="jive-pre"> main { var src = new IloOplModelSource( "myModel.mod"); var def = new IloOplModelDefinition(src); var fixedData = new IloOplDataSource( "fixedData.dat"); for (var i=1;i<=100;i++) { var opl = new IloOplModel(def,cplex); // datasource for the spreadsheet name and range. var dataElt = new IloOplDataElements(); dataElt.sheetName = "sheet1.xls"; dataElt.sheetRange= "A"+i+ ":A"+10*i; opl.addDataSource(dataElt); // datasource that defines the spreadsheet connection and // data read in the spreadsheet: demand var dataSource= new IloOplDataSource( "myData.dat"); opl.addDataSource(dataSource); // datasource with elements that do not change from one solve to the other: arrays Take and suppliers opl.addDataSource(fixedData); opl.generate(); if (cplex.solve()) { writeln( "X = ",cplex.getObjValue()); } else { writeln( "solve failed"); } opl.end(); } } </pre>
    When I tried this, it says that "Element suppliers not defined"
  • newuser2010
    newuser2010
    24 Posts

    Re: use flow control to solve a model with several sets of data

    ‏2010-11-23T21:56:37Z  
    Hi,

    you can use a similar methodology to the one described earlier except that you would split your dat file so that the data that do not change are not reinitialized each time.
    Here is an example:
    <pre class="jive-pre"> main { var src = new IloOplModelSource( "myModel.mod"); var def = new IloOplModelDefinition(src); var fixedData = new IloOplDataSource( "fixedData.dat"); for (var i=1;i<=100;i++) { var opl = new IloOplModel(def,cplex); // datasource for the spreadsheet name and range. var dataElt = new IloOplDataElements(); dataElt.sheetName = "sheet1.xls"; dataElt.sheetRange= "A"+i+ ":A"+10*i; opl.addDataSource(dataElt); // datasource that defines the spreadsheet connection and // data read in the spreadsheet: demand var dataSource= new IloOplDataSource( "myData.dat"); opl.addDataSource(dataSource); // datasource with elements that do not change from one solve to the other: arrays Take and suppliers opl.addDataSource(fixedData); opl.generate(); if (cplex.solve()) { writeln( "X = ",cplex.getObjValue()); } else { writeln( "solve failed"); } opl.end(); } } </pre>
    Attached is my file, would you please help me to take a look? Thank you very much.
  • SystemAdmin
    SystemAdmin
    1883 Posts

    Re: use flow control to solve a model with several sets of data

    ‏2010-11-23T22:06:53Z  
    When I tried this, it says that "Element suppliers not defined"
    Hi,

    Since you have a .mod that only contains the script file, you should not add the .dat file to the run configuration that executes that script. Configuration2 should only have one file: main.mod.

    Note that even after removing the .dat files, you will still get errors because by script, you're trying to initialize data (sheetName,sheetRange) that are not declared in myModel.mod

    Hope this helps,
  • newuser2010
    newuser2010
    24 Posts

    Re: use flow control to solve a model with several sets of data

    ‏2010-11-23T22:18:16Z  
    Hi,

    Since you have a .mod that only contains the script file, you should not add the .dat file to the run configuration that executes that script. Configuration2 should only have one file: main.mod.

    Note that even after removing the .dat files, you will still get errors because by script, you're trying to initialize data (sheetName,sheetRange) that are not declared in myModel.mod

    Hope this helps,
    Sorry, I did get it about (sheetName,sheetRange). How can I declare these in myModel.mod?
  • SystemAdmin
    SystemAdmin
    1883 Posts

    Re: use flow control to solve a model with several sets of data

    ‏2010-11-23T22:47:14Z  
    Sorry, I did get it about (sheetName,sheetRange). How can I declare these in myModel.mod?
    string sheetName = ...;
    string sheetRange = ...;
  • newuser2010
    newuser2010
    24 Posts

    Re: use flow control to solve a model with several sets of data

    ‏2010-11-25T00:54:31Z  
    string sheetName = ...;
    string sheetRange = ...;
    Even if I declare that, there are other errors such as "demand" not defined, "time" is not available.
  • newuser2010
    newuser2010
    24 Posts

    Re: use flow control to solve a model with several sets of data

    ‏2010-12-01T17:18:37Z  
    string sheetName = ...;
    string sheetRange = ...;
    Dear Miss Blanchard, would you please help me to take a look? Thank you very much.
  • SystemAdmin
    SystemAdmin
    1883 Posts

    Re: use flow control to solve a model with several sets of data

    ‏2010-12-01T20:40:25Z  
    Dear Miss Blanchard, would you please help me to take a look? Thank you very much.
    Hi,

    I looked back at the project you attached a few days ago. Some issues I found:
    1) Configuration 2 should only contain main.mod as I already mentioned.

    2) in the script, to define the range to read in the spread sheet, it should be:
    
    dataElt.sheetRange=
    "A"+i+
    ":D"+i;
    
    , not
    
    dataElt.sheetRange=
    "Ai:Di";
    

    Note the plus sign before the i.

    3) sheetName and sheetRange should be declared in myModel.mod:
    
    string sheetName=...; string sheetRange=...;
    

    They are initialized by script in a new data element:
    
    var dataElt = 
    
    new IloOplDataElements(); dataElt.sheetName = 
    "demand.xls"; dataElt.sheetRange=
    "A"+i+
    ":D"+i;
    


    4) Since you're defining the spreadsheet name and range in the script, the instruction to connect and read the data need to be updated to use these data.
    in myData.dat, replace
    
    SheetConnection sheet1(
    "demand.xls"); demand from SheetRead(sheet1,
    "A1:D1");
    

    by
    
    SheetConnection sheet1(sheetName); demand from SheetRead(sheet1,sheetRange);
    

    5) about "demand" not defined, "time" is not available. Looking at your model, you have:

    
    
    
    int nbHours = ...; 
    // Number of hours of auction range time = 0..nbHours; 
    // some more data 
    
    float demand[time] = ...;
    

    So the array "demand" depends on "time" which itself depends on the initialization of nbHours.
    The issue here lies in the order you added the datasources in your script.
    The datasource that initialize nbHours has to be added before the datasource that defines demand.
    If you move
    
    opl.addDataSource(fixedData);
    

    right after declaring the IloOplModel:
    
    var opl = 
    
    new IloOplModel(def,cplex); opl.addDataSource(fixedData);
    

    Then the script can be executed.

    You will find attached a modified version of the project you submitted.
    If you face additional issues, I'd recommend contacting IBM support.
  • newuser2010
    newuser2010
    24 Posts

    Re: use flow control to solve a model with several sets of data

    ‏2010-12-01T21:00:11Z  
    Hi,

    I looked back at the project you attached a few days ago. Some issues I found:
    1) Configuration 2 should only contain main.mod as I already mentioned.

    2) in the script, to define the range to read in the spread sheet, it should be: <pre class="jive-pre"> dataElt.sheetRange= "A"+i+ ":D"+i; </pre> , not <pre class="jive-pre"> dataElt.sheetRange= "Ai:Di"; </pre>
    Note the plus sign before the i.

    3) sheetName and sheetRange should be declared in myModel.mod:
    <pre class="jive-pre"> string sheetName=...; string sheetRange=...; </pre>
    They are initialized by script in a new data element:
    <pre class="jive-pre"> var dataElt = new IloOplDataElements(); dataElt.sheetName = "demand.xls"; dataElt.sheetRange= "A"+i+ ":D"+i; </pre>

    4) Since you're defining the spreadsheet name and range in the script, the instruction to connect and read the data need to be updated to use these data.
    in myData.dat, replace
    <pre class="jive-pre"> SheetConnection sheet1( "demand.xls"); demand from SheetRead(sheet1, "A1:D1"); </pre>
    by
    <pre class="jive-pre"> SheetConnection sheet1(sheetName); demand from SheetRead(sheet1,sheetRange); </pre>
    5) about "demand" not defined, "time" is not available. Looking at your model, you have:

    <pre class="jive-pre"> int nbHours = ...; // Number of hours of auction range time = 0..nbHours; // some more data float demand[time] = ...; </pre>
    So the array "demand" depends on "time" which itself depends on the initialization of nbHours.
    The issue here lies in the order you added the datasources in your script.
    The datasource that initialize nbHours has to be added before the datasource that defines demand.
    If you move
    <pre class="jive-pre"> opl.addDataSource(fixedData); </pre>
    right after declaring the IloOplModel:
    <pre class="jive-pre"> var opl = new IloOplModel(def,cplex); opl.addDataSource(fixedData); </pre>
    Then the script can be executed.

    You will find attached a modified version of the project you submitted.
    If you face additional issues, I'd recommend contacting IBM support.
    Dear Miss Blanchard, deeply appreciate your patience and help.
  • newuser2010
    newuser2010
    24 Posts

    Re: use flow control to solve a model with several sets of data

    ‏2010-12-01T21:01:14Z  
    Many thanks
  • Robop
    Robop
    9 Posts

    Re: use flow control to solve a model with several sets of data

    ‏2013-01-17T16:51:51Z  
    Hi,

    I looked back at the project you attached a few days ago. Some issues I found:
    1) Configuration 2 should only contain main.mod as I already mentioned.

    2) in the script, to define the range to read in the spread sheet, it should be: <pre class="jive-pre"> dataElt.sheetRange= "A"+i+ ":D"+i; </pre> , not <pre class="jive-pre"> dataElt.sheetRange= "Ai:Di"; </pre>
    Note the plus sign before the i.

    3) sheetName and sheetRange should be declared in myModel.mod:
    <pre class="jive-pre"> string sheetName=...; string sheetRange=...; </pre>
    They are initialized by script in a new data element:
    <pre class="jive-pre"> var dataElt = new IloOplDataElements(); dataElt.sheetName = "demand.xls"; dataElt.sheetRange= "A"+i+ ":D"+i; </pre>

    4) Since you're defining the spreadsheet name and range in the script, the instruction to connect and read the data need to be updated to use these data.
    in myData.dat, replace
    <pre class="jive-pre"> SheetConnection sheet1( "demand.xls"); demand from SheetRead(sheet1, "A1:D1"); </pre>
    by
    <pre class="jive-pre"> SheetConnection sheet1(sheetName); demand from SheetRead(sheet1,sheetRange); </pre>
    5) about "demand" not defined, "time" is not available. Looking at your model, you have:

    <pre class="jive-pre"> int nbHours = ...; // Number of hours of auction range time = 0..nbHours; // some more data float demand[time] = ...; </pre>
    So the array "demand" depends on "time" which itself depends on the initialization of nbHours.
    The issue here lies in the order you added the datasources in your script.
    The datasource that initialize nbHours has to be added before the datasource that defines demand.
    If you move
    <pre class="jive-pre"> opl.addDataSource(fixedData); </pre>
    right after declaring the IloOplModel:
    <pre class="jive-pre"> var opl = new IloOplModel(def,cplex); opl.addDataSource(fixedData); </pre>
    Then the script can be executed.

    You will find attached a modified version of the project you submitted.
    If you face additional issues, I'd recommend contacting IBM support.
    Just want to say thank you for the answer one more time: this was very helpful!