Topic
14 replies Latest Post - ‏2013-07-10T14:24:25Z by ENER_Faber_McMullen
ENER_Faber_McMullen
ENER_Faber_McMullen
18 Posts
ACCEPTED ANSWER

Pinned topic Error trying to compile a Streams app that reads data from Informix

‏2013-07-01T20:23:06Z |

Hello everyone,

I have been trying to compile a Streams app. And, I keep getting an error. Here is the SPL.

use com.ibm.streams.db::* ; // make operators from the database toolkit available
composite Main
{
    graph
        stream<rstring fname> Names = ODBCSource()
        { // return a stream called Names
            param
                connection : "DBCustomer" ; // Identify the connection to use
                access : "customer" ; // identify the statement to execute
                connectionDocument : "connections.xml" ; // identify the connections definition file
        }
        () as FileSink_1 = FileSink(Names)
        {
            param
                file : "output.csv" ;
                format : csv ;
        }
}

Here is connections.xml

<st:connections xmlns:st="http://www.ibm.com/xmlns/prod/streams/adapters"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<connection_specifications>
<connection_specification name="DBCustomer" >
    <ODBC database="test" user="informix" password="*****" />
</connection_specification>
</connection_specifications>
<access_specifications>
<access_specification name="customer">
    <query query="SELECT fname FROM customer" replays="1"
          isolation_level="READ_COMMITTED" />
    <parameters></parameters>
    <uses_connection connection="DBCustomer" />
    <native_schema>
        <column name="fname" type="CHAR" length="15"/>
    </native_schema>
</access_specification>
</access_specifications>
</st:connections>

Here is the output I get from trying to compile in Streams Studio:

---- Indexing toolkit for project InformixTest started ---- July 1, 2013 3:12:38 PM CDT
---- Indexing toolkit for project InformixTest completed in 0.324 seconds ----

---- SPL Build for project InformixTest started ---- July 1, 2013 3:12:38 PM CDT

Building main composite: Main using build configuration: Distributed

/opt/ibm/InfoSphereStreams/bin/sc -M Main --output-directory=output/Main/Distributed --data-directory=data -t /opt/ibm/InfoSphereStreams/toolkits/com.ibm.streams.db --no-toolkit-indexing --no-mixed-mode-preprocessing

Checking constraints...
Creating types...
Creating functions...
Creating operators...
Creating PEs...
Creating application model...
Building binaries...
 [CXX-operator] Names
src/operator/Names.cpp: In member function 'bool SPL::_Operator::Names$OP::establishODBCConnection()':
src/operator/Names.cpp:389: error: cannot convert 'SQLINTEGER*' to 'SQLLEN*' for argument '6' to 'SQLRETURN SQLBindCol(void*, SQLUSMALLINT, SQLSMALLINT, void*, SQLLEN, SQLLEN*)'
make: *** [build/operator/Names.o] Error 1
CDISP0141E ERROR: Compilation of the generated code has failed.

---- SPL Build for project InformixTest completed in 1.909 seconds ----

Can someone please tell me how to make Streams Studio generate C++ code that does not have this bug?

  • pcbye
    pcbye
    26 Posts
    ACCEPTED ANSWER

    Re: Error trying to compile a Streams app that reads data from Informix

    ‏2013-07-02T18:51:37Z  in response to ENER_Faber_McMullen

    Hello,

    My initial attempt to reproduce the problem with the sources you listed was unsuccessful.  However, I did have a couple of questions which might affect the compile:

    - What values do you have set for the environment variables STREAMS_ADAPTERS_ODBC_INCPATH and STREAMS_ADAPTERS_ODBC_LIBPATH?  (and I assume you have STREAMS_ADAPTERS_ODBC_IDS=1 already or it probably wouldn't get this far in the compile)

    - What version of Informix are you using?

    - What version of Streams are you using?

    My initial guess without knowing at least the first question is that you don't have the INCPATH and LIBPATH variable set to anything, and that the compiler is picking up the system default version of UnixODBC from /usr/lib and /usr/include, which would not work (and probably produce the error you are seeing and others).

    • ENER_Faber_McMullen
      ENER_Faber_McMullen
      18 Posts
      ACCEPTED ANSWER

      Re: Error trying to compile a Streams app that reads data from Informix

      ‏2013-07-02T19:30:33Z  in response to pcbye

      Thanks for the fast reply.

      INFORMIXDIR=/opt/ibm/informix

      STREAMS_ADAPTERS_ODBC_INCPATH=/opt/ibm/informix/incl/cli

      STREAMS_ADAPTERS_ODBC_LIBPATH=/opt/ibm/informix/lib

      STREAMS_ADAPTERS_ODBC_IDS=1

      I am using Informix 12.10. And, I am using Streams 3.0.

      • pcbye
        pcbye
        26 Posts
        ACCEPTED ANSWER

        Re: Error trying to compile a Streams app that reads data from Informix

        ‏2013-07-02T20:41:18Z  in response to ENER_Faber_McMullen

        Thanks for the additional information.

        I did some additional testing, and I think I might see what could be happening.

        The Informix native driver= requires the header files to be included:

        #include <infxsql.h>
        #include <sqlext.h>

        Unfortunately sqlext.h happens to also be the same as the name of one of the header files shipped with UnixODBC, which comes standard on most Linux operating systems (or at least it's an option).  Depending on whether (and where) UnixODBC is installed on the system, and what your compiler search path is, it is possible to pick up the UnixODBC version of sqlext.h, and since UnixODBC does not contain a header file named infxsql.h, pick up the correct Informix version of that header file.

        First, you can confirm whether or not you have UnixODBC installed by running:

        rpm -qa | grep -i unixODBC

        and see if it returns an rpm called "unixODBC-devel"

        Second, you can see what your compiler search path is with your Informix include dir added by running:

        echo | g++ -v -x c++ -E -I/opt/ibm/informix/incl/cli  -

        The desired output of this command would be to look something like this (mainly that /opt/ibm/informix/incl/cli is at the top of the list):

        #include "..." search starts here:
        #include <...> search starts here:
         /opt/ibm/informix/incl/cli
         /usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2
         /usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/x86_64-redhat-linux
         /usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/backward
         /usr/local/include
         /usr/lib/gcc/x86_64-redhat-linux/4.1.2/include
         /usr/include

        If you do have the unixODBC-devel RPM installed and can remove it, that might also be a good test to see if that is what's causing the problem.

        • ENER_Faber_McMullen
          ENER_Faber_McMullen
          18 Posts
          ACCEPTED ANSWER

          Re: Error trying to compile a Streams app that reads data from Informix

          ‏2013-07-02T21:25:16Z  in response to pcbye

          I ran the command you gave me "echo | g++ -v -x c++ -E -I/opt/ibm/informix/incl/cli". Here is the output.

          g++: warning: '-x c++' after last input file has no effect
          Using built-in specs.
          Target: x86_64-redhat-linux
          Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-bootstrap --enable-shared --enable-threads=posix --enable-checking=release --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-languages=c,c++,objc,obj-c++,java,fortran,ada --enable-java-awt=gtk --disable-dssi --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-1.5.0.0/jre --enable-libgcj-multifile --enable-java-maintainer-mode --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --disable-libjava-multilib --with-ppl --with-cloog --with-tune=generic --with-arch_32=i686 --build=x86_64-redhat-linux
          Thread model: posix
          gcc version 4.4.7 20120313 (Red Hat 4.4.7-3) (GCC)

           

          • pcbye
            pcbye
            26 Posts
            ACCEPTED ANSWER

            Re: Error trying to compile a Streams app that reads data from Informix

            ‏2013-07-02T22:11:07Z  in response to ENER_Faber_McMullen

            The last '-' in the command is actually important :)

            echo | g++ -v -x c++ -E -I/opt/ibm/informix/incl/cli  -

            that will give you more output

            • ENER_Faber_McMullen
              ENER_Faber_McMullen
              18 Posts
              ACCEPTED ANSWER

              Re: Error trying to compile a Streams app that reads data from Informix

              ‏2013-07-03T18:19:40Z  in response to pcbye

              [root@linuxvm etc]# echo | g++ -v -x c++ -E -I/opt/ibm/informix/incl/cli  -
              Using built-in specs.
              Target: x86_64-redhat-linux
              Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-bootstrap --enable-shared --enable-threads=posix --enable-checking=release --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-languages=c,c++,objc,obj-c++,java,fortran,ada --enable-java-awt=gtk --disable-dssi --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-1.5.0.0/jre --enable-libgcj-multifile --enable-java-maintainer-mode --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --disable-libjava-multilib --with-ppl --with-cloog --with-tune=generic --with-arch_32=i686 --build=x86_64-redhat-linux
              Thread model: posix
              gcc version 4.4.7 20120313 (Red Hat 4.4.7-3) (GCC)
              COLLECT_GCC_OPTIONS='-v' '-E' '-I/opt/ibm/informix/incl/cli' '-shared-libgcc' '-mtune=generic'
               /usr/libexec/gcc/x86_64-redhat-linux/4.4.7/cc1plus -E -quiet -v -I/opt/ibm/informix/incl/cli -D_GNU_SOURCE - -mtune=generic
              ignoring nonexistent directory "/usr/lib/gcc/x86_64-redhat-linux/4.4.7/include-fixed"
              ignoring nonexistent directory "/usr/lib/gcc/x86_64-redhat-linux/4.4.7/../../../../x86_64-redhat-linux/include"
              #include "..." search starts here:
              #include <...> search starts here:
               /opt/ibm/informix/incl/cli
               /usr/lib/gcc/x86_64-redhat-linux/4.4.7/../../../../include/c++/4.4.7
               /usr/lib/gcc/x86_64-redhat-linux/4.4.7/../../../../include/c++/4.4.7/x86_64-redhat-linux
               /usr/lib/gcc/x86_64-redhat-linux/4.4.7/../../../../include/c++/4.4.7/backward
               /usr/local/include
               /usr/lib/gcc/x86_64-redhat-linux/4.4.7/include
               /usr/include
              End of search list.
              # 1 "<stdin>"
              # 1 "<built-in>"
              # 1 "<command-line>"
              # 1 "<stdin>"
              COMPILER_PATH=/usr/libexec/gcc/x86_64-redhat-linux/4.4.7/:/usr/libexec/gcc/x86_64-redhat-linux/4.4.7/:/usr/libexec/gcc/x86_64-redhat-linux/:/usr/lib/gcc/x86_64-redhat-linux/4.4.7/:/usr/lib/gcc/x86_64-redhat-linux/:/usr/libexec/gcc/x86_64-redhat-linux/4.4.7/:/usr/libexec/gcc/x86_64-redhat-linux/:/usr/lib/gcc/x86_64-redhat-linux/4.4.7/:/usr/lib/gcc/x86_64-redhat-linux/
              LIBRARY_PATH=/usr/lib/gcc/x86_64-redhat-linux/4.4.7/:/usr/lib/gcc/x86_64-redhat-linux/4.4.7/:/usr/lib/gcc/x86_64-redhat-linux/4.4.7/../../../../lib64/:/lib/../lib64/:/usr/lib/../lib64/:/usr/lib/gcc/x86_64-redhat-linux/4.4.7/../../../:/lib/:/usr/lib/
              COLLECT_GCC_OPTIONS='-v' '-E' '-I/opt/ibm/informix/incl/cli' '-shared-libgcc' '-mtune=generic'
              [root@linuxvm etc]#

              • pcbye
                pcbye
                26 Posts
                ACCEPTED ANSWER

                Re: Error trying to compile a Streams app that reads data from Informix

                ‏2013-07-03T20:04:04Z  in response to ENER_Faber_McMullen

                OK.  That looks like how it should be, with the Informix include directory being searched before /usr/local/include or /usr/include (same as mine).

                I guess to rule out everything - can you list the the contents of /opt/ibm/informix/incl/cli

                ls /opt/ibm/informix/incl/cli

                and verify that you can read /opt/ibm/informix/incl/cli/sqlext.h

                • ENER_Faber_McMullen
                  ENER_Faber_McMullen
                  18 Posts
                  ACCEPTED ANSWER

                  Re: Error trying to compile a Streams app that reads data from Informix

                  ‏2013-07-08T20:48:00Z  in response to pcbye

                  Here is my copy of /opt/ibm/informix/incl/cli/sqlext.h. If you see anything suspicious, let me know.

                  Attachments

                  • pcbye
                    pcbye
                    26 Posts
                    ACCEPTED ANSWER

                    Re: Error trying to compile a Streams app that reads data from Informix

                    ‏2013-07-09T14:44:49Z  in response to ENER_Faber_McMullen

                    Thanks - I verified that your file is the same as mine.

                    I also had some discussion with another person on the team and I think we might know what it is.  There was a change to the Informix ODBC driver in the 4.10 client (12.10 server), involving the definition of SQLLEN (details pasted below).  The net is that we made a fix in Streams 3.1 in the DB Toolkit to address the Informix ODBC driver change, but Streams 3.0 still did not address this.

                    I assume you would not want to go to an earlier version of Informix, but is there any possibility you could update to Streams 3.1?  That would fix the problem.  If not, there may be other things we can do to work around this.

                    Details on the Informix change:

                    Informix release notes states the following:

                    1.3 What's new for the ODBC Driver, Version 4.10

                    1.3.1 Recompile required for Client SDK prior to version 4.10

                    If you have a 64-bit ODBC application that was compiled and linked with
                    a version of IBM Informix Client Software Development Kit (Client SDK)
                    that is prior to version 4.10, you must recompile the application after
                    migrating.

                    The SQLLEN and SQLULEN data types were changed to match the Microsoft
                    64-bit ODBC specification. To ensure that the correct type passes to the
                    function, be sure to analyze any functions that take either of these
                    types as parameters. This is crucial if the type is a pointer. Also note
                    that in the ODBC specification, some parameters that were previously
                    SQLINTEGER and SQLUINTEGER were changed to SQLLEN or SQLULEN.

                    For the ODBC driver on UNIX, the SQLLEN data type is now mapped based on
                    the platform. It is 4 bytes on 32-bit platforms and 8 bytes on 64-bit
                    platforms.

                    1.3.2 SQLLEN data type mapped based on UNIX platform

                    For the ODBC driver on UNIX, the SQLLEN data type is now mapped based on
                    the platform. It is 4 bytes on 32-bit platforms and 8 bytes on 64-bit
                    platforms.

                     

                    • ENER_Faber_McMullen
                      ENER_Faber_McMullen
                      18 Posts
                      ACCEPTED ANSWER

                      Re: Error trying to compile a Streams app that reads data from Informix

                      ‏2013-07-09T19:20:21Z  in response to pcbye

                      I uninstalled ClientSDK and installed ClientSDK for Informix 11.70. It compiles fine now. But I get the following runtime error.

                      09 Jul 2013 14:14:31.248 [2890] ERROR #splapplog,J[0],P[0],Names,spl_operator M[Names.cpp:establishODBCConnection:314]  - An error occurred while the system attempted an SQL operation.
                      09 Jul 2013 14:14:31.249 [2890] ERROR #splapptrc,J[0],P[0],Names,spl_operator M[Names.cpp:establishODBCConnection:315]  - Failed to connect to database with return code SQL_ERROR
                      09 Jul 2013 14:14:31.250 [2890] ERROR #splapplog,J[0],P[0],Names,spl_operator M[Names.h:logSQLError:216]  - An SQL operation failed. The SQL state is IM002, the SQL code is -11041 and the SQL message is [Informix][Informix ODBC Driver]Data source name not found and no default driver specified..

                      • Ivan_CA
                        Ivan_CA
                        10 Posts
                        ACCEPTED ANSWER

                        Re: Error trying to compile a Streams app that reads data from Informix

                        ‏2013-07-09T22:07:58Z  in response to ENER_Faber_McMullen

                        Hi,

                        It looks like you are missing a few things in your environment. Let me share my env, it might help you.

                         

                        Env:

                        export STREAMS_ADAPTERS_ODBC_INCPATH=/opt/informix/incl/cli
                        export STREAMS_ADAPTERS_ODBC_LIBPATH=/opt/informix/lib
                        export ODBCINI=/home/ivan/ids/odbc.ini
                        export INFORMIXSQLHOSTS=/home/ivan/ids/sqlhosts
                        export STREAMS_ADAPTERS_ODBC_IDS=1
                        export LD_LIBRARY_PATH=/opt/informix/lib:/opt/informix/lib/cli:/opt/informix/lib/esql
                        export INFORMIXDIR=/opt/informix

                        Also...


                        streamtool setproperty -i streams@ivan DNA.backDoorEvs=ODBCINI=/home/ivan/ids/odbc.ini,INFORMIXDIR=/opt/informix,INFORMIXSQLHOSTS=/home/ivan/ids/sqlhosts

                         

                        I hope that helps. Let us know what happens after you double-check your env vars and streams properties.

                        Thanks,

                        Ivan

                        • ENER_Faber_McMullen
                          ENER_Faber_McMullen
                          18 Posts
                          ACCEPTED ANSWER

                          Re: Error trying to compile a Streams app that reads data from Informix

                          ‏2013-07-10T13:36:08Z  in response to Ivan_CA

                          I did what you told me to do. I got the following runtime error.

                          10 Jul 2013 08:21:01.695 [2860] ERROR #splapplog,J[0],P[0],Names,spl_operator M[Names.cpp:establishODBCConnection:347]  - An error occurred while the system attempted an SQL operation.
                          10 Jul 2013 08:21:01.696 [2860] ERROR #splapptrc,J[0],P[0],Names,spl_operator M[Names.cpp:establishODBCConnection:348]  - Failed to set isolation level with return code SQL_ERROR
                          10 Jul 2013 08:21:01.697 [2860] ERROR #splapplog,J[0],P[0],Names,spl_operator M[Names.h:logSQLError:216]  - An SQL operation failed. The SQL state is HY000, the SQL code is -256 and the SQL message is [Informix][Informix ODBC Driver][Informix]Transaction not available..

                          Here are the contents of my .bashrc file.

                          export INFORMIXDIR=/opt/ibm/informix
                          export INFORMIXSERVER=demo_on
                          export ONCONFIG=onconfig
                          export INFORMIXSQLHOSTS=$INFORMIXDIR/etc/sqlhosts
                          export DB2CLIINIPATH=$HOME/dsdriver/cfg/db2cli.ini
                          export STREAMS_ADAPTERS_ODBC_IDS=1
                          export STREAMS_ADAPTERS_ODBC_INCPATH=/opt/ibm/informix/incl/cli
                          export STREAMS_ADAPTERS_ODBC_LIBPATH=/opt/ibm/informix/lib
                          export ODBCINI=$HOME/odbc.ini
                          export LD_LIBRARY_PATH=$INFORMIXDIR/lib:$INFORMIXDIR/lib/cli:$INFORMIXDIR/lib/esql
                          export DB2COMM=TCPIP
                          streamtool setproperty -i streams@streamsadmin DNA.backDoorEvs=ODBCINI=/home/streamsadmin/odbc.ini,INFORMIXDIR=/opt/ibm/informix,INFORMIXSQLHOSTS=/opt/ibm/informix/etc/sqlhosts

                          Here are the contents of my sqlhosts file.

                          demo_on    onsoctcp    linuxvm    ol_informix1170

                          Here is the record from my /etc/services file

                          ol_informix1170       5108/tcp

                          Here is my odbc.ini file.

                          [ODBC Data Sources]
                          test=INFORMIX
                           
                          [test]
                          Driver=/opt/ibm/informix/lib/cli/iclit09b.so
                          Description=IBM INFORMIX ODBC DRIVER
                          Database=test
                          LogonID=informix
                          pwd=***
                          Servername=demo_on
                          CLIENT_LOCALE=en_us.8859-1
                          DB_LOCALE=en_us.8859-1
                          TRANSLATIONDLL=/opt/ibm/informix/lib/esql/igo4a304.so

                          Also, my JDBC application is not working. I am trying to connect to the URL "jdbc:informix-sqli://localhost:5108:INFORMIXSERVER=demo_on;user=informix;password=***". The JDBC application worked back when I had Informix 12.10 installed. The only thing I changed in the URL since then is the port number to match the port number in my /etc/services file. And, I am running the JDBC application on the same machine that Informix is running on. And, here is what I get.

                          java.sql.SQLException: com.informix.asf.IfxASFException: Attempt to connect to database server (demo_on) failed.

                          at com.informix.jdbc.IfxSqliConnect.<init>(IfxSqliConnect.java:1629)

                          • Ivan_CA
                            Ivan_CA
                            10 Posts
                            ACCEPTED ANSWER

                            Re: Error trying to compile a Streams app that reads data from Informix

                            ‏2013-07-10T14:13:14Z  in response to ENER_Faber_McMullen

                            That's good news. The error -256 is coming from Informix and at least indicates that you are reaching the db server with no problems.

                             

                            It seems your database "test" was created with no log so it does not support a few things needed by Streams. Please recreate your database with the following command and this should take care of the error -256.

                            "create database test with log"

                             

                            Thanks!

                            Ivan

                             

                             

                          • ENER_Faber_McMullen
                            ENER_Faber_McMullen
                            18 Posts
                            ACCEPTED ANSWER

                            Re: Error trying to compile a Streams app that reads data from Informix

                            ‏2013-07-10T14:24:25Z  in response to ENER_Faber_McMullen

                            Thanks for your help. The Streams problem has been solved. I have not tested the JDBC problem yet. But, the JDBC application is not necessary for my project anyway. I was just using it to see if Informix was working.