Pinned topic FIVE BUGS/CAVEATS IN THE ISERIES ACCESS ODBC CONNECTOR FOR LINUX X86_64
1) Segmentation fault upon language specific DLL use
Whenever the ODBC connector attempts to get an error or warning
message, it loads a language-specific DLL and computes some offset in it.
This computation is buguous and results in a segmentation fault.
Patching the binary code of library
/opt/ibm/iSeriesAccess/lib64/libcwbcore.so does the job. At external symbol
_ZN11PiNlStrFile6loadupEv + 0x159 (C++ PiNlStrFile::loadup() + 345), replace
+0x159 49 03 84 24 18 01 00 00 add 0x118(%r12),%rax
+0x161 41 2B 50 0C sub 0xC(%r8),%edx
+0x159 48 29 C2 sub %rax,%rdx
+0x15C 49 03 84 24 18 01 00 00 add 0x118(%r12),%rax
+0x164 90 nop
_ This fix has been obtained by binary code debugging and reverse assembly.
A real C++ fix ought to be applied by IBM to the original source code before
2) Connector returns 32-bit field lengths.
Upon reading a selected DB record, field lengths are returned on
32 bits only. This is probably due to the ODBC library change: the connector is
compiled for libodbc.so/libodbcinst.so with soname = 1, while Fedora 12
provides them with soname = 2. This indicates an ABI change (probably
motivated by the SQLLEN type change itself!) Some web forums advise to symlink
libodbc.so.1 to libodbc.so.2: this allows to load the connector, but does
not garantee it is functional.
Happily, Intel CPUs are little endian machines, so bytes of a 32-bit
integer are stored at the same offset as in the equivalent 64-bit value. The
values passed as input parameters to the connector are then interpreted
correctly (providing they fit in 32 bits), but output parameters have their MS
32 bits undefined. Some code must then be added in the caller after reading
a DB field to restore the MS 32 bits. Care must be taken to properly handle
the special negative value SQL_NULL_DATA (-1). The sequence to "repare" field
length should then be something like:
field_length &= 0xFFFFFFFF;
if ((field_length & 0xFFFFFFFF) == 0xFFFFFFFF)
field_length = SQL_NULL_DATA;
Of course, this solution must be applied in conjunction with the symlinks
mentionned above, and does not work with field larger than 4 Gbytes !
It is highly probable that other types are also affected by size changes. Those
have not been identified: a similar processing must then be applied to values
of these types by the caller.
A simple recompilation with libraries with soname = 2 will totally fix the
3) DB field input with truncation returns a wrong length.
When reading a DB record, bounded length storage does not reflect the effective
input size if the corresponding field has been truncated.
For character fields, always use bound buffers of a byte length of at least
four times the field character size: this will prevent truncation in all cases,
even if all characters are returned as 4-byte UTF-8 characters.
_ This problem as already been identified in the PHP context:
but non-PHP applications are affected too.
_ IBM provides a workaround: ODBC connection keyword "DEBUG = 65536".
_ The connector probably returns the recommended size for the allocation of a
new buffer for the truncated field. It should instead return the effective
field byte size (since the truncation is not an error, but a warning).
Source code should be fixed by IBM accordingly.
4) The client "CCSID" keyword does not apply to SQL statements.
Submitting a non 7-bit ASCII SQL statement (a string with at least 1 byte
outside USASCII range) to the connector always fails unless the caller has
previoulsy called "setlocale(LC_CTYPE, <charset_name>)", <charset_name> being
the character set name used for the SQL statement.
Always call setlocale() accordingly.
_ The CCSID keyword properly applies to non-SQL data, i.e.: character field
values are translated into this CCSID upon reading. It should also apply to
SQL, since it is a nonsense to have a regular DB client dealing with two
different caracter encodings. In addition, SQL statement conversion is
performed by a deprecated standard function instead of the connector's internal
_ In the 2.6 kernel era, default should be at least to use UTF-8.
IBM should fix the source code to use it's internal charset converter and the
CCSID keyword value for SQL statement conversion.
5) Connector is not interrupt-safe.
There are some code sequences in the connector that do not preserve so-called
stacked values from being overwritten by an interrupt (aka signal) handler.
Never use signals in a program using the connector, or at least disable the
signals before entering ODBC functions and re-enable them upon return.
Bad code sequence can be identified by negative offsets from the stack
pointer: plenty of them occurs, mainly at function entries. As an example,
in /opt/ibm/iSeriesAccess/lib64/libcwbcore.so, function
ZN6cwbINI12CurrentValueEPcS0 (C++ cwbINI::CurrentValue(char*, char*)) starts
464c0: 48 89 5c 24 f0 mov %rbx,-0x10(%rsp)
464c5: 48 89 6c 24 f8 mov %rbp,-0x8(%rsp)
464ca: 48 83 ec 18 sub $0x18,%rsp
A signal interrupting the program just before execution of the instruction
at 464ca will likely destroy values stored by the two preceding instructions, by
the stacking needs of its own handler.
This error comes probably from a bad optimizing scheme in C compiler used to
produce the binary library: local storage initialization has been moved before
its allocation !
A simple recompilation with a fixed compiler should do the job.
The above temporary solutions make the connector usable, providing you have
access to the calling sources and are able to recompile these applications. In
no way they will make it operational within an existing binary application that
already fails on the original ODBC connector.
Some of these bugs have been reported to IBM in april 2010: they promised to
release a new version in ftp://ftp.boulder.ibm.com/as400/ before june 15th 2010.
As of today (Aug 13) we are still waiting.
Since IBM provides this software freely to promote iSeries, pretends to
support it although never releasing updates, and will never be able to
provide this connector in sync with all available Linux distributions (even
with those pretended to be officially supported), I strongly suggest they
opensource this ODBC connector: the community members will then be able to
fix the problems, compile it for their favourite distribution and make it a
live product): the expected side effects will be not only an improved
usability, but also a better connectivity range, promoting IBM iSeries computers
without any additional cost for this company.
_ When will we get an update (real date!) ?
_ What about opening the sources ?
Thanks for the replies.
TuomasA 270000AMP71 PostACCEPTED ANSWER
Re: FIVE BUGS/CAVEATS IN THE ISERIES ACCESS ODBC CONNECTOR FOR LINUX X86_642010-08-26T08:06:14Z in response to datasphereHi!
I can confirm that probably the same segmentation fault bug happens to me on Ubuntu 10.04 (x86_64).
Some queries are executed ok, but other queries cause the app to segfault.
Hoping to get a fix to this bug since it's a major problem for us.
Re: FIVE BUGS/CAVEATS IN THE ISERIES ACCESS ODBC CONNECTOR FOR LINUX X86_642013-09-05T19:29:10Z in response to datasphere
It is utterly amazing that IBM still has not fixed this problem. I get segfaults in windows and linux.
IBM obviously hates their customers - 3+ years waiting for a simple fix.
Re: FIVE BUGS/CAVEATS IN THE ISERIES ACCESS ODBC CONNECTOR FOR LINUX X86_642013-10-14T14:22:04Z in response to AndrewKehrig
Bug still present in 7.1 and Fedora 19 on ppc64p7
It's a shame. I'm a full IBM customer with a brand new Power720, not a cheap machine, and I can't get any support and see that old open bugs are still there.
Demonstrator of the bug there : https://bugzilla.redhat.com/show_bug.cgi?id=1009909
What to do to have things working correctly ?
Re: FIVE BUGS/CAVEATS IN THE ISERIES ACCESS ODBC CONNECTOR FOR LINUX X86_642013-10-29T19:05:48Z in response to datasphere
Hi guys, I'd just like to let you know that we've fixed a lot of the problems you've been seeing with our 64-bit driver with the release of our IBM i Access Client Solutions (ACS) and Linux Application Package. ACS is the replacement for some of the iSeriesAccess family of products (emulator, data transfer, etc...). This product is all Java based, so it runs on Linux, Mac, Windows, etc...
To go along with this product, we've also released two application packages with native tools: on Windows this includes the .NET and OLEDB providers, the AFP printer driver, and the ODBC driver. On Linux, this is essentially just the ODBC driver. This ODBC driver includes all the fixes that were included in the iSeriesAccess 7.1 SP8 driver. Additionally, it has been built with a newer version of unixODBC to support full 64-bit parameters and we also now support Debian based distributions with the included deb files. We also plan to release service packs for Linux with this product, which is something I don't believe we ever did for iSeriesAccess for Linux.
Unfortunately, we don't have PPC support for this new product yet, but that will be coming with the next service pack (which should come out in the next month or so).
Check out http://www-03.ibm.com/systems/power/software/i/access/solutions.html for more info on ACS and the Linux Application Package.
Also, if you guys find any issues with the ACS Linux package, please open a PMR. You will always get a prompt response when going through our official support channels.
Re: FIVE BUGS/CAVEATS IN THE ISERIES ACCESS ODBC CONNECTOR FOR LINUX X86_642013-10-29T19:07:28Z in response to kadler
Also, FYI: due to unixODBC not correctly bumping the SO version when they updated SQLLEN to be 64bit at 2.2.14, our driver still binds against libodbc.so.1. Some distributions that ship 2.2.14 patched their packages to correctly bump the SO version and unixODBC 2.3.1 finally bumped the SO version. When running on distributions that ship with this updated SO version, you will need to make a symlink from libodbcinst.so.1 to libodbcinst.so.2. I'm hoping that we can fix this in the future.
Re: FIVE BUGS/CAVEATS IN THE ISERIES ACCESS ODBC CONNECTOR FOR LINUX X86_642013-11-15T13:45:48Z in response to stepson
is "next month or so" is november or december when you said it at the end of october ?
I really need to fix this issue on linux on ppc64p7 arch urgently now. I can wait 1 or 2 week still, but not more.
It's a shame from IBM to not provide support for its own hardware !!
Am I alone to try to run Linux on a partition on an IBM i ?
Re: FIVE BUGS/CAVEATS IN THE ISERIES ACCESS ODBC CONNECTOR FOR LINUX X86_642013-11-20T21:53:47Z in response to stepson
IBM i Access Application SP 1 packages has been released to ESS. It should become available sometime this Friday. You can get it from ESS when it becomes available. If you have trouble finding it on ESS, see this document.
When it's available it will have the filename IBM_i_Access_Client_Solutions_-_Linux_AP_LCD8_2012_01.zip.Updated on 2013-11-20T21:54:14Z at 2013-11-20T21:54:14Z by kadler
Re: FIVE BUGS/CAVEATS IN THE ISERIES ACCESS ODBC CONNECTOR FOR LINUX X86_642014-01-06T18:41:11Z in response to kadler
Getting invalid parameter count for any query I toss at it with bound parameters. Same query on the 32bit driver works fantastic.
No more segfaults - now I just can't seem to actually query for anything more than simple select statements.
Re: FIVE BUGS/CAVEATS IN THE ISERIES ACCESS ODBC CONNECTOR FOR LINUX X86_642014-01-06T20:51:32Z in response to AndrewKehrig
Andrew, I have no problems with bound parameters here in any of my testing. I'd suggest opening a PMR so that we can collect trace data and investigate further.
Re: FIVE BUGS/CAVEATS IN THE ISERIES ACCESS ODBC CONNECTOR FOR LINUX X86_642014-01-07T18:04:43Z in response to kadler
I'm seeing completely different results depending on the kernel + unixODBC versions (all of which are greater than the listed requirement).
Can we get a list of 'tested' requirements for this driver, such as: "Will work on Ubuntu Server 12.04 LTS, Centos 6.5 x86_64, Debian 7.3 x64" etc?
Simply stating the minimum required version of unixODBC seems to not be entirely useful.
I just tested on a fully up to date Ubuntu Server 13.10 box and it's segfaulting instead of complaining about the parameter count.
I also completely removed the 2.2.14 unixODBC that's shipped with the package list and instead compiled & installed unixODBC 2.3.2. All symlinks are available for the driver, it's pointing to the proper lib version, and still segfaults.Updated on 2014-01-07T18:35:02Z at 2014-01-07T18:35:02Z by AndrewKehrig
Re: FIVE BUGS/CAVEATS IN THE ISERIES ACCESS ODBC CONNECTOR FOR LINUX X86_642014-01-08T16:33:03Z in response to AndrewKehrig
We've tested with Ubuntu Server 12.04.3 LTS, RHEL/CentOS 6.4, and openSUSE 12.3 (with unixODBC 2.3.1 from OBS). These systems will probably be getting updated to RHEL/CentOS 6.5 and openSUSE 13.1 sometime soon. We've done ad-hoc testing with other distributions as well.
The kernel should not matter in the slightest, since Linus is very adamant about not breaking userspace. Just because it shouldn't, doesn't it mean it can't, but it seems very unlikely to me. Likewise, unixODBC doesn't really do anything - it just forwards API calls to the driver and manages the driver registry. The only reason we require 2.2.14 and above is that the header files were updated to change some types and parameters to 64bit (SQLINTEGER -> SQLLEN) on 64bit. Conceivabley, you could compile your app against 2.2.14 and then run it on previous versions of unixODBC with our new driver just fine, wheras compiling against the old version and running against the new driver would yield issues.
I'd be really interested to see traces and core dumps of your issues, but you'll need to open a PMR for that.
Re: FIVE BUGS/CAVEATS IN THE ISERIES ACCESS ODBC CONNECTOR FOR LINUX X86_642014-01-15T23:02:43Z in response to kadler
I'm going to pester the person who has the IBM account information some more about opening a PMR tomorrow when they're in the office.
In the meantime, from CentOS 6.5 I get the following stacktrace snippet:Program terminated with signal 11, Segmentation fault.#0 0x00007f60bbf460c5 in odbcConv_PreConvert_C_CHAR(STATEMENT_INFO&, char const*&, unsigned long&, COLUMN_INFO&) () from /opt/ibm/iSeriesAccess/lib64/libcwbodbc.so(gdb) bt#0 0x00007f60bbf460c5 in odbcConv_PreConvert_C_CHAR(STATEMENT_INFO&, char const*&, unsigned long&, COLUMN_INFO&) () from /opt/ibm/iSeriesAccess/lib64/libcwbodbc.so#1 0x00007f60bbf4302e in odbcConvCtoSQL(STATEMENT_INFO&, int, int, char const*, char*, unsigned long, unsigned long, COLUMN_INFO&, COLUMN_INFO&, unsigned long*) () from /opt/ibm/iSeriesAccess/lib64/libcwbodbc.so#2 0x00007f60bbf3635b in STATEMENT_INFO::setParamValues(short*, char*) () from /opt/ibm/iSeriesAccess/lib64/libcwbodbc.so#3 0x00007f60bbf38025 in STATEMENT_INFO::execute() () from /opt/ibm/iSeriesAccess/lib64/libcwbodbc.so#4 0x00007f60bbf38cf0 in STATEMENT_INFO::odbcExecute() () from /opt/ibm/iSeriesAccess/lib64/libcwbodbc.so#5 0x00007f60bbf3905d in SQLExecute () from /opt/ibm/iSeriesAccess/lib64/libcwbodbc.so#6 0x00007f60bd700e90 in SQLExecute () from /usr/lib64/libodbc.so.2
And from ubuntu 13.10 I get this snippet:Program terminated with signal 11, Segmentation fault.#0 0x00007fea75e970c5 in odbcConv_PreConvert_C_CHAR (statement=..., pSource=@0x7fff5102e7b0: 0x7fea8410e910 "",ulSourceLen=@0x7fff5102e7a8: 4294967189, sourceColInfo=...) at odbcconv.cpp:53445344 odbcconv.cpp: No such file or directory.(gdb) bt#0 0x00007fea75e970c5 in odbcConv_PreConvert_C_CHAR (statement=..., pSource=@0x7fff5102e7b0: 0x7fea8410e910 "",ulSourceLen=@0x7fff5102e7a8: 4294967189, sourceColInfo=...) at odbcconv.cpp:5344#1 0x00007fea75e9402e in odbcConvCtoSQL (statement=..., nSourceType=<optimized out>, nTargetType=<optimized out>, pSource=0x7fea8410e910 "",pTarget=0x1b9ae26 "", ulSourceLen=4294967189, ulTargetLen=10, sourceColInfo=..., targetColInfo=..., pResultLen=0x7fff5102ec30)at odbcconv.cpp:17580#2 0x00007fea75e8735b in STATEMENT_INFO::setParamValues (this=0x1ba00f0, pwIndicators=0x1b9ae24, pcParameters=0x1b9ae26 "") at odbcexec.cpp:1547#3 0x00007fea75e89025 in STATEMENT_INFO::execute (this=0x1ba00f0) at odbcexec.cpp:450#4 0x00007fea75e89cf0 in STATEMENT_INFO::odbcExecute (this=0x1ba00f0) at odbcexec.cpp:204#5 0x00007fea75e8a05d in SQLExecute (hStmt=<optimized out>) at odbcexec.cpp:160#6 0x00007fea77fd8409 in SQLExecute (statement_handle=0x1be14b0) at SQLExecute.c:287
Ubuntu with unixODBC 2.3.2 and CentOS with 2.2.14
NME9_Theo_Frieling 270004NME91 PostACCEPTED ANSWER
Re: FIVE BUGS/CAVEATS IN THE ISERIES ACCESS ODBC CONNECTOR FOR LINUX X86_642014-06-27T14:02:14Z in response to AndrewKehrig
So what is the status at the moment? I tried CentOS 6.4, 6.5 but everytime the same segmentation faults. This issue is getting more annoying. Why can't IBM Fix this?
I could open a PMR but I'm not a linux expert and am afraid i cannot answer the linux questions IBM may have for me. But then again TESTING is everything IBM!!!
RossRogers 2700078J671 PostACCEPTED ANSWER
Re: FIVE BUGS/CAVEATS IN THE ISERIES ACCESS ODBC CONNECTOR FOR LINUX X86_642014-06-30T23:25:00Z in response to AndrewKehrig
As regards the `odbcConv_PreConvert_C_CHAR` stacktrace, I was having this issue through the PHP PDO ODBC interface through unix ODBC to IBM's i Access driver and this blog post fixed my problem. It was an upstream problem in the PHP source code. His fix is in the PHP trunk now, but only as of April 2014, which hasn't made it to the Ubuntu packages as of yet.
I just copied the `pdo_odbc.so` in his blog post over top of my own and it worked.
Anyways, the PHP POD ODBC may be your issue if you're seeing a `odbcConv_PreConvert_C_CHAR` stacktrace, but then again it may not.Updated on 2014-06-30T23:26:03Z at 2014-06-30T23:26:03Z by RossRogers