Topic
No replies
A. Panayotov
A. Panayotov
6 Posts
ACCEPTED ANSWER

Pinned topic UTF-8 symbols and IBM JRE 1.7.0 SR3

‏2013-12-27T09:53:28Z |

Greetings,

Summary:

UTF-8 symbols are not recognized properly and eventually omitted on AIX 6.1 + IBM JRE 1.7.0 SR3.

System and tools details:

IBM AIX 6.1
IBM JRE 1.7.0 SR3
Installed locales: en_US; EN_US

Detailed description:

Recently I encountered the following issue: some files containing UTF-8 symbols in their file names could not be transferred over (S)FTP. Further on, the same behavior was observed using the HTTP(S) and FTPS protocols, which suggested that the OS on which the application was deployed had something wrong going on. After further investigation, the eventually missing locale EN_US (according to IBM manuals, this locale is the one supporting EN UTF8) was successfully installed, but unfortunately, nothing changed. It is important to note that the protocol settings are set properly, knowing the fact that on other OS (Windows, RHEL, Solaris, SLES) everything runs flawlessly. I have isolated the following test case:

import java.io.File;
public class Test {
    
    public static void main(String[] args) {
        File test = new File("/tmp/¦•®™Ö ジをご覧ください áéíóúüñ¿");
        System.out.pringln(test.getAbsolutePath());
    }
}

The output is as follows:

/tmp/ࢦࢮ�Ö ジをご覧ください áéíóúüñ¿

As you can see, some characters are omitted and/or not recognized properly.

The overall result is a "File not found" exception which once again suggests improper file names parsing on OS level.

Few more important notes:

1)  The terminal charset is set to UTF-8

2) The used version of the JRE is 1.7.0 SR 3. The reasons behind not using the latest (SR5) versions is that an inconsistency issue was encountered after the update which is nearly impossible to isolate and debug. It seems that IBM JRE 1.7.0 SR5 and AspectJWeaver don't like each other on AIX and something nasty happens. This will be reported in another thread.

3) The application is built using IBM JDK 1.7.0 SR3 and is deployed (and run) with IBM JRE 1.7.0 SR3, so there is no difference between the used java tools.

4) The library java.nio.files.File cannot be used since it is protected because of rather understandable reasons.

 

Should you have any questions, do not hesitate to ask.

 

Sincerely,
A. Panayotov

Updated on 2013-12-27T14:35:37Z at 2013-12-27T14:35:37Z by A. Panayotov