IBM Support

Warning ERROR_TOKEN with COBOL syntax similar to MY-VAL(MY-IDX,2) in IBM Developer for z/OS

Troubleshooting


Problem

Warning "ERROR_TOKEN unexpected tokens ignored" is displayed when editing a COBOL program in IBM Developer for z/OS editors.

Symptom

COBOL syntax would be similar to the following:

MOVE 5 TO MY-VAL(MY-IDX,2).


raises a warning:

"ERROR_TOKEN unexpected tokens ignored"


If many of these warnings appear in the code, it can be disturbing for the developer.

Cause

This is working as designed and follows the COBOL rules (see links section below ). The IBM COBOL compiler will also give a warning:
  
IGYPS0001-W A blank was missing before character "1" in column 30.
A blank was assumed.

Resolving The Problem

The problem can be solved by using an IDz preprocessor mechanism.
This technote gives an example.
 
  • When the member is opened in the IDz COBOL Editor, a warning is displayed.


 
  • A preprocessor is configured on the property group:


 
  • Editor preferences are set to not ignore spaces: uncheck Ignore changes to tab and space characters in Preferences > COBOL > Editor > Preprocessor Integration and also set for the LPEX and/or PL/I Editor.


 
  • Preprocessor is invoked



 
  • Warning disappears and COBOL Editor shows the difference with the original code:






Java code of the preprocessor

The main class:
  • reads the COBOL member line by line
  • add a blank if needed using Java regular expression
 

  public static void main(String[] args) {
File in;
File out;
  OutputStreamWriter writer = null;
  BufferedReader reader = null;
  PrintStream log = null;

  try {
  log = new PrintStream(
  "c:\\temp\\mycblpreproc.log");
  log.println("JAVA PREPROCESSOR start");
  log.println("reading from " + args[0]);
  log.println("writing to " + args[1]);
  in = new File(args[0]);
  out = new File(args[1]);

  if (in.exists()) {
  writer = null;
  reader = new BufferedReader(new InputStreamReader(
      new FileInputStream(in),
      Charset.forName("UTF-8").newDecoder()));

  writer = new OutputStreamWriter(
      new FileOutputStream(out),
      Charset.forName("UTF-8").newEncoder()
);

String strLine;

  // read the file line by line
  while ((strLine = reader.readLine()) != null) {
  log.println("read line=" + strLine);
  CobolLine cblLine = new CobolLine(strLine);
  //if line is not a comment then we fix
  //compile warning IGYPS0001
  if (!cblLine.isComment())
strLine=cblLine.fixCompileWarningIGYPS0001();
  log.println("write line=" + strLine);
writer.write(strLine);
  writer.write("\n");

}
writer.flush();
  log.println("JAVA PREPROCESSOR end");
}
  } catch (FileNotFoundException e) {
  // TODO Auto-generated catch block
e.printStackTrace();
  } catch (IOException e) {
  // TODO Auto-generated catch block
e.printStackTrace();
  // } catch (InterruptedException e) {
  // // TODO Auto-generated catch block
  // e.printStackTrace();
  } finally {
  try {
  if (writer != null)
writer.close();
  if (reader != null)
reader.close();
  if (log!=null)
log.close();
  } catch (IOException e) {
  // TODO Auto-generated catch block
e.printStackTrace();
}
}
}
 

  /**
* change occurrences of MY-ARRAY(MYS-IDX,2)
* to MYARRAY(MYS-IDX, 2)
* regular expression is
*  $1=(  
*  followed by $2=([a-zA-Z0-9-]+) -> characters a to Z and/or numbers and/or -
*  followed by $3=([,])  -> ,
*  followed by $4=(\\d+) -> a number
*  followed by $5=([)])  -> )
*
* @return the line modified
*/
  public String fixCompileWarningIGYPS0001(){
  String regexIGYPS0001 = "([(])([a-zA-Z0-9-]+)([,])(\\d+)([)])";
  return line.replaceAll(regexIGYPS0001, "$1$2$3 $4$5");
}



Known issues
  • The above code does not check if the modified line (ie with added blanks) goes over the 72 column.
  • The preprocessor should not be configured to ignore spaces otherwise it will not detect any differences before and after preprocessing.



To test the attached preprocessor sample

The attached jar file can be used as-is, but it will attempt to create a log file in c:\temp\mycblpreproc.log
  1. Save the jar in c:\temp\mycblpreproc.jar for example.
  2. Configure it as Local preprocessor on your property group:



with these settings:
 
Preprocessor name <your IBM JDK path>, for example C:\Users\IBM_ADMIN\IBM\SDP9\jdk\bin\javaw.exe
Preprocessor arguments -classpath c:\temp\mycblpreproc.jar com.ibm.support.rdz.preprocessor.MyCobolPreprocessor "${resource_loc}" "${resource_fn}.cee"
Preprocessor output filename ${resource_fn}.cee



To modify the attached preprocessor sample

Import the jar file in RDz as Java project:
  • Menu File > Import > General > Existing projects into Workspace
  • Select "Select archive file"


 
  • Click Finish


You can then modify the code and export it again to a jar file:
  • Right-click on the Java project MyRDzPreprocessors and select menu Export
  • Select menu Java > Jar file

Disclaimer

All source code and/or binaries attached to this document are referred to here as "the Program". IBM is not providing program services of any kind for the Program. IBM is providing the Program on an "AS IS" basis without warranty of any kind. IBM WILL NOT BE LIABLE FOR ANY ACTUAL, DIRECT, SPECIAL, INCIDENTAL, OR INDIRECT DAMAGES OR FOR ANY ECONOMIC CONSEQUENTIAL DAMAGES (INCLUDING LOST PROFITS OR SAVINGS), EVEN IF IBM, OR ITS RESELLER, HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

[{"Type":"MASTER","Line of Business":{"code":"LOB35","label":"Mainframe SW"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"SSTRMM","label":"IBM Developer for z\/OS"},"ARM Category":[{"code":"a8m0z00000009EiAAI","label":"IBM Developer for Z\/OS-\u003EIDz Client-\u003EEditors-\u003ECobol Editor-\u003EContent Assist"}],"ARM Case Number":"","Platform":[{"code":"PF033","label":"Windows"}],"Version":"All Versions"}]

Product Synonym

idz

Document Information

Modified date:
28 September 2023

UID

swg21683178

Page Feedback