Topic
  • 2 replies
  • Latest Post - ‏2013-11-05T17:02:14Z by MQVA_javier_vidal
MQVA_javier_vidal
MQVA_javier_vidal
6 Posts

Pinned topic Error in TextExtractor Operator

‏2013-10-31T15:59:55Z |

Hello, I'm trying to use a text extractor operator and I get the following error on initialization. Any ideas?

Thanks

 

Operator: com.ibm.streams.text.analytics::TextExtract Op_5
 
Source: 
(stream<Tweet_Annotated> Op_5_out0) as Op_5  = TextExtract(Op_3_out0)
{
param 
inputDoc:"text";
languageCode:"es";
tokenizer:"multilingual";
moduleName:"twitterStatusesAnt";
modulePath:"/home/streamsadmin";
}

OPERATOR TRACE:

31 Oct 2013 16:53:51.342 [21072] INFO spl_pe M[BackoffSpinner.cpp:setYieldBehaviour:50]  - BackoffSpinner: use pthread_yield()
31 Oct 2013 16:53:51.343 [21072] INFO spl_pe M[PEImpl.cpp:initialize:269]  - Initializing PE performance metric collection...
31 Oct 2013 16:53:51.344 [21072] INFO spl_pe M[PEImpl.cpp:initialize:273]  - Initialized PE performance metric collection.
31 Oct 2013 16:53:51.344 [21072] INFO spl_pe M[PEImpl.cpp:process:646]  - Start processing...
31 Oct 2013 16:53:51.346 [21072] DEBUG spl_metric M[OperatorMetricsImpl.cpp:~OperatorMetricsImpl:108]  - Deallocating metrics data structure for PE '129, ' operator 'Op_5'
31 Oct 2013 16:53:51.347 [21072] INFO spl_metric M[OperatorMetricsImpl.cpp:dumpMetricAtExit:147]  - The Number of tuples processed (port 0) metric of the Op_5 operator has a value of 0 at exit.
31 Oct 2013 16:53:51.347 [21072] INFO spl_metric M[OperatorMetricsImpl.cpp:dumpMetricAtExit:147]  - The Number of tuples dropped (port 0) metric of the Op_5 operator has a value of 0 at exit.
31 Oct 2013 16:53:51.347 [21072] INFO spl_metric M[OperatorMetricsImpl.cpp:dumpMetricAtExit:147]  - The Number of tuples currently queued (port 0) metric of the Op_5 operator has a value of 0 at exit.
31 Oct 2013 16:53:51.348 [21072] INFO spl_metric M[OperatorMetricsImpl.cpp:dumpMetricAtExit:147]  - The Number of window punctuations processed (port 0) metric of the Op_5 operator has a value of 0 at exit.
31 Oct 2013 16:53:51.348 [21072] INFO spl_metric M[OperatorMetricsImpl.cpp:dumpMetricAtExit:147]  - The Number of final punctuations processed (port 0) metric of the Op_5 operator has a value of 0 at exit.
31 Oct 2013 16:53:51.348 [21072] INFO spl_metric M[OperatorMetricsImpl.cpp:dumpMetricAtExit:147]  - The Number of window punctuations queued (port 0) metric of the Op_5 operator has a value of 0 at exit.
31 Oct 2013 16:53:51.348 [21072] INFO spl_metric M[OperatorMetricsImpl.cpp:dumpMetricAtExit:147]  - The Number of final punctuations queued (port 0) metric of the Op_5 operator has a value of 0 at exit.
31 Oct 2013 16:53:51.349 [21072] INFO spl_metric M[OperatorMetricsImpl.cpp:dumpMetricAtExit:147]  - The Size of the queue for a threaded port (port 0) metric of the Op_5 operator has a value of 0 at exit.
31 Oct 2013 16:53:51.349 [21072] INFO spl_metric M[OperatorMetricsImpl.cpp:dumpMetricAtExit:147]  - The Number of tuples submitted (port 0) metric of the Op_5 operator has a value of 0 at exit.
31 Oct 2013 16:53:51.349 [21072] INFO spl_metric M[OperatorMetricsImpl.cpp:dumpMetricAtExit:147]  - The Number of window punctuations submitted (port 0) metric of the Op_5 operator has a value of 0 at exit.
31 Oct 2013 16:53:51.350 [21072] INFO spl_metric M[OperatorMetricsImpl.cpp:dumpMetricAtExit:147]  - The Number of final punctuations submitted (port 0) metric of the Op_5 operator has a value of 0 at exit.
31 Oct 2013 16:53:51.350 [21072] DEBUG spl_operator M[PEImpl.cpp:instantiateOperators:462]  - Exception received during instantiateOperator(): SPL::SPLRuntimeException (pe().getJavaVM() returned NULL) at 'static SPL::JNI::JVMControl* SPL::JNI::JVMControl::getJVM(const char*, SPL::rstring*, size_t, jboolean*)' [./src/SPL/Runtime/Toolkit/JavaOp.cpp:1239]
Exception Code=NoMessageId with 0 substitution text strings
 
additional data -- 
================= BACKTRACE ===============
/mnt/Streams/InfoSphereStreams/system/impl/bin/../../../lib/libstreams-apputils.so.0(Distillery::DistilleryException::DistilleryException(std::string const&, std::string const&, std::string const&, Distillery::DistilleryExceptionCode const&, std::vector<std::string, std::allocator<std::string> > const&, std::string const&)+0x81) [0x2b5f4424b241]
/mnt/Streams/InfoSphereStreams/system/impl/bin/../../../lib/libstreams-spl-runtime.so.0(SPL::SPLRuntimeException::SPLRuntimeException(std::string const&, std::string const&, Distillery::DistilleryExceptionCode const&, std::vector<std::string, std::allocator<std::string> > const&, std::string const&)+0x6c) [0x2aaaab55b12c]
/mnt/Streams/InfoSphereStreams/system/impl/bin/../../../lib/libstreams-spl-runtime.so.0(SPL::SPLRuntimeException::throwWithContext(char const*, char const*, char const*, int, ...)+0x22a) [0x2aaaab55b82a]
/mnt/Streams/InfoSphereStreams/system/impl/bin/../lib/libstreams-stdtk-javaop.so.0(SPL::JNI::JVMControl::getJVM(char const*, SPL::rstring*, unsigned long, unsigned char*)+0x2e5) [0x2aaaaaf46975]
/mnt/Streams/Workspaces/Big_Data_Examples/TwitterListener/output/twitter.application.TwitterListener/Distributed/bin/twitter.application.TwitterListener-e.dpe(SPL::_Operator::Op_5$OP::Op_5$OP()+0x34f) [0x2aaaaace6667]
/mnt/Streams/Workspaces/Big_Data_Examples/TwitterListener/output/twitter.application.TwitterListener/Distributed/bin/twitter.application.TwitterListener-e.dpe [0x2aaaaace87ee]
/mnt/Streams/InfoSphereStreams/system/impl/bin/../../../lib/libstreams-spl-runtime.so.0(SPL::Operator::instanceOf(std::string const&)+0xb1) [0x2aaaab6547c1]
/mnt/Streams/InfoSphereStreams/system/impl/bin/../../../lib/libstreams-spl-runtime.so.0 [0x2aaaab8f8797]
/mnt/Streams/InfoSphereStreams/system/impl/bin/../../../lib/libstreams-spl-runtime.so.0 [0x2aaaab8fa240]
/mnt/Streams/InfoSphereStreams/system/impl/bin/../../../lib/libstreams-spl-runtime.so.0 [0x2aaaab78099d]
/mnt/Streams/InfoSphereStreams/system/impl/bin/../../../lib/libstreams-spl-runtime.so.0 [0x2aaaab8f4988]
/mnt/Streams/InfoSphereStreams/system/impl/bin/../../../lib/libstreams-runtime.so.0(Streams::PECServer::runPE()+0x504) [0x2b5f42f3da34]
/mnt/Streams/InfoSphereStreams/system/impl/bin/../../../lib/libstreams-apputils.so.0(Distillery::threadStart(void*)+0x5f) [0x2b5f442ee2ff]
/lib64/libpthread.so.0 [0x3ae4c0677d]
/lib64/libc.so.6(clone+0x6d) [0x3ae40d49ad]
===========================================
31 Oct 2013 16:53:51.351 [21072] ERROR #splapptrc,J[31],P[129],Op_5,spl_operator M[PEImpl.cpp:instantiateOperators:463]  - CDISR5030E: An exception occurred during the execution of the Op_5 operator. The exception is: pe().getJavaVM() returned NULL
31 Oct 2013 16:53:51.351 [21072] ERROR #splapptrc,J[31],P[129],Op_5,spl_pe M[PEImpl.cpp:process:675]  - CDISR5079E: An exception occurred during the processing of the processing element. The error is: pe().getJavaVM() returned NULL.
31 Oct 2013 16:53:51.352 [21072] INFO spl_pe M[PEImpl.cpp:shutdown:908]  - Shutdown request received by PE...
31 Oct 2013 16:53:51.352 [21072] INFO spl_pe M[PEImpl.cpp:shutdown:912]  - shutdownRequested set to true...
31 Oct 2013 16:53:51.352 [21072] DEBUG spl_pe M[PECleaners.cpp:prepareToFinalize:97]  - Shutting down PE ports while still initializing
31 Oct 2013 16:53:51.353 [21072] DEBUG spl_pe M[PEImpl.cpp:preparePortsForTermination:747]  - Shutting down PE ports...
31 Oct 2013 16:53:51.353 [21072] DEBUG spl_pe M[PEImpl.cpp:preparePortsForTermination:750]  - Completed shuting down PE ports.
31 Oct 2013 16:53:51.353 [21072] INFO spl_pe M[PEImpl.cpp:shutdown:919]  - Exit PEImpl::shutdown()
31 Oct 2013 16:53:51.354 [21072] INFO spl_pe M[PECleaners.cpp:finalizeImpl:116]  - Closing ports...
31 Oct 2013 16:53:51.354 [21072] DEBUG spl_pe M[PETransportPorts.cpp:close:418]  - Closing all PE input ports...
31 Oct 2013 16:53:51.354 [21072] DEBUG spl_pe M[PETransportPorts.cpp:close:420]  - All PE input ports are closed.
31 Oct 2013 16:53:51.354 [21072] DEBUG spl_pe M[PETransportPorts.cpp:close:570]  - Closing all PE output ports
31 Oct 2013 16:53:51.355 [21072] DEBUG spl_pe M[PETransportPorts.cpp:close:572]  - All PE output ports are closed and transport senders have been terminated
31 Oct 2013 16:53:51.355 [21072] INFO spl_pe M[PECleaners.cpp:finalizeImpl:118]  - Closed all ports...
31 Oct 2013 16:53:51.355 [21072] INFO spl_pe M[PEImpl.cpp:process:695]  - Following operators failed at runtime: 'Op_5'.  Throwing...
31 Oct 2013 16:53:51.356 [21072] ERROR #splapptrc,J[31],P[129],Op_5,spl_operator M[PEImpl.cpp:process:696]  - CDISR5053E: Runtime failures occurred in the following operators: Op_5.
31 Oct 2013 16:53:51.367 [21075] TRACE VIEW M[PEVisualizer.cpp:run:99]  - PEVisualizer::run: enter
31 Oct 2013 16:53:51.389 [21075] TRACE VIEW M[PEVisualizer.cpp:run:108]  - PEVisualizer::run: wait for data
31 Oct 2013 16:53:51.482 [21020] INFO spl_pe M[PEImpl.cpp:~PEImpl:387]  - cleaning up...
31 Oct 2013 16:53:51.551 [21075] TRACE VIEW M[PEVisualizer.cpp:run:111]  - PEVisualizer::run: shutdown exit
31 Oct 2013 16:53:51.552 [21020] DEBUG spl_pe M[PETransportPorts.cpp:close:570]  - Closing all PE output ports
31 Oct 2013 16:53:51.552 [21020] DEBUG spl_pe M[PETransportPorts.cpp:close:572]  - All PE output ports are closed and transport senders have been terminated
31 Oct 2013 16:53:51.552 [21020] DEBUG spl_pe M[PETransportPorts.cpp:~PETransportOPortCollection:578]  - Destroying transport senders...
31 Oct 2013 16:53:51.553 [21020] DEBUG spl_pe M[PETransportPorts.cpp:~PETransportOPortCollection:580]  - Destroyed all transport senders.
31 Oct 2013 16:53:51.553 [21020] DEBUG spl_pe M[PETransportPorts.cpp:close:418]  - Closing all PE input ports...
31 Oct 2013 16:53:51.553 [21020] DEBUG spl_pe M[PETransportPorts.cpp:close:420]  - All PE input ports are closed.
31 Oct 2013 16:53:51.553 [21020] DEBUG spl_pe M[PETransportPorts.cpp:~PETransportIPortCollection:426]  - Destroying transport receiver...
31 Oct 2013 16:53:51.554 [21020] DEBUG spl_pe M[PETransportPorts.cpp:~PETransportIPortCollection:429]  - Destroyed transport receiver.
31 Oct 2013 16:53:51.554 [21020] DEBUG spl_metric M[PEMetricsImpl.cpp:~PEMetricsImpl:87]  - Deallocating metrics data structure for PE '129'
31 Oct 2013 16:53:51.555 [21020] ERROR spl_metric M[PEMetricsImpl.cpp:dumpMetricAtExit:107]  - The Number of tuples processed (port 0) metric of the processing element has a value of 0 at exit.
31 Oct 2013 16:53:51.555 [21020] ERROR spl_metric M[PEMetricsImpl.cpp:dumpMetricAtExit:107]  - The Number of bytes processed (port 0) metric of the processing element has a value of 0 at exit.
31 Oct 2013 16:53:51.556 [21020] ERROR spl_metric M[PEMetricsImpl.cpp:dumpMetricAtExit:107]  - The Number of window punctuations processed (port 0) metric of the processing element has a value of 0 at exit.
31 Oct 2013 16:53:51.556 [21020] ERROR spl_metric M[PEMetricsImpl.cpp:dumpMetricAtExit:107]  - The Number of final punctuations processed (port 0) metric of the processing element has a value of 0 at exit.
31 Oct 2013 16:53:51.557 [21020] ERROR spl_metric M[PEMetricsImpl.cpp:dumpMetricAtExit:107]  - The Number of tuples submitted (port 0) metric of the processing element has a value of 0 at exit.
31 Oct 2013 16:53:51.557 [21020] ERROR spl_metric M[PEMetricsImpl.cpp:dumpMetricAtExit:107]  - The Number of bytes submitted (port 0) metric of the processing element has a value of 0 at exit.
31 Oct 2013 16:53:51.558 [21020] ERROR spl_metric M[PEMetricsImpl.cpp:dumpMetricAtExit:107]  - The  Number of window punctuations submitted (port 0) metric of the processing element has a value of 0 at exit.
31 Oct 2013 16:53:51.559 [21020] ERROR spl_metric M[PEMetricsImpl.cpp:dumpMetricAtExit:107]  - The  Number of final punctuations submitted (port 0) metric of the processing element has a value of 0 at exit.
31 Oct 2013 16:53:51.559 [21020] ERROR spl_metric M[PEMetricsImpl.cpp:dumpMetricAtExit:107]  - The A count of previously established connections that were subsequently detected as being broken. This is an incrementing counter over the lifetime of a PE's current process (port 0) metric of the processing element has a value of 0 at exit.
31 Oct 2013 16:53:51.560 [21020] ERROR spl_metric M[PEMetricsImpl.cpp:dumpMetricAtExit:107]  - The The current number of required connections that are not connected and are in the process of connecting to their receiver (port 0) metric of the processing element has a value of 0 at exit.
31 Oct 2013 16:53:51.560 [21020] ERROR spl_metric M[PEMetricsImpl.cpp:dumpMetricAtExit:107]  - The The current number of optional connections that are not connected and are in the process of connecting to their receiver (port 0) metric of the processing element has a value of 0 at exit.
31 Oct 2013 16:53:51.561 [21020] ERROR spl_metric M[PEMetricsImpl.cpp:dumpMetricAtExit:107]  - The Number of tuples transmitted (port 0) metric of the processing element has a value of 0 at exit.
31 Oct 2013 16:53:51.561 [21020] ERROR spl_metric M[PEMetricsImpl.cpp:dumpMetricAtExit:107]  - The Number of bytes transmitted (port 0) metric of the processing element has a value of 0 at exit.
31 Oct 2013 16:53:51.562 [21020] ERROR spl_metric M[PEMetricsImpl.cpp:dumpMetricAtExit:107]  - The Number of connections (port 0) metric of the processing element has a value of 0 at exit.
  • Stan
    Stan
    76 Posts

    Re: Error in TextExtractor Operator

    ‏2013-11-04T19:50:47Z  

    I assume this in an imcomplete code snippet and the 'use com.ibm.streams.text.analytics::*;' 

    statement is in the SPL code.  If not please include it.

    The smoking gun here is:    [PEImpl.cpp:instantiateOperators:463]  - CDISR5030E: An exception occurred during the execution of the Op_5 operator. The exception is: pe().getJavaVM() returned NULL

    Check that a supported version of Java is installed and that JAVA_HOME is set properly to located the JDK (not a JRE). 

    http://pic.dhe.ibm.com/infocenter/streams/v3r1/topic/com.ibm.swg.im.infosphere.streams.install-admin.doc/doc/ibminfospherestreams-install-prerequisites-java.html

    http://pic.dhe.ibm.com/infocenter/streams/v3r1/topic/com.ibm.swg.im.infosphere.bigdata.teda.doc/doc/acc_teda_install_plan_env.html

    Also check that the ULIMIT for max User Processes is large enough to allow the new JVM process to start.

     

     

  • MQVA_javier_vidal
    MQVA_javier_vidal
    6 Posts

    Re: Error in TextExtractor Operator

    ‏2013-11-05T17:02:14Z  
    • Stan
    • ‏2013-11-04T19:50:47Z

    I assume this in an imcomplete code snippet and the 'use com.ibm.streams.text.analytics::*;' 

    statement is in the SPL code.  If not please include it.

    The smoking gun here is:    [PEImpl.cpp:instantiateOperators:463]  - CDISR5030E: An exception occurred during the execution of the Op_5 operator. The exception is: pe().getJavaVM() returned NULL

    Check that a supported version of Java is installed and that JAVA_HOME is set properly to located the JDK (not a JRE). 

    http://pic.dhe.ibm.com/infocenter/streams/v3r1/topic/com.ibm.swg.im.infosphere.streams.install-admin.doc/doc/ibminfospherestreams-install-prerequisites-java.html

    http://pic.dhe.ibm.com/infocenter/streams/v3r1/topic/com.ibm.swg.im.infosphere.bigdata.teda.doc/doc/acc_teda_install_plan_env.html

    Also check that the ULIMIT for max User Processes is large enough to allow the new JVM process to start.

     

     

    Thanks Stan!!

    My problem is solved but in fact i'm not sure how. It was not the java problem because I have a Java primitive operator in the same application, and also a JSONHelper that also uses java. I changed the ulimit, but I have also been reviewing the TextOpetator samples and I made several changes..

     

    Thanks again.