Skip to main content

alphaWorks  >  Forums  >  Scalable Highly Expressive Reasoner  >  developerWorks

OutOfMemoryError whilst loading large ontology    Point your RSS reader here for a feed of the latest messages in this thread


     

 
 

My developerWorks
 Welcome, Guest
Sign in or register
Permlink Replies: 5 - Pages: 1 - Last Post: Nov 10, 2008 9:10 AM Last Post By: KavithaSrinivas Threads: [ Previous | Next ]
silasdavis

Posts: 6
Registered: Jul 29, 2008 11:11:25 AM
OutOfMemoryError whilst loading large ontology
Posted: Aug 14, 2008 02:01:31 PM
Click to report abuse...   Click to reply to this thread Reply
I am getting an out of memory error whilst trying to load a Large ontology (~6 million lines of OWL). Any clues to why this might be? (I am assuming that this ontology is well within the limits of SHER's capability since it managed 300 million triples)

I can provide you with some details if required. Here's the trace:


Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.util.Arrays.copyOfRange(Arrays.java:3209)
at java.lang.String.<init>(String.java:216)
at java.lang.StringBuffer.toString(StringBuffer.java:585)
at org.eclipse.eodm.rdf.resource.parser.xml.RDFXMLParserImpl.checkStatementValid(RD
FXMLParserImpl.java:876)
at org.eclipse.eodm.rdf.resource.parser.xml.RDFXMLParserImpl.exportStatement(RDFXML
ParserImpl.java:752)
at org.eclipse.eodm.rdf.resource.parser.xml.RDFXMLParserImpl.addNodeElement(RDFXMLP
arserImpl.java:252)
at org.eclipse.eodm.rdf.resource.parser.xml.RDFXMLContentHandler.startElement(RDFXM
LContentHandler.java:149)
at org.apache.xerces.parsers.AbstractSAXParser.startElement(Unknown Source)
at org.apache.xerces.parsers.AbstractXMLDocumentParser.emptyElement(Unknown Source)
at org.apache.xerces.impl.XMLNSDocumentScannerImpl.scanStartElement(Unknown Source)
at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown Source)
at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
at org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source)
at org.eclipse.eodm.rdf.resource.parser.xml.RDFXMLParserImpl.parse(RDFXMLParserImpl.java:138)
at org.eclipse.eodm.owl.resource.parser.impl.OWLParserImpl.addOWLDocument(OWLParserImpl.java:550)
at com.ibm.iodt.sor.translator.EODMTranslator.addOWLDocument(EODMTranslator.java:899)
at com.ibm.iodt.sor.OntologyStoreManager.addOWLDocument(OntologyStoreManager.java:220)
at com.ibm.research.sher.utils.LoadForSHER.loadDocuments(LoadForSHER.java:79)
at com.ibm.research.sher.utils.LoadForSHER.main(LoadForSHER.java:129)
KavithaSrinivas

Posts: 11
Registered: Jul 30, 2008 09:34:58 AM
Re: OutOfMemoryError whilst loading large ontology
Posted: Aug 20, 2008 05:25:47 PM   in response to: silasdavis in response to: silasdavis's post
Click to report abuse...   Click to reply to this thread Reply
Can you give us more details? Is the 6 million lines of OWL a single file? If it is, I'd suggest breaking it up because the parser loads and builds an OWL model in memory. Our standard approach to dealing with this issue is to break up the instance data into many smaller files, and load each one into the database.
silasdavis

Posts: 6
Registered: Jul 29, 2008 11:11:25 AM
Re: OutOfMemoryError whilst loading large ontology
Posted: Nov 08, 2008 07:16:56 PM   in response to: KavithaSrinivas in response to: KavithaSrinivas's post
Click to report abuse...   Click to reply to this thread Reply
I managed to get the ontology to load by splitting my ontology into multiple files. I found I had to include all the OWL class declarations at the beginning of each file.

This was successful. However running:

19 java -Xmx1024M com.ibm.research.sher.utils.QueryAnswering -ai rps rps-sher-config.cfg


Produced:

08-Nov-2008 23:25:11 com.ibm.research.sher.analysis.impl.ConsistencyChecker checkConsistency
INFO: SKIPPING CONSISTENCY CHECK since KB cannot entail contradictions
rps is consistent.


Which doesn't tell me where the contradiction might be. Any suggestions?
KavithaSrinivas

Posts: 11
Registered: Jul 30, 2008 09:34:58 AM
Re: OutOfMemoryError whilst loading large ontology
Posted: Nov 09, 2008 08:41:30 PM   in response to: silasdavis in response to: silasdavis's post
Click to report abuse...   Click to reply to this thread Reply
SHER is stating that there are no possible ways of producing an inconsistency in your ontology, given the constructs present in it (e.g., it has no negation?). So the message says that SHER recognized that, and is skipping the consistency check.
silasdavis

Posts: 6
Registered: Jul 29, 2008 11:11:25 AM
Re: OutOfMemoryError whilst loading large ontology
Posted: Nov 10, 2008 09:06:29 AM   in response to: KavithaSrinivas in response to: KavithaSrinivas's post
Click to report abuse...   Click to reply to this thread Reply
Ah, I had interpreted the message as "Since the ontology entails contradictions, SHER will not perform a consistency check". As you say, I suppose the constructs do not allow contradictions for this part of the ontology.

I'll let you know how I get on with querying...
KavithaSrinivas

Posts: 11
Registered: Jul 30, 2008 09:34:58 AM
Re: OutOfMemoryError whilst loading large ontology
Posted: Nov 10, 2008 09:10:27 AM   in response to: silasdavis in response to: silasdavis's post
Click to report abuse...   Click to reply to this thread Reply
Great. Do let us know if you run into trouble querying.

Point your RSS reader here for a feed of the latest messages in all forums