IBM Accelerator for Machine Data Analytics, Part 2: Speeding up analysis of new log types

Understand the code

Review the code in email.aql and extractor_email.aql, shown in Listing 3.

                
|--------10--------20--------30--------40--------50--------60--------70--------80--------|
module email;
                
create view email_base as 
select 
R.match 
from 
Regex(/([A-Za-z.\d;'\-<>]+\@[A-Za-z.\d;'\-<>]+(?=(?=,)\s*[A-Za-z.\@\d';\
-<>]+|))/, Document.text) R;
create view toEmail as 
select 
e.match
from
email_base e
where MatchesRegex(/To:\s/, LeftContext(e.match, 4));
                
create view To as 
select D.match as span, GetText(D.match) as text, GetString('To') as field_type
from toEmail D;

export view To;

create view fromEmail as 
select 
e.match
from
email_base e
where MatchesRegex(/From:\s/, LeftContext(e.match, 6));

create view From as 

select D.match as span, GetText(D.match) as text, GetString('From') as field_type
from fromEmail D;    

export view From;                           
            

Observe the following in the code.

• A base view email_base was created.
• A view fromEmail is created to pick emails representing the sender.
• A view From is created the view fromEmail. This is the final view to export!
• A similar toEmail view is created to pick emails representing the receiver.
• A view called To is created in the view toEmail. This is the final view to export!

Note: The final views that are exported have the following simple naming convention that must be followed.

• The view must contain a field called span, representing the span where the value is found.
• The view must contain a field called text, representing the text value found.
• The view must contain a field called field_type, representing the view.

You will soon get some insights into how these naming conventions are used, but first, you can publish the custom application.

11 of 18 | Previous | Next

Comments

Back to top

Table of contents

• Before you start
• Variety in machine data analysis
• The situation at a fictitious Sample Outdoors company
• Ten features to accelerate machine data analysis for new log types
• Prepare the data for the new log type
• Out-of-the-box support
• Take control! Prepare for customization
• Understand the Extraction application
• Peek into the tooling
• Create your own email log type
• Understand the code
• Understand the wiring under the hood
• Publish the customized application
• New log type in action!
• Conclusion
• Download
• Resources
• About the author

---

11 of 18 | Previous | Next

Dig deeper into Information Management on developerWorks

IBM SmartCloud trial. No charge.

Unleash the power of hybrid cloud computing today!

---

Public and private solutions in one trial.

Special offers