Developing risk indicators

Surveillance Insights supports risk indicators that are written in Java.

Risk indicator developers must use the Java-based framework that is provided with Surveillance Insight.

The following diagram shows the trade framework:

Figure 1. Trade Framework
Diagram showing the trade framework
  1. Run the TradeAnalytics job to execute all of the used risk indicators. For more information, see Executing trade use case implementations
  2. The job gets all of the published trade models from RMS and finds all of the trade risk indicators that are present in the models.
  3. The job then uses the Java reflection API to instantiate the risk indicators by using the implementationClass property that is set on the risk indicator.
  4. All of the risk indicators are expected to extend the TradeRiskIndicator abstract class and implement the detectRisk method in it. This method is called by the TradeAnalytics and the TradeDataSource object, which contains the methods to read the trade data.
  5. The risk indicator implementation then calls the necessary get methods from the TradeDataSource object and executes the risk detection logic.
  6. The results of the detectRisk method are risk evidence and trade evidence. Risk evidence is populated in the RiskEvidence object. These are high-level representations of the detected risk. Each piece of risk evidence must be linked to the corresponding trade evidence that is actual raw data evidence for the risk.
  7. The TradeAnalytics job persists the evidence to the SIFS database

Procedure

  1. Create a risk indicator class.

    Create a new Java project and add a new class to it that extends the TradeRiskIndicator abstract class, as shown in the following example:

    Class MyRiskIndicator extends TradeRiskIndicator {
public void detectRisk(TradeDataSource tds){
  // access data from TradeDataSource
// read configuration parameters
// implement the risk detection logic
//create risk evidence and link them to trade evidence
// set risk evidence and trade evidence in the super class.
}
}

    The name of the class should match the implementationClass property that was set for the risk indicator in Design Studio.

  2. Implement the risk detection logic.

    Implement the detectRisk method that is mandated by the abstract super class. In this method, provide the implementation of the risk detection. The high-level steps are shown in the code snippet in Step 1.

    1. Access the data.

      The data sources for the analytics are provided by the TradeDataSource class. This class contains methods for fetching the various trade data that is ingested into the Financial Transaction Respository.

      APIs for orders and executions (equity market):

      public Dataset<Row> getOrderDataset()
public Dataset<Row> getExecutionDataset()

      APIs for quotes and trade data obtained from the equity market:

      public Dataset<Row> getQuoteDataset()
public Dataset<Row> getTradeDataset()

      APIs for forex transactions and quotes:

      public Dataset<Row> getFXTransactionsDataset()
public Dataset<Row> getFXQuotesDataset()

      APIs for forex transactions and quotes:

      public Dataset<Row> getTradeConfig()

      This class also provides access to the tradeConfig.csv file, which contains ticker-wise configuration parameters such as the thresholds that are used by various risk indicators. Since these thresholds may be different for each ticker, this file provides a way to specify what the thresholds are for each ticker. Surveillance Insights does not provide the entries for all of the tickers in the market. It must be loaded with suitable values before you run the trade use cases. If a ticker is not found in this file, it can be filtered out from the detection logic of the risk indicators which are using the thresholds from this file. See the references section for details on the schema of this file.

    2. Read configuration parameters.

      Risk Indicators configuration properties are provided in the risk model. The trade framework populates the configuration from the model into the RiskIndicatorConfig object while instantiating the risk indicator. This configuration can be accessed by using the getConfig() method in the super class. See the references section for the API details.

    3. Implement the detection logic.

      After fetching the data using the above APIs, implement the logic to detect the risk as per the definition of the risk indicator.

      For indicators which are dependent on the risk evidence generated by other risk indicators, please follow these guidelines:

      • Set the priority property of this indicator in the Design Studio to a value that is higher than that of the indicators which this indicator is dependent upon. This will ensure that this indicator is run after the other dependent indicators are run.
      • Use the getConfig().getAnalysisDate() method to get the date for which the analytics is being run.
      • Use this date to get the evidence for the dependent risk indicators by using the following API:
        TradePersistence tp = new TradePersistence(getSpark());
        Dataset<Row> ds= tp.getRiskEvidence(stringArray,evidenceDate,modelLookupCode)

        Here the stringArray contains the risk indicator IDs of the dependent indicators. These need to be provided as custom properties for the risk indicator in the risk mode.

        The modelLookupCode can be obtained using the API:

        getConfig().getModelLookupCode();
    4. Create and link evidence.

      The output of the detectRisk method is void but if it finds any risk indicators in the data, it sets the results of the detection in the riskEvidence and tradeEvidence datasets in the super class.

      Risk Evidence is a generic definition of proof of risk found in the data that is analyzed by the risk indicator. It contains the indicator ID, date and time of risk, description of the risk indicator in the current context, and so on.

      Trade Evidence is the actual raw data evidence that is related to the risk evidence.

      For example, Risk Evidence for Bulk Order indicates that a bulk order was found at a specific date and time for a specific ticker. The trade evidence indicates the actual orders (raw data) that contributed to the bulk order. Surveillance Insights defines the normalized schema for storing the raw data. The data that is returned by the Trade Data Source APIs can be used for this purpose.

      The RiskEvidence and TradeEvidence are linked with each other through a transactId which needs to be generated by the risk indicator. This is a unique ID across the cluster and can be generated using the Spark 2 function: monotonically_increasing_id().

      The TradeAnalytics job uses this ID to link the RiskEvidence with one or more TradeEvidences in the SIFS database. This will impact the trade chart and the trade report in the Surveillance Insight dashboards.

      While creating risk evidence, the following fields must be set:

      • Name
      • Description: Use <Indicator Name>, <Description> format for this field
      • Ticker
      • EvidenceTime
      • Score: Translate the risk to a score between 0 and 1
      • Involved Party List: Trader list if relevant to the indicator or the partyIds if available. Otherwise, this is null.

      In order to create trade evidence, the framework provides a TradeDataExtractor class that extracts the necessary fields from the Order, Quote, Trade , Execution, and FXTransaction data sets:

      public  static Dataset<TradeEvidence> getExecutionTradeEvidence(Dataset<Row> ds)
      public static Dataset<TradeEvidence> getOrderTradeEvidence(Dataset<Row> ds)
      public static Dataset<TradeEvidence> getTradeEvidence(Dataset<Row> ds)
      public static Dataset<TradeEvidence> getFXTradeEvidence(Dataset<Row> fxTransactions)
      public static Dataset<TradeEvidence> getQuotePriceMovementTradeEvidence(Dataset<Row> quoteDS)

      Pass the raw data records that are extracted from the TradeDataSource to the appropriate method in the TradeDataExtractor, and use the output to setTradeEvidence().

    5. Set risk and trade evidence in the super class.

      The evidence created in the previous step are saved to the class variables. This is done through the following methods in the super class:

      public void setRiskEvidence(Dataset<RiskEvidence> riskEvidence)
public void setTradeEvidence(Dataset<TradeEvidence> tradeEvidence)
  3. Package and deploy the risk indicator.

    The new indicator is packaged as a JAR file and made available to Trade Surveillance by dropping the JAR file in the /home/sifsuser/lib directory.