Sunday 23 January 2022

OUAF: Plug-in Driven Batch Programs (Part…1)

OUAF:  Plug-in Driven Batch Programs (Part…1)

Usually, OUAF batch programs are written in Java. Lately Oracle has introduced a feature to build batch programs using plugins that are written either in scripting or groovy.

With Plug-in driven batch program feature, the implementer shall create custom batch programs in Oracle utilities SaaS products such as CCS (Customer Cloud Service), MSCS (Meter Solution Cloud Service) and WACS (Work and Asset Cloud Service). This feature is available for on-premises OUAF (C2M, MDM, WAM, MWM) products as well.

OUAF framework provides base background processes that can call series of algorithms to carry out specific business logic that needs to be performed. These are called plug-in driven background processes.

 There are two types of plug-in driven batch processes supported by OUAF:

 

(1)  Processing System Records:  Acts on OUAF entity records that are stored in the database tables. These types of processes require SQL to select the records along with the logic to process the records.

 

(2)  Uploading Records - Imports data from a file and adds new entity records in the database. These types of processes require an algorithm to map the data from the file to appropriate new OUAF entity records to be created in system

 

 Processing System Records

OUAF provides the two different batch process templates to support plug-ins for selecting and processing the records:
 

(a)  Ad-hoc Process: Implementer can create specific custom business logic that needs to be performed on a group of records.  This type of custom batch program shall be created by duplicating base batch control Plug-in Driven Generic Template (F1-PDBG).

 

(b)  Extract Process: Implementer shall use this to produce file extracts that needs to be integrated with external systems. This type of custom batch program shall be created by duplicating base batch control Plug-in Driven Extract Template (F1-PDBEX). 


Ad-hoc Process: Base batch control Plug-in Driven Generic Template (F1-PDBG). User can duplicate this to create custom batch program.

  

Plug-in Driven Batch process supports the following systems events. Implementer shall create custom algorithms and plug into the appropriate system event spots to meet business requirements.




 

System Event

Algorithm usage

Select Records

Record set selection. Used to define the SQL to use to select the records to process

Process Record

Process specific record in record set

Post Processing

What additional processing is needed for successful jobs. This is called after all threads are completed

Error Post-Processing

What additional processing is needed for jobs in error. This is called if the batch process fails to complete due to an error.

File Upload

File Input Processing (Replaces Select Records). This is used to upload records from a file

Level of Service

Determine the Level of Service provided by a batch control.

  

(i)   Select Records System Event: This algorithm will define the SQL as parameter. The batch job will directly access the SQL parameter value in the metadata. This algorithm contains 3 parameters:

  • SQL Statement - SQL statement with record selection criteria (supported with pre-defined bind parameters). Note that implementer is responsible for performance of the SQL statement
  • Strategy - how the records will be processed (Job-Level SQL Select or Thread-Level SQL Select)
  • Key field - used to denote records to process (usually, the primary Key of the MO or Table)
           Example:



(ii)    Process Record – System Event: This algorithm process records passed from ‘Select Records’ Plug-in. The algorithm plugged into this spot is responsible for doing the work for each record based on the desired logic. This is implemented as a ‘Service Script’ and may contain logic to process database transactions and optionally output records to a file. This plug-in spot has the following hard parameters:

 

Hard Parameter

Parameter Usage

parm/hard/*

Hard parameters (including ids)

parm/hard/isFirst

Used for header processing (Set to false for non-extract programs)

parm/hard/isLast:

Used for footer processing (Set to false for non-extract programs)

parm/hard/selectedFields/*

Fields passed from SQL

 

(iii)  Post Processing – System Event: This algorithm executes after all the threads of a given batch job have completed. This allows for some special processing to occur at the end of a batch job that is completed successfully. Example could be another dependent batch job can be kicked off after successful end of batch job.

(iv) Error Post-Processing - System Event: This algorithm executes when a batch process encounters an error that causes execution to stop. This allows for some special processing to occur to handle the failure of the batch job. Example could be an outbound message service may be invoked to perform a task related to the failure. 

(v)  File Upload - System Event: This algorithm is used if implementer needs to build an import process from a file to the product objects/MO. This algorithm associated with the batch control is responsible for using provided APIs to read the content of the file and store the data in appropriate table (eg: staging table). 

(vi) Level of Service – System Event: This algorithm is used to check the performance of the batch process whether is running as expected or it could analyze the data processed by the background process to assess quality of the data. This algorithm will return following value to indicate the 'level of service'.

  • Normal - the algorithm did not detect any issues
  • Warning - the algorithm found some issues that may or may not indicate a problem
  • Error - the algorithm found some issues that should be investigated
  • Disabled - the algorithm could not properly execute the level of service logic

Monday 25 February 2019

Oracle Utilities Customer To Meter (C2M) - Why is it called “Integration Free Single Instance System”?

    This post describes the difference between separate instances of CCB + MDM and single instance C2M and explains how the internal integration is done in C2M without having SOA middleware layer.

    Prior to the advent of C2M, the CC&B and MDM systems are installed separately (Database and Application Server) and Oracle SOA middleware is used for integrating these systems. There exists a CCB - MDM integration pack to glue these systems.

    When a CCB objects is created or updated, underlying MO audit algorithm is used to create corresponding outbound sync request. This will be sent to MDM via SOA middleware.  CCB - MDM productized integration pack carries out the integration / synchronization of following entities between these two systems using SOA middleware infrastructure and BEPL process orchestration.
From CCB
To MDM
CCB Person
MDM Contact
CCB Service Agreement
MDM Usage Subscription
CCB Service Point
MDM Service Point
CCB Meter Information
MDM Device
CCB Meter Configuration
MDM Device Configuration
CCB SP Meter History
MDM Installation Event
CCB Bill Cycle
MDM Measurement Cycle
CCB Billable Usage request/response
MDM Bill Determinant request/response

The below diagram depicts SOA middleware layer used between CCB and MDM systems for such integrations. 



    As Oracle Utilities C2M being a single instance system, it reduces complexity and does NOT use SOA middleware and productized CCB - MDM integration pack for master data synchronization and integration.

    C2M system uses X1 OUAF layer entities (BO, Algorithms, Extendable Lookups, Master configurations) for data interaction.  The X1 layer acts as internal middleware between CCB (C1) and MDM (D1) for data synchronization. This new X1 is purely a logical layer and does not have any MOs in it.   












    In C2M, data synchronization between CCB and MDM related entities are preformed using the following approaches. 

       Internal Data Synchronization

       Joint Maintenance Synchronization

       Joint Maintenance Superset Script Synchronization 

The below diagram depicts C2M integration free data synchronization between CCB and MDM entities. 



 1. Internal data synchronization approach: Audit algorithms on CCB entities are used for data synchronization. MO audit algorithms creates specific outbound sync requests which is similar to what is practiced in CCB - MDM separate instances integration. The following entities are synchronized using this approach.

 
From CCB
To MDM
C1 Person
D1 Contact
C1 Service Agreement
D1 Usage Subscription
C1 Bill Cycle
D1 Bill Cycle
C1 Contract Option
D1 Dynamic Option
C1 Contract Option Event
D1 Dynamic Option Event









 2. Joint Maintenance Synchronization Approach: This approach is used to synchronize entities having single user interface but have different MOs in CCB(C1) and MDM(D1).

From
To
D1 Service Point
C1 Service Point
D1 Service Point
C1 Bill Cycle
C1 Premise
D1 Service Point
D1 TOU Group
C1 TOU Group
D1 TOU Map
C1 TOU Map
D1 TOU Map Template
C1 TOU Map Template
D1 TOU Map Type
C1 TOU MAP Type
D1 Device Type
C1 Meter Type
 









       
  • Service Point: Pre-Processing algorithm on MDM Service Point BO used to add, update or delete CCB Service Point.  MDM Service Point BO audit algorithm is used to update CCB Bill Cycle due to MDM SP measurement cycle change 
  • Premise: Audit algorithm on CCB Premise MO is used to update MDM Service Point details as and when required
  •  MDM Admin Entities: If CCB holds subset of MDM data, audit algorithms on MDM BOs with be used to add, update or delete corresponding CCB entities. TOU Group, TOU Map, TOU Map Template, TOU Map Type, Device Type are synchronized from MDM(D1) to CCB(C1) using this approach.

  3. Joint Maintenance Superset Script Synchronization Approach:  This approach is used to maintain similar entities having different data in CCB and MDM.                             X1-JointMaintenance extendable lookup holds details about superset script, CCB BO and MDM BO for data maintenance. BPA scripts are used for maintaining following administration entities in this approach.

CCB
MDM
Superset BPA Script
C1 Unit of Measure
D1 Unit of Measure
X1-UOMMnB
C1 Time of Use
D1 Time of Use
X1-TOUMnB
C1 Service Quantity Identifier
D1 Service Quantity Identifier
X1-SQIMnB
C1 Service Type
D1 Service Type
X1-SvcTyMnB
C1 Service Point Type
D1 Service Point Type
X1-SPTypMnB











Note: Device, Device Configuration and Measuring component data are specific to MDM (D1) so these entities are not synchronized with CCB.