Tuesday, March 8, 2011

SOA Suite Internals

SOA Suite is nothing but an application created oracle developers and deployed in WebLogic container !.  (Eg. One of the application is SOA Infra.ear, Adaptors, B2B etc. )

4 Separate engine will get invoked based on the composite application. Each engine is nothing but web applications created by Oracle using JSP,Servlet, EJB and JMS etc.

1. BPEL Process
2. Mediator
3. Business Rules
4. Human Task.
5. BPMN ( can configure sepearate threads for this....not part of BPEL process)

SOA Infra is the core application. It uses MDS database schema to store all the application information.
Interesting concept of learning SOA Suite is look at different angle for every questions eg. From an Architect view, Service Manager view,

SOA Suite components: - Mediator, BPEL/BPMN process, Human Tasks, Business Rules, and Spring, BPM, BPMN etc.

The BAM, Service bus are outside SOA Suite.

BAM is used for creating real time dashboard and alerts. Event capture, data collection from composite sensors. I consider SOA Suite as a just a BPEL engine. BAM Requires IE7.
Composite application is a collection of instructions that is deployed in SOA INFRA application.
Adaptor service (Based on JCA) is a value addition for SOA Suite.

Difference between SOA Suite (Mediator) and OSB (Oracle Service bus)

Mediator:
  • Tiny, Light weight Service Bus
  •  Used for VETRO Pattern (Validate, Enrich, Transform, Routing, Operate)
  •  Value mapping and cross reference for supporting Canonical Data Model
  •  Message Transformation with XSLT
  •  Part of SCA.(Service Component Architecture)
SOA Service BUS:
  • Large powerful service bus.
  •  Great for Enterprise wide integration
  •  Message Transformation with XSLT  and XQUERY
  •  Not Integrated with SCA
Domain: one more weblogic instance that I could start in one or more physical machines.

SOA Suite Management Tools:

Enterprise Fusion middleware Control (EM)
  • WebLogic Console
  • Enterprise Manager
  • Oracle WebService Manager (OWSM) for policy administration
Oracle Data base schema's used by SOA Suite:
  • MDS  - Meta data services. Used by SOA Suite. Other oracle product are also use this. Eg. WebCenter.
  • SOA INFRA - Dehydration store.
  • ORA BAM - for BAM
  • ORA SDAM - User Messaging service
Directory Structure:
  • Location of log file
  • ORAINVntory /log



Installation:

Copying files to hard drive
1. RCU for database schema
2. Install weblogic server 10.3.3
3. Run patch installer ps 2
4. Run patch installer ps 3

One WebLogic Domain - Multiple WebLogic Clusters, Each cluster have on SOA Suite Cluster. Sharing the database schema means same SOA suite cluster.
Common/bin - execute config.sh for creating domains.

Each development group deploy composite applications in separate soa partions...each partition can contain SOA bundles and each SOA bundles can have one or more composite applications.

Each composite application will be available in domain directory. In EM console, look at the SOA Folder (You can create separate partitions), you can see the composite applications.

All the composite applications data will be stored in MDS schema. Each composite applications consists of several components, each components have their own xml file and composite application have composite.xml configuration file.

The ANT script can be used to deploy the composite application.

  • SCA_DeployComposite
  • SCA_undeploycomposite.
  • SCA_startComposite
  • ant-sca-test.xml  - attach generate and validate configuration plan.
Composite application - SOA Archive.

How to transfer the same SOA Archive from development to production environment without re-compiling ?

Use configuration plan file along with SOA Achieve.

The instance of the composite applications gets created when some one invoke the composite application using the url.

What happens when WLS comes up with SOA Infra application ?

1. Read SOA Infra configuration from MDS
2. Start the basic services and wait for the requests.
3. Route request to service engines and binding components
4. Manage life cycle.

SOA Infra application will get connected to database using default connections pools. The composite application can connect to seperate application databases.
1. SOA Infra database connections to MDS
2. The composite application could connect to application database using separate connection pools at run time.

The audit level: (In common properties of composite applications)
Statefull instances can not complete in a single thread. Helloworld do not require to keep track of the state. The completed instance can not viewable if Audit level is off.  Instance id and some basic details can be retrieved by using audit level "Production".

Audit level could be 1) Off 2) Development 3) Production.

Adut level can be in composite level, engine level, or infra level.  This can do in component level as well.


For each composite applications, there could be:

Inbound JCA -
Outbound JCA - External reference.  Reference can not be shared. can be configured for items like time out, fail over etc.


Adaptors:

1. DB Adaptor
2. JMS Adaptor
3. File/FTP Adaptor

Database adaptor have it's own connection pool and data source to connect to a database, also admin needs to create separate database connection pool for developers to use. JNDI name for this is important as JDeveloper uses JNDI name to connect to database adaptor.

We can export the Jar file. No need to compile the composite application similer to Java files. These are just xml files . use packaging by using the ANT tasks or use JDeveloper.

If you are are using SOA Service bus, it doesn't make sense to expose the composite applications as webservices from SOA Suite, instead expose it via service bus.

Binding components: Send message from SOA Composite application to external applications/services.

JCA Adatpor (in and out)
Web Service (SOAP over HTTP)
HTTP Binding
BAM Adaptor (Out only)
B2B binding component
ADF-BC service (out)
EJB Service (Out only)
Direct Binding Service (In and Out)

BPEL:
For BPEL every thing outside is WSDL. Eg. File adatpor is a WSDL, invoking an EJB is WSDL. JDeveloper creates WSDL for every interaction outside BPEL. Each WSDL is a web service.
BPEL is basically orchestrating the web services.  Internally BPEL has global variable and local variable. The local variables limited to scope. During monitoring each variable value can been seen. BEPL can have multiple activities and conditional logics. BPEL process never ends untill it is over evan if the server crashes.

The state of BPEL process is stored in SOA INFRA table. Each variables value is stored in SOA Infra table. So when the server crashes it will start when the server comes back.

Concurrently we can only run run the composite application instances, each run run instances will take one thread. However if the instances running does not mean that it is consuming a thread. This is applicable only for As synchronous. For Synchronous, we may have a Thread issue if the concurrent users exceeds the available threads.

BPEL Process can have transaction. Define dehydration point so that the transaction ends and data commits.
Create Dehydration points for transaction oriented BPEL


Configure Threads and set the Audit trail 

Use  EMConsole - SOA Infrastructure - SOA Administration - BPEL Properties.

Sensors:
Use an auditing tool. Created at the design time. Use composite sensors. Composite sensors can publish the data only to database. Enterprise manager can use for pulling the data.

Trouble shooting
1. SOA Folder is not available in EM. - Check the SOA Infra application and make sure that it is up.you can start the SOA Infra application using EM.

2. Oracle Technology Network - Documentation - ORacle Fusion middleware. Go to SOA suite. http://www.oracle.com/technetwork/middleware/fusion-middleware/documentation/index.html


FAULT

3. Each Fault can be recovered from its recover state. it can send an email to administrator.

4. Recovering from SOA Composite Application Faults at the SOA
Infrastructure Level - Click the Faults and Rejected Messages tab.- Select Home. 1. Click soa-infra. 1. Select SOA Infrastructure

Dehydration points:

Storing the current status of the BPEL process into the Database is known as dehydration. The Dehydration Store database is used to store BPEL process status data, especially for asynchronous BPEL processes. Also, all successfully executed BPEL process instances are stored in the dehydration store. The database schema is created for this as a part of SOA Suite installation is ORABPEL schema. The following link from The BPEL Cookbook will give you more details about the dehydration store http://www.oracle.com/technology/pub/articles/bpel_cookbook/blanvalet.html


Business Rule engine:

.decs is the rule that represent the rules. Mediator, BPEL etc can execute business rules services except the database adaptors.

SOA Composite application can use for editing the business rules. Dynamic deployment . No deployment required.

Third party rules engines can connect with SOA Suite BPEL, you just use a custom JCA Adaptor or use WSDL that comes from the Ilog.

Security - External LDAP for WebLogic
Go to security realm and providers, then create a new authentication  provider, Select type as Active Directory. Control flag specify if the authentication provide is only one or one of many. If many, then use "optional" from the drop down box under common.

We can have multiple ldap system added to weblogic admin. The control flag determines which one will get precedents. Ideally we can have one external LDAP and one internal LDAP and have one admin id on each.


Deployment options:
  • ANT
  • WLST
  • Console
  • Copy files to folder  


BPM Workflow vs. BPM Worklist application: Same functionality. BPM worklist can do extra stuff with the BPMN.

Worklistapp - Fueog was acquired by BEA. Renamed as aqualogic BPM and have an application called workspace. (BPM workspace application) - BPMN engine inside soa suite.
BPM WorkFlow: localhost:port/bpm/workspace
BPM Worklist: localhost:port/integration/worklistapp

BPMN 
  • Modeling language which allow me to model the business steps. It could be system centric or human centric.
  • Use BPMN to logically model. Eg. step1, setp2, step3 etc...
  • Now decompose step1 and add orchestrations. eg. human workflow, system integration etc.
  • You can take a BPMN model and convert in to BPELs.  
  • For business Analysts.
  • Business Processing Modeling Notation
  • GUI For specifying Business process
  • Managed by OMG. BPMN 2.0 in progress
  • BPD – Business Process Diagram (Private -internal, Abstract-public, Collaboration – Global )
  • High level steps. BPMN will decompose by different BPEL.
  • BPMN Allocation diagram – Sequence object flow in detail

BPEL:
  • Business Process Execution Language
  • An XML based language that allows to describe the business flow
  • Business Process orchestration and execution
  • For Developers.

BPA Suite - For process modelling
BPM Suite - installed as part of SOA Suite install.


BAM (Business Activity Monitoring)

Do the historical and analytical focus on CPM - Corporate Performance Management and
BI - Business intelligence.

Not to mean to provide reports for several weeks, instead use it for hours reporting. Do not use it find out the throughput of all the services, instead use EM for throughput and messaging.

Use it for gathering data that is meant for business analyst. E.g. How many orders you processed? BAM will store the data in database. Database is for fail over not meant to pull the data from database and generate report.

BAM is primarily a run time tool, which pull the data from memory

BAM Architect create the data object. BAM Report creator, BAM report viewers.

Use ICommand utility to import a data object defined in an XML file. BAM adaptor is seperate running outside and providing info to BAM. BAM Adaptor can run in a seperate SOA Suite server.

For BAM Adaptor configuration with SOA Suite, we can use RMI or SOAP webserivice. Better to use SOAP webservice as it is easy.
BAMCommonConfig.xml
BAMServerConfig.xml etc.

Each user has to assigned to a group (eg. Report viewer) in order to view the reports. By default all users do not have any access.  or use "registerusers" utility to configure users. All users should exist in the LDAP store.

BAM Distribution list can be created to send group email etc.

BAM Event generator can be used to create an event (Eg. if some one reject more than 50% orders)



General Items:

1. Audit level of the composite application, Engine and SOA infra.
2. Deployment scripts
3. Deployment segregation for each development group - Partition, SOA Bundle, Composite application etc.
4. Configuring the Server urls - use load balancer url instead of using the server urls. (Soa infrastructure common properties)
5. Sequence of the shutdown and start up (1. Shutdown the SOA Infra application first
6. Audit log file - setup (development, Off, Production)
7. Log file location and rotation.
8. Do you recommend one composite application per service - Depends on usage of service bus. Multiple services can be deployed in one composite application.
9. If no BAM sensors, how do we get the audit information reporting . - Use People sensores can be used to collect the Audit level information ...Use another reporting structure.
10. BPEL process state - synchronous vs Asynchronous.
11. Cap the Audit trail in database using BPEL Service Engine Properties.
12. Configure threads and audit trail settings for each engine.
13. Monitoring BPEL process and faults. Developer can add a fault handler framework.
14. Fault service - Can do bulk re-retry. It can generate email as well. Create single fault recovery, Build fault recovery.
15. Create Dehydration points for transaction oriented BPEL

10 comments:

  1. Good article!! But its needs some refinements as far as content goes.

    Cheers,
    Sps

    ReplyDelete
  2. Hi,

    Really very nice and many corners has been covered.

    ReplyDelete
  3. nice read, full of information... keep up the good job

    ReplyDelete
  4. Good job,very informative

    ReplyDelete
  5. Hi can you please explain how to configure Gmail account to Oracle BAM Server to send alerts

    ReplyDelete
    Replies
    1. hi, you can refer to Lucas Jellemas Oracle soa suite handbook 11g for configuring external mail server with soa (for eg gmail)

      Thanks,
      Sai

      Delete
  6. Hi,

    Good post.

    Question:

    Can we invoke sca_startComposite or sca_getDefaultCompositeRevision from ant? I would like to pass the arguments at run time.

    I tied to get these using the ant and ant-sca-mgmt.xml. At the same time I would like to try the above option.

    Thanks for your help.

    -Raj.

    ReplyDelete
  7. Hi!
    It's a very good article i had never seen it before in any of the site.could you please send me the Topics on OSB too... my E-mail ID is' pavandatti@gmail.com'

    ReplyDelete
  8. Hi Ravi Saraswathi,

    "Synchronous Article" regarding or in the words of SOA suite.Very clearly explained about SOA Really Super.

    ReplyDelete
  9. hi,
    which table in SOAINFRA schema, project scope variables are stored ?

    ReplyDelete