Saturday, March 15, 2014

What is next after SOA ?


Companies are still in the process of implementing SOA. Some organizations using SOA as an IT strategy that organizes discrete functions contained in enterprise applications in to interoperable, standards-based services.
 
How successful are they in implementing SOA?
 
The whole idea about SOA is consumer decoupled from producer, business process decomposed in to  re-usable services and we should be combining re-usable services in to business processes, there  by achieve re- usability, loose coupling, create service in asynchronously with standard service  contract. Also implement discoverability for services that has well defined interface  definitions, use standard data format across organization and use standard Wire and Security protocols.
 
 
I don't think many organizations are achieving the above goals! How many organizations can claim achieving IT agility by implementing SOA?
 
 
Some people believe that by using SOA software, you achieve SOA for the organization!. You don't need a vendor product for implementing SOA for your organization.
 
OK. What is next? I have no doubt that IT Industry is moving towards COTS products. Currently nobody write an accounting or a payroll application. We just buy it from off the shelf, however since the business process are not standard across different IT organizations, we still need custom development software/web applications.  
 
 
Solution: SAAS - within organization?
 
The key to success is, how can we create re-usable applications/code base/services that can use within the organization which allow customization based on the business need?.
 
How can we implement SAAS within the organization without complexity of heavy vendor specific SOA products ? Why can't we use simple JDK/simple containers for writing applications? Use POJO instead of EJBs!.
 
I was having conversation with my friends (Jaswant and Nathan) this morning about some of these trends - Currently industry is getting matured in IAAS (Infrastructure as a service). We have many cloud providers that provide infrastructure such as Amazon, Rackspace, CSC, Terremark etc. Organizations have VMWare  or open source based hypervisors in-house that enable us to create Virtual images as needed and create/update/move/delete hosts based on the requirements. It is still cheaper (financially) to use an in house data center instead of using external vendor provide data center for long term !.
 
 
We have the technology available today to standardize, automate compute resources including Middleware and database complemented by storage and networking capabilities and provide it based on the customer demand.
 
We can self-provision Hosts, Operating Systems, Middleware, Database and Storage using a Web- based graphical user interface that serves as an IT operations management console for entire IT.
 
What is missing in this space is an integration layer or workflow based system that authorize and approve the creation of compute resources for the organization for internally or externally hosted data center.
 
We need a software distribution model in which re-usable applications are hosted internally and made available to departments within IT to use it, customize it, and enhance it based on their requirements. Ideally we should have this from an external vendor (externally hosted). We have few companies in this space such as salesforce, however industry don't have enough on demand computing software delivery models.
 
Let us work towards implementing a SAAS model within the organization !.
 
 

Sunday, December 22, 2013

Some spiritual thoughts!

As you might have seen in many places, "Happiness is not a destination, it is a way of life" !. You get happiness by leading an effective life. So the question is how can we lead an effective life? . One of the way is to look is at the people you consider who successfully leading a good life. Will money help?

You need money for food, shelter, transportation, enjoyment and other basic necessities. Any extra money you have may not bring happiness (You agree?) 

If you ask the richest person in the area, he or she still needs more money to do more things. Collecting more money is a never ending process. So money will not bring happiness or satisfaction.

I always thought about creating a start-up business (I did it once and still thinking!) however will establishing a business and bringing additional money and comfort bring you happiness ? 

If your family is not happy then you can't be happy in home. If your team members are not happy then you can't be happy in office!. 

How to lead an effective life ?
The first question is are you independent or are you depend on others to live?  Any job you do is to serve others and we must need support from family and community. Without family or community one can not survive, however the question is - Are you blaming others for your life misfortunes? I became a software engineer because of my mom!. I can't live a good life because of my wife/husband?  Example: wife/husband is irritating me so I can't relax in my home!  

One must be truly independent, don't blame others for your life. You decide what to do in your life and how to lead your life. 

I recently read a book "The 7 Habits of Highly Effective People: Powerful Lessons in Personal Change" by 
Stephen R. Covey. He explains that all problems that are experienced by individuals fall into one of three categories, which are direct control, indirect control, or no control.

The problems that are classified under direct control are the problems that involve our own behavior. The problems classified as indirect control and no control encompasses problems that we can do nothing about. We always worry about the issues that are outside our control.

Even though issues arise outside your control, the response to the events are your choice. 

I read about one incident where a group of girls and boys visited a library. As they were settling around a table, one cockroach suddenly dropped on one of the girls. The girl got panic and shouted. The cockroach jumped on to another girl. As they were all shouting panicking the librarian approached. He calmly took the cockroach using a towel and placed in the trash bag.

You can't control the incidents in your life; however you can control your reaction to the incidents. 

I read an article by Louis V. Gerstner. He is former CEO of IBM and the author of book "Who Says Elephants Can't Dance?"  As per, Louis, You need to think about this in your early career - "About 10 years from now, what do I want to spend my time on?". He planned for his retirement. He retired from IBM as CEO at age of 61 and is still active and pursuing his dreams.

How many of you (including myself) thought about what we should do after 10 years? or during the retirement ? We flow at the direction of the wind. Wherever wind takes us we go!.What I will be doing after 10 years ? or What I want to do after 10 years?. I am going to give a serious thoughts and work towards the end goal. 

You decide your future. You are independent. There are things that you can't control such as a war or natural calamities such as tornado; however what you can control is your response.













Technology Corner (an article published for KAGW)

By Ravi Saraswathi and Jaswant Singh authors of the book “Oracle SOA BPEL Process Manager 11gR1 – A Hands-on Tutorial” ISBN: 978-1-84968-898-7. It is the comprehensive hands-on industry-leading-practice guide to deliver real-world SOA composite applications using the Oracle SOA Suite BPEL Process Manager platform. Ravi blogs at www.ravisaraswathi.com. Jaswant blogs at techblog.baghel.com. Ravi was also the editor of KAGW 2007.

 


Technology Trends – Cloud Computing:

In this edition of the technology corner, we are introducing and discussing one of the leading technology trends called cloud computing. It doesn’t matter whether you are an IT or non-IT professional, business owner and/or simple home user; cloud computing is transforming your day-to-day life and business activities. In this article, one will learn about cloud computing and how it is changing our day-to-day digital life.




What is cloud computing? We all consume software services for our day-to-day life and business activities. These software runs on the platform and platforms are created using infrastructure components such as network, compute and storage. In traditional setup, we own the software, platform & infrastructure and responsible for maintenance. In other words, we spend for initial setup and spend for the refresh and operations for the life. The cloud computing offers an option to consume software, platform and/or infrastructure services as pay-per-use model like utility bills. We only pay for the time we used these services without incurring the initial setup and maintenance cost. The building blocks for software services are software, platform and infrastructure. The cloud computing primary delivery models are aligned with the building blocks of the software services. These delivery models are Software-as-a-Service (SaaS), Platform-as-a-Service (PaaS) and Infrastructure-as-a-Service (IaaS).

So What? I am an IT professional. How does it change my life? I am a business owner. What benefits it has for me? I am not an IT professional. Why do I care? I am a general user at home? What it offers to me? 

Why cloud computing? Ease of setup, access, automation, flexibility and utilization; managed by industry experts that let you focus on your core competencies; and no setup cost.

Cloud computing is transforming life for everyone. For business owner, it provides opportunities to transform the capital expense into operational expense for their IT needs and focus on your business core competencies. For IT professional, it provides an opportunity to reduce the risk and time to deliver IT services for business function but poses a challenge as well to learn to deliver business function utilizing cloud computing. For non-IT professional, you can expect better availability, accessibility and standardization for the services you are using to run your day-to-day business activities.

In this article, we will discuss in detail the benefits for general user at home. We are assuming that general user these days are using some kind of computing,  communication and entertainment platform such as smart phone, smart television, tablet, laptop and/or desktop with internet access. For your surprise, all of us are already using Software-as-a-Service for our modern life style. Some of us are even consuming Platform-as-a-Service and Infrastructure-as-a-Service to run our personal and professional life.  

Many of us at some point in our life lost personal documents, contacts, photos, music and/or videos due to hardware issues with computers at home and not able to access them while away from home.  Imagine your desktop and/or laptop CPU, RAM and Disk sizes can be adjusted as needed in few moments and you are able to access and stop/start/restart from anywhere and you are guaranteed to not lose the your data with built-in recovery.  It is all possible with cloud computing when you are utilizing Infrastructure-as-a-Service to provide desktop services. 

You can quickly create your personal blog, web sites and web applications with fraction of cost leveraging Platform-as-a-Service of Cloud Computing. 

We all are using Email, Contacts, Calendar, Photos, Videos, Movies, Music, Documents and/or Drive as Software-as-a-Services from various service providers to run our day-to-day personal activities. The cloud computing delivery of these services enables easy collaboration & sharing and access from anywhere using a smart phone, smart television, tablet, laptop or desktop with internet. One needs to select a service provider based on their brand value, reputation and service offering as most of us are consuming the freemium services. Freemium is a business model by which a proprietary product or service is provided free of charge, but money is earned by either advertisements or charge for advanced features. The leading brands for these services are Goolge, Amazon, Apple, Yahoo, MSN, and Aol. Imagine your email, contacts, movies, music and/or documents are available to us anywhere we go. It is all possible with cloud computing. Most of us have not used CDs and DVDs for Music & Movies for several years already and in near future we will not find CDs and DVDs similar to Audio or Video Tapes.

Cloud Computing is enabling us to not worry about losing and/or upgrading a software, smart phone, smart television, tablet, laptop and/or desktop. Our critical data and services are always available and accessible from anywhere.

Tuesday, June 25, 2013

My SOA book co-authored with Jaswant - Oracle SOA BPEL Process Manager 11gR1 – A Hands-on Tutorial is published and available. 


ISBN-13: 9781849688987
Publisher: Packt Publishing

Book is available from the following Websites:

What you will learn from this book

Learn BPEL language and create basic and complex BPEL processes using JDeveloper and Oracle SOA Suite BPEL Process Manager Platform
  • Follow step-by-step instructions for creating and configuring adaptors, human workflow, and advanced BPEL orchestration techniques
  • Practise deploying, testing, debugging, tuning, error handling, and troubleshooting Oracle SOA Suite Platform and BPEL processes
  • Integrate BPEL Process manager with Oracle Service Bus, Business Rules Engine, Business Activity Monitoring (BAM), and Single-Sign-On (SSO)
  • Learn how to achieve scalability, high availability, and failover-failback capabilities to deliver Business Continuity and Disaster Recovery (BCDR) objectives
  • Discover the evolution and future of BPEL and compare BPEL with BPMN and other leading process technologies









In Detail
BPEL, Business Process Execution Language is the definitive standard in writing and defining actions within business processes. Oracle BPEL Process Manager R1 is Oracle’s latest offering, providing you with a complete end-to-end platform for the creation, implementation, and management of your BPEL business processes that are so important to your service-oriented architecture.

"Oracle SOA BPEL Process Manager 11gR1 – A Hands-on Tutorial" is your guide to BPEL design and development, SOA Suite platform troubleshooting, and engineering in a detailed step-by-step guide working real-world examples and case studies.

Using industry-leading practices you will start by creating your first BPEL process and move onto configuring your processes, then invoking, orchestrating, and testing them. You will then learn how to use architect and design services using BPEL, performance tuning, integration, and security, as well as high availability, troubleshooting, and modeling for the future.

"Oracle SOA BPEL Process Manager 11gR1 – A Hands-on Tutorial" is your complete hands-on guide to Oracle SOA BPEL Process Manager 11g.

What you will learn - Details
  • Learn BPEL language and create basic and complex BPEL processes using JDeveloper and Oracle SOA Suite BPEL Process Manager Platform
  • Follow step-by-step instructions for creating and configuring adaptors, human workflow, and advanced BPEL orchestration techniques
  • Practise deploying, testing, debugging, tuning, error handling, and troubleshooting Oracle SOA Suite Platform and BPEL processes
  • Integrate BPEL Process manager with Oracle Service Bus, Business Rules Engine, Business Activity Monitoring (BAM), and Single-Sign-On (SSO)
  • Learn how to achieve scalability, high availability, and failover-failback capabilities to deliver Business Continuity and Disaster Recovery (BCDR) objectives
  • Discover the evolution and future of BPEL and compare BPEL with BPMN and other leading process technologies
Approach
This hands-on, example-driven guide is a practical getting started tutorial with plenty of step-by-step instructions for beginner to intermediate level readers working with BPEL PM in Oracle SOA Suite

Who this book is written for
Written for SOA developers, administrators, architects, and engineers who want to get started with Oracle BPEL PM 11g. No previous experience with BPEL PM is required, but an understanding of SOA and web services is assumed

Table of Contents

Preface
Chapter 1: Creating Basic BPEL Processes
Chapter 2: Configuring BPEL Processes
Chapter 3: Invoking a BPEL Process
Chapter 4: Orchestrating BPEL Services
Chapter 5: Test and Troubleshoot SOA Composites
Chapter 6: Architect and Design Services Using BPEL
Chapter 7: Performance Tuning – Systems Running BPEL Processes
Chapter 8: Integrating the BPEL Process Manager with Service Bus, Registry, and SOA Deployment
Chapter 9: Securing a BPEL Process
Chapter 10: Architecting High Availability for Business Services
Chapter 11: The Future of Process Modeling
Chapter 12: Troubleshooting Techniques
Index
 http://www.amazon.com/dp/1849688982/?tag=packtpubli-20


SOA Implementation

Enabling or implementing Service Oriented Architecture for your organization has nothing to do with implementing SOA Suite software! You enable re-use of your applications by implementing the loose coupling (Web services) and achieve application agility. You can say that you are successful with implementing SOA when you enable reuse of applications or services. 
 
I see many companies purchase SOA Suite and implement few new applications then claims that they implemented SOA! Achieving interoperability and loose coupling with standard well defined re-usable interface is important for any SOA implementation.

The views of SOA may be different for business, IT Architect, Developer or an IT manager, however just deploying few applications in SOA Suite will not enable SOA for your organization.  

SOA Suite enable loose coupling by creating the web services from the monolithic systems to enable HTTP access of web apps/services.  It doesn’t matter what language, operating system or protocol being used for your applications or legacy systems.

The bottom line for SOA implementation is implementation of Web services. In any organization, the port number 80 (HTTPS – 443) of all webservers is open! So any organization can access the web services published by any other organization. Of course you will have appropriate authentication and authorization built as part of the web services.

Web services provide a interoperable messaging framework. WSDL (Web service description language) provide a description of web services. WSDL answers the following questions about the web services. 

What (what this service does), How (This service gets invoked - HTTP, SMTP, Queue) & Where (where is this service located?) The WSDL contains Types (XML definitions usually you put import XML schema), Messages, Port Types, Binding and Services.

 

Monday, June 24, 2013

SOA Suite Internals


Question: Which Operating system environment SOA Suite runs well ? 

Answer: PowerPoint !. 


SOA Suite application (Deployed on top of WebLogic) consists of the following modules/engines (Yes, I consider SOA Suite is an application deployed on top of WebLogic) 

  • Mediator 
  • BPEL
  • Human Task
  • Business Rules
  • UMS (User Messaging Service) 
  • BPMN (You need BPM Suite) 


Mediator
Mediator will do the routing. It is a software based router! It can do dynamic routing or content based routing. Mediator can filter content from the SOAP header information. The SOAP does not get in to composite application if we have mediator to do the validation. Mediator does VETRO Pattern (Validate, Enrich, Transform, Routing, and Operate). Mediator is part of SCA.

Do we need to validate the XML document before sending to BPELs? It may be a good idea to check the document based on the content or syntax.

Basically, Mediator does the following:

  • Validate XML Schema
  • Filter Data
  • Transform XML data
  • Route data from Service client to target services
  • Manage event as a subscriber and publisher


Every event will have a name and an XML document associated with it. Mediator can be configured to listen for events from clients. The message exchange can be one-way or request-response. You can add error handling with Mediator such re-try logic.

We can assign threads for each of SOA Suite service engine. I don't think that the service engine threads relates to WebLogic threads. WebLogic threads are allocated dynamically by work manager. It is a kind of self-tuning.


Mediator have ParallelWorkerThreadCount, ParallelMaxRowsRetrieved,Parallelthreadsleep etc. It is better to keep the default values instead of changing it without knowing the full impact.

BPEL (Business Process Execution Language) Engine

BPEL is a markup language to orchestrate services. It is designed to integrate web services. BPEL processes are executable and can run over long time. The entire language looks like flow charts. It is a programming language built on top of XML specifications.  


 User Messaging Service (UMS)

Enable communication between Users and deployed applications. You can send email notification from composite applications to users using UMS. UMS Can be invoked as a web service or using RMI from an EJB. All messages will be stored in a database. The UMS has a server and an EMail Driver. On Enterprise Manager Console, select "User Messaging Service" from the left navigation to configure UMS. SMTP Server name and protocol can be added to the configurations. You can invoke UMS from your BPEL to send and receive the messages from users.  

Dehydration store 

Dehydration store is couple of Table in Data base created as part of RCA Script for storing transaction data.  
 In SOA Suite, some of the transactions may be running in days or weeks! The container may restart during this period. Dehydration store is required to store these transactions and restart when the container comes back to original state. 

Dehydration makes sense only on Asynchronous transactions. Assume that you have wait time defined between BPEL activities and you shut down the SOA Suite during the wait period. The Dehydration store will ensure that the wait time is honored when your start the SOA Suite container.


LDAP integration with SOA Suite: 
SOA Suite has embedded LDAP. You can add external LDAP as well. JXplorer  software can use to connect to LDAP directory. Other available LDAP browsers are openLdap browser and ldapbrowser.

UDDI:

According to the UDDI model, each organization should have separate UDDI registries that publish the available web services. The UDDI is a scam!. I couldn't find any organization that has successfully implemented run time UDDI for heavy transaction business critical applications for production environment.  


Composite application:

Composite applications are web services!

How do we group together all web services? You can use composite application. It enables course grained services that composed of different components. This is an end product of SOA Suite developed components.  All composite application will have a composite.xml. All ear file will have an application.xml. These are called deployment descriptors.

SCA define collection of speciations. Service component architecture (SCA) specification provides a way for developing an assembled SOA application.

You can combine all the application components such as EJBs, Mediator, Services, Adaptors etc. in to the single file called as composite application.

Usually the naming convention of composite application is SCA_<Composite Name>_1.0.jar. If a new version of application is deployed then the entire request will go to new version. The new version will becomes default composite application. The enterprise manager console indicates the default application with a dot. You can mark the old version as retired. Any active instance will finish its run, however new request will go to new default version of the composite application.

Always deploy the SOA composite applications as bundle. Use SAR (SOA Archive). You can use ant-sca-package script for creating the SOA Archive, or use JDeveloper.
Other option is to use sca_package scripts from WLST. The main advantage of SAR file is, all composite application gets deployed or none. The container will roll back all the deployments in case one of the composite application fails to deploy.

SOAP Message

A typical SOAP Message consists of the following: 

<SOA:Envelope>
<SOAP: Header>
</SOAP:Header>
<SOAP:Body>
<SOAP: Part name="desc">
<xml>
<SOAP: part name="pic">
<xml>
</soap:Body>
</soap:Envelope>



Monday, May 27, 2013

My "SOA BPEL Process Manager" book is to publish soon

http://www.packtpub.com/oracle-soa-bpel-pm-11g-r1-tutorial/book
(Authored by me and my friend Jaswant)

 http://www.packtpub.com/oracle-soa-bpel-pm-11g-r1-tutorial/book

 

Saturday, November 24, 2012

HP Pavalion DV6 -Windows 7 Wireless connectivity issue

I used to loose internet connetion once in a while with my new HP Pavalion DV6 -Windows 7 laptop. Found a solution today !.

Go to Control Panel  or windows 7 search > Device Manager > locate the wireless network adapter in the list, right click and choose Properties. On the last tab, Power Management, uncheck the box for "Allow computer to turn off this device to save power"

Hope this helps some one !.

Saturday, September 8, 2012

Hadoop - my notes !!


Hadoop is for parallel processing. Use for distributed processing of large data across clusters. It brings the ability to cheaply process large amounts of data, Example: 10 to 100 gigabytes or above.  It can use for ETL. It is a platform for data storage. Hadoop is written basically on java; however it uses some of the native libraries as well.  Instead of using costly data warehouse software, you can use Hadoop! It has ability to process non-structured data as well.
  • Map Reduce - It is a process engine and a distributed computing framework. It will distribute the large file in to small separate files. Processes large jobs in parallel across many nodes and combines the results. Reading around 4 Terabyte data may take some time as reading speed usual low as 55 Mega Bytes per second. 
  • Apache Hive – SQL-like language and metadata repository
  • Apache Pig – High level language for expressing data analysis programs
  • Apache HBase – Hadoop database for random, real-time read/write access
  • Apache Mahout – Library of machine learning algorithms for Apache Hadoop
  • Apache Oozie – Server-based workflow engine for Hadoop activities
  • Apache Zookeeper – Highly reliable distributed coordination service
  • Apache Sqoop – Integrating Hadoop with RDBMS
  • Apache Flume – Distributed service for collecting and aggregating log and event data
  • Fuse-DFS – Module within Hadoop for mounting HDFS as a traditional file system
  • Hue – Browser-based desktop interface for interacting with Hadoop. GUI (Provide authentication, workflow etc)
  • HDFS - Distributed, scalable, fault tolerant file system.  High bandwidth clustered storage - Distributed file system. Store the files in blocks and store the data across the cluster. The default block size is 64M and it can increase up to 128M. There can be many nodes (e.g. 10 or 1000)
It stores three replicas for data. So if one node goes down then it should not be a problem. Hadoop uses the internal algorithm to locate the block of the data. First node is selected randomly.
Fair scheduler - Priority pool and non priority pool.
Interface to HDFS: (How to get data in out of Hadoop).Use command line. Use commands to export and import files from regular file system to Hadoop.
Scheduling - OOZE.  No rollback available in Hadoop !!

Webmethod

 Webmethod is not a product; it contains a group of products.  Will webmethod better than Oracle SOA Suite and Service bus for creating services ?? Webmethod do not support BPEL. Software AG believes BPMN is the future. !!
Main Webmethod containers:
  • Webmethod integration Server - It is a Tomcat Container. Provide similar functionality as Oracle SOA or Service bus.  Webmethod DSP (Dynamic Server Pages) is similar to JSP. Integration server provide transformation and translation service. Development is done on the server itself! It has lock and unlock functionalities similar to check in and checkout.
Example of a service is "shipping", "billing" etc. Service is different from business process. Service is similar to java class. 
One of available service is “Flow Service” - GUI available such as "repeat", "Flow", "Sequence (Arrange and group together)", "loop", etc for transformation and translation.
  • Webmethod broker - Messaging platform. Publish subscribe. (No programming) use Designer for change. Write trigger service to subscribe it from broker.
  • My WebMethod Server - Portal container. It manages all Webmethod products. Have default Portlets for many WebMethod products. My WebMethod server is responsible for administering all WebMethod products. A Single GUI to administer all WebMethod products.
  • Central site is similar to Registry and repository.
  • Webmethod designer IDE is based on Eclipse - Can create Portlets and BPM.  To create web services use development tool: Webmethod Developer.
One of the advantage of WebMethod is it automatically create a script and image file during the install which allow administrators to install further installs from the original installation.
Software AG market Webmethod product. Broker is written in C language. Other Webmethod products are written in java. Broker clustering is complicated. My WebMethod server is used for administering Integration server and other products. Designer is used for creating Portlets.
Designer is nothing but an eclipse IDE with ads on.
Webmethod uses Terracotta or coherence for clustering!  I am not sure how to monitor of the services in WebMethods. All we could see the audit feature. Trading network can use for monitoring all the services. It is a B2B component.  
Software AG Webmethod BPM Suite consists of the following:
  • Webmethod integration server, MWS, Analytical Engine. Webmethod Designer, Rules Engine is an added product with Webmethod 8.
Central site and ARIS process governance (Work flow engine used between ARIS and Integration server) product are separate products. Central site and ARIS process governance is used for syncing process models between ARIS and Integration server. The license is included as part of the BPM Suite licensing.
Integration server has a process engine. MWS has a task engine. Prediction engine and Web service data collector are also included, Webmethod broker. Mash Zone is a separate product for creating dashboards (www.mashzone.com).
Governance product is not part of BPM Suite. Software AG partner with FICO and providing a rule/decision engine as part of Webmethod 8.
 ARIS 7.2 new version has the feature of import and export functionality.
IT has a scheduler available to pick up the files. Use fully qualified name of the service for scheduling. All the services in Webmethod integration server is web service. You can create WSDL file and invoke it. 
By using adaptors, integration server is used to integrate multiple systems. E.g. Integrate with mainframe, database, JMS, SAP etc.
Webmethod can pull file from another server or accept the file via http port.  Integration server can be in DMZ, but recommended way is to have one integration server in DMZ and configure reverse HTTP Gateway server configuration and have one Integration server in Secured network.

Exalogic

Oracle claims that Exalogic is an elastic cloud but requires you to buy hardware for elastic cloud! You cannot scale unless you have enough hardware available in inventory! This is against basic principle of the cloud.

 A pre-configured solution makes the integration headaches go away, but Exalogic is far behind by achieving this goal. I haven’t reviewed all the features of new version of Exalogic cloud software version 2.0.

 I wish, if Oracle rollouts an Exalogic version with pre-configured hardware and software. This is supposed to be hardware and software engineered to work together.

I like to see pre-installed Load balancer, OS, clustered middleware software (WebLogic, SOA Suite, Web center etc.)   which is ready to use from the day it connects to datacenter. Why people spend time for installing and configuring the software after buying a hardware appliance?

The major issue is the certification for all oracle software products. I think, some of the core products and versions are not even certified to run on Exalogic. The other major concern is the migration from one version to another version of the hardware. I couldn't find an automated migration strategy for Exalogic between versions. Also what people are suppose to do with older versions of Exalogic after migrating to new versions ?

How about comparing Exalogic with features of Web Sphere Cloudburst or Amazon EC2 in terms of elasticity, price, hardware features and software adaptability?

Monday, April 23, 2012

BPM (Business Process Management)

Business Process Management Tools:

Options for BPM: Oracle BPM 11g, Webmethod 12, TIBCO, IBM BPM, JPBM
Business process Modeling tools: ARIS from Software AG, BPM Studio by Oracle.

Why a BPM tool required ?  - For adding work flow capability that includes notifications, reporting and ability to provide a view of each of the work item. Deployment of BPMN. Irrespective of the claims by the vendors, none of the available BPM tools allow business users to change the process without involvement of
IT. Business users still need to contact IT for rule modification, change process etc. Another cliam is BPM tool allows align IT to business.

Flexibility comes with complexity!

The vendor’s claim that BPM allows developing and maintaining business process with zero code, just write XML!. I think this is just an accelerator tool that enables developers to create process and make modification to the process.

Process composer tool can be utilized by the business to do the simulation and process creation. but business users has to contact IT for development.

Why can't industry create some common business process?

Example. Credit card management has different business process. Different organization uses these processes. Can industry create common use cases and re-usable business process so that multiple businesses can share the business process across the industry by different organizations?

Usuall BPM Tool allows:
  • Process orchestration
  • Have very lightweight business rule engine
  • License include limited portal functionalities
  • BPM process composer - use browser to build diagrams. 
  • Built on top of J2EE container SOA Suite.
  • Process spaces - portal concept for uI. (Webcenter spaces - separate component). Collaborative work environment. social aspects.
  • Process analysts - bam, bi etc.
  • BPM studio allows you to do the simulation.

The major issue is with complexity involved in install, configure and deploy. Also amazing to know that number of tools/web applications and amount of memory and CPU require for maintaining the BPM Suite.

We are adding complexity in terms of number of software components, to have simplicity for developers (or save development time)!

Business process Workflows has different components such as task assignments, management of rules, work list applications, Task forms, notifications, ADF based forms (Java standard JSF) and identity management, reports, audit trails etc.

  

Monday, December 26, 2011

2012 Goals

It is time to document 2012 goals and do an analysis of 2011. I would say most of my major personal goals are achieved for year 2011 such as new home, bringing my mother etc.

2012 goals :

1. Official goals
2. Personal goals...

Should I publish my goals for this year here  ? :) .

 It is important to set a few goals as possible and achieve 100 % of it. Also it is important to get a 360 degree feedback from your colleagues and family members. There is always room for improvement in personal and official life.

OK. Keep the official and personal goals in a separate document.

Personal focus of 2012  is  - determined to spend more quality time with kids. It is amazing how fast they are growing !.  Increase energy levels, do the things that bring happiness such as spending time with friend's family for dinner etc. Any way goals are important than the focus areas,  I am yet to determine how to report the progress of my 2012 items....

Thursday, December 15, 2011

Solaris 11 - Cloud OS !?

Solaris 11 released by Oracle. Mark Hurd "claimed it as a first cloud OS”!.

Oracle recognized the Solaris LDOMs as a Oracle VM so licensing is much is easier now, is it? This is a good thing. Oracle is comparing Solaris with IBM AIX / HP? (Not to Redhat Linux!)

T4 Processor, T4 Server Family, super cluster - how good is the new processor? What are the performance benchmarks? Multi threaded architecture? Common across all software platform - reliability, availability, security, scalability, Performance, virtualization!

Engineer to work together with WebLogic and Oracle Database. It has capability to add storage pool at Zone level. Custom repositories available. Oracle VM templates can be stored at repositories. Repositories are encrypted. What is the future of Oracle Enterprise Manager? Look like lot of push for implementing Enterprise manager for monitoring, provisioning and reporting?

Tuesday, May 17, 2011

JVM Issues !!!

Resource for JVM Troubel shooting: 
  • Log files messages -  -verbose:gc, –XX:+PrintGC, –XX:+PrintGCDetails,–XX:+PrintGCTimeStamps, -XX:+HeapDumpOnOutOfMemoryError
  • Java VisualVM - Available from JDK 6. Federates several existing tools, including JConsole, jstat, jinfo, jstack, and jmap
  • Other Tools - Jprobe, Jconsole, HPROF - Heap Profiler, Eclipse Memory Analyzer, IBM Heap & thread analyzer
  • JVM Utilities - Jps, Jmap, Jinfo, Jhat, Jstack
  • Unix Tools - Top, vmstat, ulimit –a , sar –r , mpstat, netstat
Typical JVM - 6 Parameters

  -Xms1024m \ - Minimum Heap   -Xmx1024m \ - Maximum Heap
  -Xss512k \ - Set maximum native stack size for any thread
  -XX:+UseConcMarkSweepGC
  -XX:GCTimeRatio=99 \ - The ratio of GC time to application time
  -XX:MaxGCPauseMillis=20 \ - Pause times of 20ms desired.
 -XX:PermSize=512m \ - Permanent Size 
 -XX:MaxPermSize=512m \  
 -XX:NewSize=384m \ - Minor GC (Young Generation) -XX:MaxNewSize=384m \
  -XX:SurvivorRatio=6 \

JVM Collectors - Options

1. Mark and sweep
Minimizes pause times, avoid "stopping the world“
Collect the tenured generation concurrently with the execution of the application
Start VM - mark each object – Remove
-XX:+UseConcMarkSweepGC

2. Throughput collector Parallel Collector
Distributes GC load across CPUs. Maximize throughput
XX:+UseParallelGC
Can be used for Multiprocessor Machines  (with four or more processors)
 

Saturday, March 26, 2011

Do you want to develop BPEL ??

1. Go to http://www.oracle.com/technetwork/developer-tools/jdev/overview/index.html

2. Download JDeveloper 

3. Install and configure

4. (Search for "JDeveloper Extensions" using CTRL - F (browser Search) in http://www.oracle.com/technetwork/developer-tools/jdev/overview/index.html Select "Oracle Fusion Middleware")

OR

Go directly to http://www.oracle.com/ocom/groups/public/@otn/documents/webcontent/156082.xml
and download Oracle SOA Composite editor (File name:soa-jdev-extension.zip). Ensure that you are downloading same version as your JDeveloper


5. Select JDeveloper select Help->Check for Updates. Click Next and Select Install from Local File and browse to where you saved soa-jdev-extension.zip. Click Open, click Next and then Finish. JDeveloper will then install the extension; finally restart JDeveloper when prompted

Sunday, March 20, 2011

Apache Conference 2010

Interesting topics for this year Apache conference ! Planning to participate.

... Enterprise Solutions (from ActiveMQ to Axis2 to ServiceMix, OFBiz to Chemistry, the gang's all here!)
... Cloud Computing (Hadoop, Cassandra, HBase, CouchDB, and friends)
... Emerging Technologies + Innovation (Incubating projects such as Libcloud, Stonehenge, and Wookie)
... Community Leadership (mentoring and meritocracy, GSoC and related initiatives)
... Data Handling, Search + Analytics (Lucene, Solr, Mahout, OODT, Hive and friends)
... Pervasive Computing (Felix/OSGi, Tomcat, MyFaces Trinidad, and friends)
... Servers, Infrastructure + Tools (HTTP Server, SpamAssassin, Geronimo, Sling, Wicket and friends)

conference theme is "Open Source Enterprise Solutions, Cloud Computing, and Community Leadership"


Friday, March 11, 2011

SOA Suite - Tuning

TUNING
  • Heap size - JVM will grab from OS for the memory usage. Use EM -Weblogic Domain-right click and use JVM performance to view jvm performance. For 32 bit, 3 GB is the limit. For 64 bit, we can have more eg. 12 GB. Use 60 - 85 % of the available memory for heap. For sun JDK use ensure that permgen size is changed. For SOA, we need to increase the Permgen size. For Jrockit, no perm size.
  • Default JTA time out is 30 sec. for SOA Suite, we need to increase the transaction time out.   
  • Set the processor parameter in database > 300
  • Set the session parameter > 200.
  • Sys user with SysDBA Privilages for the schema. 
  • Make sure we are using 64 bit JVM.
  • Payload validation (while configuring the server urls) decrease the performance
  • Audit level - may decrease the performance.
  • Database adaptor connection pool setting , Database adaptor connection pool setting for developers to use.
  • Trouble shoot - Ensure that the file adaptors have data appropriately. Will the instance is up and running.
  • BPEL properties - Audit trail level size, document size for a variable, payload validation for incoming and outgoing messages.
  • Mediator properties.
  • Tune JCA Adaptor - File adaptor (inbound, outbound throttling, thread count, concurrentThreshold use staging, serialize translation etc)
  • Database adaptor tuning.
  • Database tuning - will run out table spaces very often. init.ora. how long the instances will be in database ? untill you purge the instances. Table space should be auto extended. We should have a mechanisam to clean up the instances in database. Change from development mode to production mode to reduce the instance data in database. We can purge only the completed instances. Other option is to transfer the data to another data base for futher reference.
  • Database connection pools. Do not do the connection testing and profiling in production.
  • Purge data - Delete all the completed instances.  rejected messages,  and orphan message.
  • Developers can write a pojo or a composite application to send email using UMS for completed messages, rejected messages etc.

Monitoring

  • Use WLDF
  • Use EM Console - CPU, Memory, Heap , Requests. Etc. EM has the socket connection to all the instances.
  • Logging level - notification, error or warning , notification, trance.
  • Disable fetching in em.

Thursday, March 10, 2011

Security - WebService security and SAML

Security

 
1) Transport Level - Secure connections bewteen consumer and producer. Use SSL. SSL provide authetical, confidentiality and message integrety. Issue if there is an intermediatery in between (eg. router) then it will get the SOAP message in plain text.

 
2) Message Level - Secure message. Digitally signed and encrypted. For authetication use username, x.509 or SAML tokens.

 
WS-Security specification has usernameTokenProfile(Sign, Encrypt and propogate). For authetication, we can use
  • SAML-Assetion
  • X.509 certificates
  • Kerberos Tickets
  • Userid/Password credentials
  • Ws-Security defines how to attach XML signature and XML Encryption headers to SOAP message.
  • <wsse:UserNameToken>
SAML framework is for exchanging security information between different partie through XML documents
  • Authetication Assertion
  • Autherization Assertion
  • Attribute Assertion
WS-Security allows SAML assertion to be placed inside a SOAP header. It defines how to insert the information in to SOA envolpe.

 

 

 
Security applied to various levels of the network protocol stack

 
  • Application -  Application Specific: Passwords
  • Presentation - Software Encryption: Secure Socket Layer (SSL)
  • Session -  End-to-End Authentication/Encryption
  • Transport -  Port Filtering
  • Network-  IP Address Filtering
  • Data Link -  Snoop Prevention
  • Physical -  Point-to-Point Hardware Encryption: (Class I & II)

 
Difference between Hacking and Cranking 

 
Hacking

 
  • Don’t learn to hack…Hack to learn!
  • Make things work & Fix
  • Modify & Improve
  • Find weaknesses & strengthen security

 
Cracking

 
  • Obtain root access
  • Infiltrate
  • Damage
  • Change
  • Control
  • Steal

 

 

 
RFC - Request for comment !.

Tuesday, March 8, 2011

SOA Suite Internals

SOA Suite is nothing but an application created oracle developers and deployed in WebLogic container !.  (Eg. One of the application is SOA Infra.ear, Adaptors, B2B etc. )

4 Separate engine will get invoked based on the composite application. Each engine is nothing but web applications created by Oracle using JSP,Servlet, EJB and JMS etc.

1. BPEL Process
2. Mediator
3. Business Rules
4. Human Task.
5. BPMN ( can configure sepearate threads for this....not part of BPEL process)

SOA Infra is the core application. It uses MDS database schema to store all the application information.
Interesting concept of learning SOA Suite is look at different angle for every questions eg. From an Architect view, Service Manager view,

SOA Suite components: - Mediator, BPEL/BPMN process, Human Tasks, Business Rules, and Spring, BPM, BPMN etc.

The BAM, Service bus are outside SOA Suite.

BAM is used for creating real time dashboard and alerts. Event capture, data collection from composite sensors. I consider SOA Suite as a just a BPEL engine. BAM Requires IE7.
Composite application is a collection of instructions that is deployed in SOA INFRA application.
Adaptor service (Based on JCA) is a value addition for SOA Suite.

Difference between SOA Suite (Mediator) and OSB (Oracle Service bus)

Mediator:
  • Tiny, Light weight Service Bus
  •  Used for VETRO Pattern (Validate, Enrich, Transform, Routing, Operate)
  •  Value mapping and cross reference for supporting Canonical Data Model
  •  Message Transformation with XSLT
  •  Part of SCA.(Service Component Architecture)
SOA Service BUS:
  • Large powerful service bus.
  •  Great for Enterprise wide integration
  •  Message Transformation with XSLT  and XQUERY
  •  Not Integrated with SCA
Domain: one more weblogic instance that I could start in one or more physical machines.

SOA Suite Management Tools:

Enterprise Fusion middleware Control (EM)
  • WebLogic Console
  • Enterprise Manager
  • Oracle WebService Manager (OWSM) for policy administration
Oracle Data base schema's used by SOA Suite:
  • MDS  - Meta data services. Used by SOA Suite. Other oracle product are also use this. Eg. WebCenter.
  • SOA INFRA - Dehydration store.
  • ORA BAM - for BAM
  • ORA SDAM - User Messaging service
Directory Structure:
  • Location of log file
  • ORAINVntory /log



Installation:

Copying files to hard drive
1. RCU for database schema
2. Install weblogic server 10.3.3
3. Run patch installer ps 2
4. Run patch installer ps 3

One WebLogic Domain - Multiple WebLogic Clusters, Each cluster have on SOA Suite Cluster. Sharing the database schema means same SOA suite cluster.
Common/bin - execute config.sh for creating domains.

Each development group deploy composite applications in separate soa partions...each partition can contain SOA bundles and each SOA bundles can have one or more composite applications.

Each composite application will be available in domain directory. In EM console, look at the SOA Folder (You can create separate partitions), you can see the composite applications.

All the composite applications data will be stored in MDS schema. Each composite applications consists of several components, each components have their own xml file and composite application have composite.xml configuration file.

The ANT script can be used to deploy the composite application.

  • SCA_DeployComposite
  • SCA_undeploycomposite.
  • SCA_startComposite
  • ant-sca-test.xml  - attach generate and validate configuration plan.
Composite application - SOA Archive.

How to transfer the same SOA Archive from development to production environment without re-compiling ?

Use configuration plan file along with SOA Achieve.

The instance of the composite applications gets created when some one invoke the composite application using the url.

What happens when WLS comes up with SOA Infra application ?

1. Read SOA Infra configuration from MDS
2. Start the basic services and wait for the requests.
3. Route request to service engines and binding components
4. Manage life cycle.

SOA Infra application will get connected to database using default connections pools. The composite application can connect to seperate application databases.
1. SOA Infra database connections to MDS
2. The composite application could connect to application database using separate connection pools at run time.

The audit level: (In common properties of composite applications)
Statefull instances can not complete in a single thread. Helloworld do not require to keep track of the state. The completed instance can not viewable if Audit level is off.  Instance id and some basic details can be retrieved by using audit level "Production".

Audit level could be 1) Off 2) Development 3) Production.

Adut level can be in composite level, engine level, or infra level.  This can do in component level as well.


For each composite applications, there could be:

Inbound JCA -
Outbound JCA - External reference.  Reference can not be shared. can be configured for items like time out, fail over etc.


Adaptors:

1. DB Adaptor
2. JMS Adaptor
3. File/FTP Adaptor

Database adaptor have it's own connection pool and data source to connect to a database, also admin needs to create separate database connection pool for developers to use. JNDI name for this is important as JDeveloper uses JNDI name to connect to database adaptor.

We can export the Jar file. No need to compile the composite application similer to Java files. These are just xml files . use packaging by using the ANT tasks or use JDeveloper.

If you are are using SOA Service bus, it doesn't make sense to expose the composite applications as webservices from SOA Suite, instead expose it via service bus.

Binding components: Send message from SOA Composite application to external applications/services.

JCA Adatpor (in and out)
Web Service (SOAP over HTTP)
HTTP Binding
BAM Adaptor (Out only)
B2B binding component
ADF-BC service (out)
EJB Service (Out only)
Direct Binding Service (In and Out)

BPEL:
For BPEL every thing outside is WSDL. Eg. File adatpor is a WSDL, invoking an EJB is WSDL. JDeveloper creates WSDL for every interaction outside BPEL. Each WSDL is a web service.
BPEL is basically orchestrating the web services.  Internally BPEL has global variable and local variable. The local variables limited to scope. During monitoring each variable value can been seen. BEPL can have multiple activities and conditional logics. BPEL process never ends untill it is over evan if the server crashes.

The state of BPEL process is stored in SOA INFRA table. Each variables value is stored in SOA Infra table. So when the server crashes it will start when the server comes back.

Concurrently we can only run run the composite application instances, each run run instances will take one thread. However if the instances running does not mean that it is consuming a thread. This is applicable only for As synchronous. For Synchronous, we may have a Thread issue if the concurrent users exceeds the available threads.

BPEL Process can have transaction. Define dehydration point so that the transaction ends and data commits.
Create Dehydration points for transaction oriented BPEL


Configure Threads and set the Audit trail 

Use  EMConsole - SOA Infrastructure - SOA Administration - BPEL Properties.

Sensors:
Use an auditing tool. Created at the design time. Use composite sensors. Composite sensors can publish the data only to database. Enterprise manager can use for pulling the data.

Trouble shooting
1. SOA Folder is not available in EM. - Check the SOA Infra application and make sure that it is up.you can start the SOA Infra application using EM.

2. Oracle Technology Network - Documentation - ORacle Fusion middleware. Go to SOA suite. http://www.oracle.com/technetwork/middleware/fusion-middleware/documentation/index.html


FAULT

3. Each Fault can be recovered from its recover state. it can send an email to administrator.

4. Recovering from SOA Composite Application Faults at the SOA
Infrastructure Level - Click the Faults and Rejected Messages tab.- Select Home. 1. Click soa-infra. 1. Select SOA Infrastructure

Dehydration points:

Storing the current status of the BPEL process into the Database is known as dehydration. The Dehydration Store database is used to store BPEL process status data, especially for asynchronous BPEL processes. Also, all successfully executed BPEL process instances are stored in the dehydration store. The database schema is created for this as a part of SOA Suite installation is ORABPEL schema. The following link from The BPEL Cookbook will give you more details about the dehydration store http://www.oracle.com/technology/pub/articles/bpel_cookbook/blanvalet.html


Business Rule engine:

.decs is the rule that represent the rules. Mediator, BPEL etc can execute business rules services except the database adaptors.

SOA Composite application can use for editing the business rules. Dynamic deployment . No deployment required.

Third party rules engines can connect with SOA Suite BPEL, you just use a custom JCA Adaptor or use WSDL that comes from the Ilog.

Security - External LDAP for WebLogic
Go to security realm and providers, then create a new authentication  provider, Select type as Active Directory. Control flag specify if the authentication provide is only one or one of many. If many, then use "optional" from the drop down box under common.

We can have multiple ldap system added to weblogic admin. The control flag determines which one will get precedents. Ideally we can have one external LDAP and one internal LDAP and have one admin id on each.


Deployment options:
  • ANT
  • WLST
  • Console
  • Copy files to folder  


BPM Workflow vs. BPM Worklist application: Same functionality. BPM worklist can do extra stuff with the BPMN.

Worklistapp - Fueog was acquired by BEA. Renamed as aqualogic BPM and have an application called workspace. (BPM workspace application) - BPMN engine inside soa suite.
BPM WorkFlow: localhost:port/bpm/workspace
BPM Worklist: localhost:port/integration/worklistapp

BPMN 
  • Modeling language which allow me to model the business steps. It could be system centric or human centric.
  • Use BPMN to logically model. Eg. step1, setp2, step3 etc...
  • Now decompose step1 and add orchestrations. eg. human workflow, system integration etc.
  • You can take a BPMN model and convert in to BPELs.  
  • For business Analysts.
  • Business Processing Modeling Notation
  • GUI For specifying Business process
  • Managed by OMG. BPMN 2.0 in progress
  • BPD – Business Process Diagram (Private -internal, Abstract-public, Collaboration – Global )
  • High level steps. BPMN will decompose by different BPEL.
  • BPMN Allocation diagram – Sequence object flow in detail

BPEL:
  • Business Process Execution Language
  • An XML based language that allows to describe the business flow
  • Business Process orchestration and execution
  • For Developers.

BPA Suite - For process modelling
BPM Suite - installed as part of SOA Suite install.


BAM (Business Activity Monitoring)

Do the historical and analytical focus on CPM - Corporate Performance Management and
BI - Business intelligence.

Not to mean to provide reports for several weeks, instead use it for hours reporting. Do not use it find out the throughput of all the services, instead use EM for throughput and messaging.

Use it for gathering data that is meant for business analyst. E.g. How many orders you processed? BAM will store the data in database. Database is for fail over not meant to pull the data from database and generate report.

BAM is primarily a run time tool, which pull the data from memory

BAM Architect create the data object. BAM Report creator, BAM report viewers.

Use ICommand utility to import a data object defined in an XML file. BAM adaptor is seperate running outside and providing info to BAM. BAM Adaptor can run in a seperate SOA Suite server.

For BAM Adaptor configuration with SOA Suite, we can use RMI or SOAP webserivice. Better to use SOAP webservice as it is easy.
BAMCommonConfig.xml
BAMServerConfig.xml etc.

Each user has to assigned to a group (eg. Report viewer) in order to view the reports. By default all users do not have any access.  or use "registerusers" utility to configure users. All users should exist in the LDAP store.

BAM Distribution list can be created to send group email etc.

BAM Event generator can be used to create an event (Eg. if some one reject more than 50% orders)



General Items:

1. Audit level of the composite application, Engine and SOA infra.
2. Deployment scripts
3. Deployment segregation for each development group - Partition, SOA Bundle, Composite application etc.
4. Configuring the Server urls - use load balancer url instead of using the server urls. (Soa infrastructure common properties)
5. Sequence of the shutdown and start up (1. Shutdown the SOA Infra application first
6. Audit log file - setup (development, Off, Production)
7. Log file location and rotation.
8. Do you recommend one composite application per service - Depends on usage of service bus. Multiple services can be deployed in one composite application.
9. If no BAM sensors, how do we get the audit information reporting . - Use People sensores can be used to collect the Audit level information ...Use another reporting structure.
10. BPEL process state - synchronous vs Asynchronous.
11. Cap the Audit trail in database using BPEL Service Engine Properties.
12. Configure threads and audit trail settings for each engine.
13. Monitoring BPEL process and faults. Developer can add a fault handler framework.
14. Fault service - Can do bulk re-retry. It can generate email as well. Create single fault recovery, Build fault recovery.
15. Create Dehydration points for transaction oriented BPEL