Centralize WebLogic Server Log, Diagnostics Log, Stdout and Stderr Files

It is a common requirement at customers to redirect WebLogic server log and stdout files to a central directory other than the default location under the logs directory of a WebLogic server. The obvious advantages are log files are separated from WebLogic domain configuration files, which leave the domain less likely being corrupted by human errors. This becomes more obvious when an environment have more than one WebLogic domains.
To achieve this, first of all, determine a common root directory structure, under which server log and stdout files are redirected to. For example /u01/app/oracle/logs. In the case of multiple WebLogic domains, a subdirectory for each domain can be created under this common directory. Secondly, log configurations on each WebLogic domain are updated to use this common log directory. These configuration changes can be made either manually from the WebLogic Console and EM Console or using WebLogic WLST scripts.

The manual approach

1) Inside WebLogic Console, update the Log file name property under the Logging > General section of each WebLogic server. Set the Log file name to [COMMON_LOG_DIR]/[DOMAIN_NAME]/[SERVER_NAME].log. For example:
/u01/oracle/logs/iptSOAdomain01/soa_server1.log

2) Inside WebLogic Console, update the Server Start > Arguments section to add the below arguments to overwrite the deafult WebLogic Stdout, Stderr files location.

-Dweblogic.Stdout=[COMMON_LOG_DIR]/[DOMAIN_NAME]/[SERVER_NAME].out
-Dweblogic.Stderr=[COMMON_LOG_DIR]/[DOMAIN_NAME]/[SERVER_NAME]_err.out

For example:
-Dweblogic.Stdout=/u01/app/oracle/logs/iptSOAdomain01/soa_server1.out
-Dweblogic.Stderr=/u01/app/oracle/logs/iptSOAdomain01/soa_server1_err.out

3) In FMW Console, update the Log Files > Log Path property for each of the FMW log handlers to redirect the diagnostics files as needed to the new location
This configuration is made within the EM console of a WebLogic domain. To update this configuration, navigate to /[FMW_HOME]/[DOMAIN_NAME]/[SERVER_NAME] > Log Configuration > Log Files section of each WebLogic server. Update the Log Path of a log handler with the new location for the diagnostic log files as below.
[COMMON_LOG_DIR]/[DOMAIN_NAME]/{weblogic.Name}-diagnostic.log
For example:
/u01/app/oracle/logs/iptSOAdomain01/soa_server1-diagnostic.log

After making the above configuration changes, restart the WebLogic AdminServer and the managed servers on which the configurations were updated so that all WebLogic server logs, stdout, stderr and diagnostics logs information are redirected to the new central location.

The scripting approach

Almost all WebLogic tasks that are performed in WebLogic admin Console or EM console can also be achieved using WebLogic WSLT scripts. Below scripts can be used to make the configuration changes to redirect WebLogic server log file, stdout, stderr and diagnostics files to a different location than the default.

# This script is used to redirect WebLogic log, diagnostic and out files from the default locations to a central custom directory

domain=raw_input(“domain name:”)
adminHost=raw_input(“admin host:”)
adminPort=raw_input(“admin port:”)
adminUser=raw_input(“admin user:”)
adminPassword=raw_input(“admin password:”);

# The new log location
logPath=”/u01/app/oracle/logs/” + domain + “/”
connect(adminUser, adminPassword, “t3://” + adminHost + “:” + adminPort);
domainConfig()
servers = cmo.getServers()

edit()
startEdit()

cd(‘/Log/’ + domain)
cmo.setFileName(logPath + domain + ‘.log’)
save()

for server in servers:
sName = server.getName()
cd(‘/Servers/’ + sName + ‘/Log/’ + sName
cmo.setFileName(logPath + sName + ‘.log’)
cd(‘/Servers/’ + sName + ‘/ServerStart/’ + sName)
cmo.setArguments(cmo.getArguments() + ‘ -Dweblogic.Stdout=’ + logPath + sName + ‘.out’ + ‘ -Dweblogic.Stderr=’ + logPath + sName + ‘_err.out ‘)
configureLogHandler(target=sName, name=’odl-handler’, path=logPath + sName + ‘-diagnostic.log’)

save()
activate()

## End of wlst/python code

 

Experience Oracle JET

Recently I was pulled to work on an internal project which I got myself the opportunity to take the Oracle JET for a spin. No, I’m not talking about Larry’s personal JET, but the Oracle JavaScript Extension Toolkit, a set of Oracle’s JavaScript libraries with integrations with a collection of the popular open source JavaScript framework such as Knockout, JQuery, and Require.JS. Despite of the initial pain, the overall experience is fun and I love tool.

The basics

Oracle JET adopts the popular Model View ViewModel (MVVM) design pattern. If you’re from the MVC world, then the concept of MVVM should not be too new. You can think of the ViewModel part the equivalent of Control.

But, be warned, it can be overwhelming and slow initially with using Oracle JET due to the technologies and frameworks involved. Best way to get up to speed and productive is to read through the documentations (Get Started, Developer Guide), play with the Cookbook examples at Oracle JET site http://www.oracle.com/webfolder/technetwork/jet/index.html, and also read through the Knockout, JQuery documentations and tutorials if you’re new or rusty with these technologies.

To help quick start a new application, Oracle JET provides a few starter templates such as basic, navbar and navdrawer. After setting up the development environment per (http://www.oracle.com/webfolder/technetwork/jet/globalGetStarted.html ), a new application can be generated as below:

yo oraclejet <project name> --template=navbar

The rest of development work is to customize and extend the generated application template per business requirements. For standard Web applications with typical UI components such as Forms, Lists and Tables etc, the development processes are straightforward, assuming you’re well versed with HTML, CSS, JavaScript and are now familiar with Oracle JET and the JavaScript frameworks mentioned above.

Data visualization

Beyond the basics, the real power and excitement I found with Oracle JET is its support for rapid data visualization through a rich set of prebuilt diagrams, charts, grids and maps among many other goodies. Oracle JET provides over 100 data visualization cookbook examples across more than a dozen graph categories. (http://www.oracle.com/webfolder/technetwork/jet/jetCookbook.html?component=home&demo=rootVisualizations ). To add data visualization to your Web applications, can be as straightforward as coping the code from the Cookbook and then replace the data model with data according to your business requirements.

As examples, I created two data visualization examples below using my LinkedIn profile data.

Example 1: My profile outline – an illustration using Sunburst UI component

“max-width:600px; width:100%; height:600px;” id=“sunburst” data-bind=“ojComponent: { component: ‘ojSunburst’, animationOnDisplay: ‘auto’, animationOnDataChange:’auto’, nodes: nodeValues, innerRadius:.85, rootNodeContent: {renderer: centerCallback} }” > div>

Without going to the details, the customization work that’s done in this example involves:

a) Prepare my custom profile data in JSON format and store in the nodeValues variable in the Model;

b) In the centerCallback function in the Model, use HTML DOM to render my personal profile image and title (my name)

Example 2 : Employment vs. technology skills – an illustration using SankeyLayout UI component

“diagram” data-bind=“ojComponent: { component: ‘ojDiagram’, layout: layoutFunc, selectionMode: ‘none’, highlightMatch: ‘any’, highlightedCategories: highlightedCategoriesValue, hoverBehavior : hoverBehaviorValue, linkHighlightMode : ‘linkAndNodes’, nodeHighlightMode : ‘nodeAndLinks’, styleDefaults : styleDefaults, data : data, nodeProperties:function(data){return data}, linkProperties:function(data){return data}, tooltip: {renderer: tooltipFunction} }” style=“max-width:800px;width:100%;height:700px;”> div>

In this example, the customization work is to prepare my Employment vs. Skills data in JSON format in the model and store in the data variable.

Mobile support

Another powerful feature with Oracle JET I’d like to mention is its support for building hybrid Mobile applications for iOS, Android and Windows environments. This is done through the Apache Cordova container (http://cordova.apache.org/ ). To build a mobile application, you will need install Cordova in the development environment first and then generate the application template using one of the Oracle JET templates but provide with the hybrid option (rather than web) and specify the target mobile environment as below.

npm -g install cordova
yo oraclejet:hybrid <project name> --platforms=ios|android|windows

As with a Web application, the rest processes are to customize and extend the application template per your business requirements.

Well, Oracle JET offers far more than what I mentioned here. To experience the power and excitement, the best way is to try it yourself. Hope you have a fun ride.

Explore Apache Spark – Run the Example Applications on Windows 10

A good way to learn a new software tool or package is to start with the examples. It’s no exception with the Apache Spark cluster computing tool (http://spark.apache.org/ ). The Apache Spark installation provides a run-example (or run-example.cmd for Windows) script under the bin directory of the Spark installation to run the example applications.

However, when you try to run the script for any example application on a Windows environment, you may run into a few errors. Unfortunately the Spark online documentation doesn’t provide specific instructions for running the example applications on Windows environments.

So in this post I’m going to share a few tips learned to run the Spark Java examples successfully on a Windows environment. Or more specifically, Spark 2.1.0 on Windows 10.

1) First error, missing winutils.exe file

When you run the run-example.cmd script for an example application, for example:

 C:\spark-2.1.0-bin-hadoop2.7\bin>run-example org.apache.spark.examples.JavaSparkPi 2

You may encounter the error below:

ERROR Shell: Failed to locate the winutils binary in the hadoop binary path
java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.
        at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:379)
        at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:394)

As the error indicates, a winutils.exe file is needed in the “hadoop binaries” location. So there is an easy fix for this issue, which is to find and download a winutils.exe file that is compatible with the Spark version in use. For example: https://github.com/steveloughran/winutils/blob/master/hadoop-2.7.1/bin/winutils.exe.

Once you have the winutils.exe file, you need to put this file under the bin directory of a home directory to be set as HADOOP_HOME, for example: C:\Hadoop.  And in directory C:\Hadoop\bin you have the winutils.exe file.

2) Second error, failed to delete temporary directory

So with step 1, the example applications run successfully. However the below error occurs towards the end of the run.

java.io.IOException: Failed to delete: C:\Users\XXXXXX\AppData\Local\Temp\spark-db51e4a5-fc66-4516-9a71-f71e8d6987cd\userFiles-e64c83b8-8aa8-418c-9eb1-0cdb7eed82a4
        at org.apache.spark.util.Utils$.deleteRecursively(Utils.scala:1010)

This error was thrown b/c Spark script was not able to delete the temporary directory which appears to be still hold by some of the Spark components. This error can be safely ignored but it’s something the Spark application needs to fix.

3) Run the example applications using Java command directly

So far the example applications were run using run-example script. You can also run the example applications directly (i.e. not using the run-example script) using Java. In this case, you may need to provide a Spark master URL parameter to the application. For example:

java -Dspark.master=…  -cp … org.apache.spark.examples.JavaSparkPi 2

The format for the master URL on a local Windows environment should be as below.

-Dspark.master=local[N]

Where, N is the number of threads you want to assign to the process.

Providing the master URL with a non-local format such as below does not work.

-Dspark.master=spark://localhost:7077

Well, we passed the first step of trying out the Apache Spark example applications. So let’s dive deeper into Sparks, till then.

WebLogic 12C Domain Creation – a Comparison of Approaches

In this post, I’m going to discuss the different approaches to create WebLogic domains, and the pros and cons of each approach. In summary, there are three main approaches to create WebLogic domains: 1) use the WebLogic configuration script (config.sh or config.cmd on Windows); 2) use the WebLogic Scripting Tool (WLST); and 3) use Oracle OEM 12C. Furthermore, each approach may have various forms in executions depending on the situations. The below diagram illustrates these approaches and the variations in detail.

Use WebLogic Configuration Scripts (config.sh or config.cmd on Windows)

This is the most known and commonly used approach by customers. The configuration scripts can run in three modes: GUI mode, Console mode, and Silent mode.

GUI mode

On Windows or on a UNIX shell terminal with GUI enabled, the configuration script runs in GUI mode. This is the most intuitive approach of all. Users are guided through a wizard process to complete the domain creation. Towards the end of the configuration process, the user has the option to save the users inputs in relate to the domain creation in a Response file. This response file can later be used in the creation of WebLogic domains in Silent mode. The red triangle in the above diagram denotes this correlation.

Console mode

When executing the configuration scripts on a UNIX terminal without GUI support, then the configuration script runs on Console mode or interactive mode. This is the ancient old Text console mode with the users providing inputs through the keyboard interactively. This is no doubt the least intuitive approach.

Silent mode

The GUI and Console mode needs the user to provide inputs throughout the domain creations process. The script provides a Silent mode run unattended once started. To run the script in Silent mode, provide the – silent flag and -resposeFile followed by a response file (properties file or XML format) with information as needed for the domain creation.

The configuration scripts is good for creating WebLogic domains with the basic domain structures like clusters, servers, machines etc. But for advanced configurations such as data sources, security realms, network channels etc, these tasks are left out of the configuration processes using the configuration scripts. To complete these advanced configurations, the user can either logon to WebLogic console to complete the configurations manually or use WLST scripts.

 

Use WebLogic WLST Scripts

WSLT scripts are the APIs for the full life cycle management (create, manage, monitor) of WebLogic domains. Depending on the WebLogic domain templates and/or WLST scripts to use, there are three main variations of this approach.

Use the default WebLogic domain template (wls.jar)

The default WebLogic domain template provides the bare minimum structure of a WebLogic domain which includes a default WebLogic administration server (AdminServer). In WebLogic 12C, this template is located at <WEBLOGIC_HOME>/common/templates/wls.jar. Using WLST scripts, a WebLogic domain can be built out with any additional components as needed by the customer. The target domain structure and the components as needed are normally defined in domain properties file(s). Since the base template is simple and with no constraints, the user has the freedom to build out the target domain from the ground up to meet the requirements as desired. So this is the most flexible approach, which also requires writing more or extensive WLST scripts.

Use a custom domain template

In the case that the new domain to create is just an extension of an existing WebLogic domain or similar to an existing WebLogic domain, then it’s more convenient to create the new WebLogic domain based on the template of an existing WebLogic domain. A custom WebLogic domain template (in the form of a jar file) can be created from an existing WebLogic domain using the readDomain and writeTemplate WLST scripts. This template can later be used to create a new WebLogic domain. The red circle in the above diagram denotes this correlation.

This approach is ideal for creating a new WebLogic domain that’s similar or an extension of an existing WebLogic domain, such as adding new components (data sources, JMS resources …), updating properties (domain name, host, port …) etc.

Use custom domain WLST scripts exported from an existing WebLogic domain

Using the WLST configToScript script, an existing WebLogic domain can also be exported (or reverse-engineered) as a domain template in the form of WLST scripts rather than a jar file. These scripts can then be used to create a new WebLogic domain similar to the original WebLogic domain. The red star in the above diagram denotes this correlation.

There a few updatable properties (domain name and location, WebLogic user name and password etc) as defined in the exported domain.properties file. This generated scripts only run in online mode, which means it requires WebLogic domain with a WebLogic Admin server already running.

As any reverse-engineered scripts, the generated WLST is human readable but not very much user friendly. So trying to modify and customize the generated scripts for the creation of a new domain with many differences from the original domain it can be a hassle and error prone.

 

Use OEM 12C

Oracle OEM is Oracle’s grand tool for the life cycle management of Oracle FMW and database systems. OEM 12C provides the capability to provisioning WebLogic and FMW domains. There are two ways to provision a WebLogic domain through OEM: 1) go through a wizard process within OEM console, and 2) use the OEM command line EMCLI scripts. OEM provisions domains based on pre-defined Profiles, which are essentially a group of WebLogic WLST scripts organized in a specific flows. The setup of Profiles is not trivial, but once setup, it can be reused to automate the provisioning WebLogic domains. Another advantage is the WebLogic domain creation is under the control and management of a centralized tool. This approach depends on the availability of an OEM installation which may not apply to all the customers.

 

Oracle OHS 12C Configuration – a small Tip

In Oracle FMW 12C, an exciting improvement to the OHS architecture is its integration into the WebLogic domain based architecture. That means Oracle HTTP Server (OHS) instances are now configured as WebLogic domains just like most of the other Oracle FMW products (SOA, OSB, OAM, OIM etc). With this change, the life cycle (start, stop) and configurations of OHS server instances can be managed through FMW control console (/em), Node Manager or WLST scripts.

Here I want to share a small tip for configuring OHS instances on a WebLogic domain.

When extending or configuring a WebLogic domain for OHS, one of the steps is to configure OHS Server, below is an example.

Two fields on this screen are all that I want to talk about, Admin Host and Admin Port. Since OHS instances are now configured and managed within a WebLogic domain, it’s no surprise to assume the Admin Host here means the WebLogic AdminServer host, and the Admin Port means WebLogic AdminServer port. But these assumptions are wrong. If you had used the WebLogic AdminServer host and port here, the OHS server instances would fail to start, and no useful information would be logged in logs. To correct the error, you can correct the setting in file admin.conf at this location: ASERVER_DOMAIN/config/fmwconfig/component/OHS/<ohs instance> of each OHS instance.

So, to the point finally, the right information to provide for Admin Host and Admin Port on this screen is the Listen Address and port used for the communication of OHS server with the Node Manager of the OHS domain. The formal definition from the Oracle document (Oracle® Fusion Middleware Creating WebLogic Domains Using the Configuration Wizard 12c (12.1.3)) is copied below.

Admin Host: The listen address to use for the selected OHS server for communication with Node Manager. The address should only allow loopback communication within the host (for example, 127.0.0.1).

Admin Port: The listen port to use for the selected OHS server for communication with Node Manager on this system. The port must be unique.

Aside from these two fields that could be confusing, the rest of screens are pretty standard and straight forward. One final word, with the familiar WebLogic interface, you’ll find OHS 12C is a lot easier to configure and manage. So try it out.