Skip navigation.
Downloads | Product Documentation | Support

Configuring and Managing WebLogic JDBC

# Using WebLogic Server with Oracle RAC

More and more customers are looking for solutions to make their back-end systems more scalable and more available. In response to these requests, BEA supports Oracle Real Application Clusters (RAC) for use with WebLogic Server.

The following sections describe the requirements and configuration tasks for using Oracle Real Application Clusters with WebLogic Server:

- Overview of Oracle Real Application Clusters
- Environment
- Configuration Considerations for Oracle
- Configuration Options in WebLogic Server with Oracle RAC
- XA Considerations and Limitations with Oracle RAC
- JDBC Store Recovery with Oracle RAC

Both Oracle RAC and WebLogic Server are complex systems. To use them together requires specific configuration on both systems, as well as clustering software and a shared storage solution. This document describes the configuration required at a high level. For more details about configuring Oracle RAC, your clustering software, your operating system, and your storage solution, see the documentation from the respective vendors.

# Overview of Oracle Real Application Clusters

Oracle Real Application Clusters (RAC) is a software component you can add to a high-availability solution that enables users on multiple machines to access a single database with increased performance. RAC comprises two or more Oracle database instances running on two or more clustered machines and accessing a shared storage device via cluster technology. To support this architecture, the machines that host the database instances are linked by a high-speed interconnect to form the cluster. The interconnect is a physical network used as a means of communication between the nodes of the cluster. Cluster functionality is provided by the operating system or compatible third party clustering software. Figure B-1 shows an Oracle RAC configuration.

Figure B-1 Oracle Real Application Clusters Configuration

Oracle RAC offers features in the following areas:

- Scalability
- Availability
- Load balancing
- Failover

## Oracle RAC Scalability with WebLogic Server

An Oracle RAC installation appears like a single standard Oracle database and is maintained using the same tools and practices. All the nodes in the cluster execute transactions against the same database and RAC coordinates each node's access to the shared data to maintain consistency and ensure integrity. You can add nodes to the cluster easily and there is no need to partition data when you add them. This means that you can horizontally scale the database tier as usage and demand grows by adding RAC nodes, storage, or both. You can then scale WebLogic Server by adding a data source that maps to the new node.

## Oracle RAC Availability with WebLogic Server

Because every RAC node in the cluster has equal access and authority, the loss of a node may impact performance but does not result in downtime. Depending upon your configuration, when a RAC node fails, in-flight transactions are redirected to another node in the cluster either by WebLogic Server or by the Oracle Thin driver. Note that Oracle RAC does not provide failover for database connections; nor does WebLogic Server. But transactions are failed over in the sense that they are driven to completion, based on the time of the failure.

## Oracle RAC Load Balancing with WebLogic Server

If your application requires load balancing across RAC nodes, BEA supports this capability through use of JDBC multi data sources with Oracle RAC nodes. The data sources that form a multi data source are accessed using a round-robin scheme. When switching connections, WebLogic Server selects a connection from the next data source in the order listed. For more information about using multi data sources with Oracle RAC, see Using Multi Data Sources with Oracle RAC

When multi data sources are not an option, load balancing when using XA is not supported. In a configuration without a multi data source, WebLogic Server relies on the connect-time failover feature provided by the Oracle Thin driver to work with Oracle RAC. As described in Oracle's Technical Note 235118.1, the Oracle Thin driver cannot guarantee that a transaction is initiated and concluded on the same Oracle RAC instance when the driver is configured for load balancing. As Oracle RAC requires that all database operations inside a global transaction be routed to the same Oracle instance, this known limitation means that you cannot use driver-level load balancing when using XA with Oracle RAC and therefore you cannot use a primary/primary RAC configuration.

## Oracle RAC Failover with WebLogic Server

Although Oracle RAC offers JDBC connect-time failover features, for most configurations, BEA recommends using WebLogic JDBC multi data sources to handle failover instead. While connect-time failover does not provide the ability to pre-create connections to alternate RAC nodes, multi data sources have multiple connections available at all times to handle failover. For more information see Using Multi Data Sources with Oracle RAC.

Note: Transparent Application Failover (TAF) requires the use of the Oracle OCI driver. Because BEA requires the use of the Oracle Thin driver, TAF is not supported.

# Environment

When using WebLogic Server with Oracle RAC, consider the following requirements:

- Hardware Requirements
- Software Requirements

Note: See the WebLogic Platform Supported Configurations documentation for the latest WebLogic Server hardware platform and operating system support, and for the Oracle RAC versions supported by WebLogic Server versions and service packs. See the Oracle documentation for hardware and software requirements for running the Oracle RAC software.

## Hardware Requirements

A typical WebLogic Server/Oracle RAC system includes a WebLogic Server cluster, an Oracle RAC cluster, and hardware for shared storage.

### WebLogic Server Cluster

The WebLogic Server cluster can be configured in many ways and with various hardware options. See Using WebLogic Server Clusters for more details about configuring a WebLogic Server cluster.

### Oracle RAC Cluster

For the latest hardware requirements for Oracle RAC, see the Oracle RAC documentation. However, to use Oracle RAC with WebLogic Server, you must run Oracle RAC instances on robust, production-quality hardware. The Oracle RAC configuration must deliver database processing performance appropriate for reasonably-anticipated application load requirements. Unusual database response delays can lead to unexpected behavior during database failover scenarios.

### Shared Storage

In an Oracle RAC configuration, all data files, control files, and parameter files are shared for use by all RAC instances. An HA storage solution that uses one of the following architectures is recommended:

- Direct Attached Storage (DAS), such as a dual ported disk array or a Storage Area Network (SAN)
- Network Attached Storage (NAS)

For a complete list of supported storage solutions, see your Oracle documentation.

## Software Requirements

To use WebLogic Server with Oracle RAC, you must install the following software on each RAC node:

- Operating system patches required to support Oracle RAC. See the release notes from Oracle for details.
- Oracle 9i or Oracle 10g database management system
- Clustering software for your operating system. See the Oracle documentation for supported clustering software and cluster configurations.
- Shared storage software, such as Veritas Cluster File System. Note that some clustering software includes a file storage solution, in which case additional shared storage software is not required.

Note: See the WebLogic Platform Supported Configurations documentation for the latest WebLogic Server hardware platform and operating system support, and for the Oracle RAC versions supported by WebLogic Server versions and service packs. See the Oracle documentation for hardware and software requirements required for running the Oracle RAC software.

# Configuration Considerations for Oracle

Once you have installed and configured Oracle RAC, you must configure the listener process for each RAC instance as described in the sections that follow. For information about installing and configuring your operating system and the Oracle software for Oracle RAC nodes see the Oracle documentation.

## Configuring the Listener Process for Each Oracle RAC Instance

For Oracle RAC, the listener process establishes a communication pathway between a user process and an Oracle instance. When you use Oracle RAC with WebLogic Server, the user process is typically a JDBC data source.

When a data source is created, it attempts to create a pool of database connections for applications to borrow. If a pooled database connection becomes inoperative or if the data source is configured to grow in capacity, the data source attempts to create additional database connections up to the maximum specified in the configuration file. In all of these instances, the Oracle listener process handles the connection request on the Oracle RAC instance.

Figure B-2 shows the Oracle listener process functionality.

Figure B-2 Oracle Listener Process Functionality

To enable this functionality, you must configure the listener process for each RAC instance in the Oracle cluster. BEA requires that you configure a local listener on each RAC instance. Each database instance should be configured to register with its local listener only.

Oracle instances can be configured to register with the listener statically in the `listener.ora` file or registered dynamically using the instance initialization parameter `local_listener`, or both. BEA recommends using dynamic registration.

A listener can start either a shared dispatcher process or a dedicated process. When using with WebLogic Server, BEA recommends using dedicated processes.

## Disabling Remote Listeners

Although Oracle RAC allows you to configure remote listeners to handle connection failover, remote listeners are typically too slow and BEA does not support their use. To disable remote listeners, delete any listed remote listeners in `spfile.ora` on each RAC node. For example:

>     *.remote_listener=''

BEA recommends using JDBC multi data sources to handle failover instead.

For more information see Configuration Considerations for Failover.

# Configuration Options in WebLogic Server with Oracle RAC

When using WebLogic Server with Oracle 9i RAC or Oracle 10g RAC, you must configure your WebLogic Domain so that it can interact with RAC instances and so that it performs as expected. The following sections describe configuration options and requirements:

- Choosing a WebLogic Server Configuration for Use with Oracle RAC
- Required JDBC Drivers
- Configuration Considerations for Failover
- Using Multi Data Sources with Oracle RAC
- Using Multi Data Sources with Global Transactions
- Using Multi Data Sources without Global Transactions
- Using Connect-Time Failover with Oracle RAC
- Using Connect-Time Failover without Global Transactions

## Choosing a WebLogic Server Configuration for Use with Oracle RAC

BEA supports several configuration options for using Oracle RAC with WebLogic Server:

- To connect to multiple Oracle 9i RAC or Oracle 10g RAC instances when using global transactions (XA), BEA recommends the use of transaction-aware WebLogic JDBC multi data sources, which support failover and load balancing, to connect to the RAC nodes. For more information see Using Multi Data Sources with Global Transactions.
- To connect to multiple Oracle 9i RAC or Oracle10g RAC instances when not using XA, BEA recommends the use of (non-transaction-aware) multi data sources to connect to the RAC nodes. Use the standard multi data source configuration, which supports failover and load balancing. For more information see Using Multi Data Sources without Global Transactions.
- To connect to multiple Oracle RAC nodes when multi data sources are not an option and when not using XA, use Oracle 9i RAC or Oracle 10g RAC with connect-time failover. Note that load balancing is not supported in this configuration. For more information see Using Connect-Time Failover without Global Transactions.

The following table may help you as you try to determine which configuration is right for your particular application:

Does Your Application Require |
--- | ---
Load Balancing? | Failover? | Global Transactions (XA)? | JDBC Store?
Y | Y | Y | N | See Using Multi Data Sources with Global Transactions
Y | Y | N | Y | See Using Multi Data Sources without Global Transactions
N | Y | N | Y | See Using Connect-Time Failover without Global Transactions

## Required JDBC Drivers

To use WebLogic Server with Oracle RAC, your WebLogic JDBC data sources must use the Oracle JDBC Thin driver 10g to create database connections.

## Configuration Considerations for Failover

Consider the following information when configuring for failover.

### Multi Data Source-Managed Failover

Multi data sources offer failover for global transactions without the limitations and known issues associated with a data source configuration with connect-time failover. For a description of multi data source failover features, see "Multi Data Source Failover Enhancements" in Programming WebLogic JDBC.

With this configuration, pictured in Figure B-3, you get:

- Faster failover controlled by the multi data source
- Automatic failback by the WebLogic Server health monitor

The multi data source handles failover for database connections when a RAC node becomes unavailable. When WebLogic Server tests a connection and the connection fails, it attempts to recreate the connection. If that attempt fails, the server disables the data source and routes connection requests to other data sources (which correspond to other RAC nodes) in the multi data source. WebLogic Server periodically tries to recreate the database connections in the disabled data source. When WebLogic Server is successful in recreating the connections, it next re-enables the data source and begins routing connection requests to the data source again. Because of the connection request routing and automatic health checking features, there is minimal delay in satisfying connection requests after a failure compared to when relying on the Oracle Thin driver connect-time failover configuration.

### Connect-Time Failover

When multi data sources are not an option, WebLogic Server relies on the connect-time failover feature of the Oracle Thin driver to handle connection failover when a RAC instance becomes unavailable and a primary/primary configuration is not an option. When WebLogic Server tests a connection and the connection fails, the server replaces it by getting a new connection, and the driver again determines which RAC instance to use based on instance availability.When a connection fails a connection test, WebLogic Server automatically closes all connections in the data source. WebLogic Server replaces the connection with a new one, relying on the driver to determine to which node it should connect. In this case, the primary RAC node has failed, so the new connection is to the secondary RAC node. WebLogic Server tests the new connection before satisfying the request.

### Delays During Failover

Occasionally, when one RAC node fails over to another, there may be a delay before the data associated with a transaction branch in progress on the now failed node is available throughout the cluster. This prevents incomplete transactions from being properly completed, which could further result in data locking in the database. To protect against the potential consequences of such a delay, WebLogic Server provides two configuration attributes that enable XA call retry for Oracle RAC: `XARetryDurationSeconds` and `XARetryIntervalSeconds`.

`XARetryDurationSeconds` controls the period of time during which WebLogic Server will repeatedly retry XA operations such as recover, commit and rollback for pending transactions. `XARetryIntervalSeconds` controls the frequency of the retry attempts within the established time period.

To enable XA call retries, add a value for `XARetryDurationSeconds` to all JDBC data sources in your WebLogic domain that connect to an Oracle RAC instance. For example:

>     <jdbc-data-source xmlns="http://www.bea.com/ns/weblogic/91"
>       xmlns:sec="http://www.bea.com/ns/weblogic/91/security"
>       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>       xmlns:wls="http://www.bea.com/ns/weblogic/91/security/wls"
>       xsi:schemaLocation="http://www.bea.com/ns/weblogic/91/domain.xsd">
>       <name>oracleRACXAPool</name>
>       ...
>       <jdbc-xa-params>
>         ...
>         <xa-retry-duration-seconds>300</xa-retry-duration-seconds>
>       </jdbc-xa-params>
>     </jdbc-data-source>

Use the following formula to determine the value for `XARetryDurationSeconds:`

`XARetryDurationSeconds` = (longest transaction timeout for transactions that use connections from the data source) + (delay before XIDs are available on all RAC nodes, typically less than 5 minutes)

For example, if your application sets the longest transaction timeout as 180 seconds, you should set `XARetryDurationSeconds` to 180 seconds + 300 seconds, for a total of 480 seconds.

Note: It is generally better to set `XARetryDurationSeconds` higher than minimally necessary to make sure that all transactions are completed properly. Setting the value higher than minimally required should not affect application performance during normal operations. The additional processing only affects transactions that have been prepared but have failed to complete.

You can also optionally set a value for `XARetryIntervalSeconds`. This value determines the time between XA retry calls. By default, the value is 60 seconds. Decreasing the value will decrease the amount of time between XA retry attempts. The default value should suffice in most cases.

To enable `XARetryDurationSeconds` and `XARetryIntervalSeconds` from the Administration Console, use the following steps:

1. If you have not already done so, in the Change Center of the Administration Console, click Lock & Edit.
2. In the Domain Structure tree, expand Services > JDBC, then select Data Sources.
3. On the Summary of Data Sources page, click the data source name.
4. Select the Configuration: Connection Pool tab.
5. Scroll down and click Advanced to show the advanced connection pool options.
6. Update `XA Retry Duration` and `XA Retry Interval`.
7. Click Save.

Optionally, you can use the `weblogic.Admin` command line utility, the WebLogic Scripting Tool (WLST), or a JMX program.

### Failure Handling Walkthrough for Global Transactions

What happens to inflight transactions to a database node if that node fails? When the primary Oracle RAC node fails? Does WebLogic Server support transparent failover? To answer these and other questions about how WebLogic Server handles failures, let's walk through the transaction processing steps and describe how a failure would be handled at each stage along the way.

The first stage at which a failure may occur is before the application calls for the transaction to be committed. If a database or RAC node fails at this stage, the application receives an exception and must get a new connection and make a new attempt at processing the transaction. WebLogic Server does not support transparent failover.

If a failure occurs after the application has called for the transaction to be committed, the handling of any in-flight transaction depends upon whether the `PREPARE` operation is complete. If the `PREPARE` operation is not complete, the transaction manager rolls back the transaction and sends the application an exception for the failed transaction. If the `PREPARE` operation is complete, the transaction manager attempts to drive the in-flight transaction to completion using another node.

If a failure occurs during the `COMMIT` operation, the transaction manager attempts to retry the `COMMIT` operation several times. Note that the connection is blocked during these attempts. If the `COMMIT` operation is not successful during the first set of retry attempts, the application receives an exception. The transaction manager then continues to retry the `COMMIT` operation periodically until it is successful; if the transaction cannot be completed successfully within the abandon time period, the transaction is driven to completion heuristically.

## Using Multi Data Sources with Oracle RAC

To connect WebLogic Server to multiple Oracle RAC nodes using multi data sources, first configure a JDBC data source for each RAC instance in your RAC cluster with the Oracle Thin driver. Then configure a multi data source, using either the algorithm for load balancing or the algorithm for failover, and add the data sources to it.

Figure B-3 shows a typical multi data source configuration.

Figure B-3 Multi Data Source Configuration

You can use the Administration Console or any other means that you prefer to configure your domain, such as the `weblogic.Admin` command line utility, the WebLogic Scripting Tool (WLST), or a JMX program. For information about configuring a WebLogic JDBC multi data source see Configuring JDBC Multi Data Sources.

To use a database connection in this configuration, your applications look up one multi data source on the JNDI tree and then request a connection. The multi data source determines which data source to use to satisfy the connection request based on the algorithm type specified in the configuration (that is, failover or load balancing).

### Attributes of a Multi Data Source

The multi data source may have the following attributes, depending on the role of RAC in your system—load balancing or failover:

- `AlgorithmType="Load-Balancing"` or `AlgorithmType="Failover"`

- With the Load-Balancing option, connection requests are distributed among available data sources; with the High-Availability option, connection requests are served by the first available pool in the list. When a data source becomes defunct, connection requests are served by the next data source in the list.

- `FailoverRequestIfBusy="true"`

- With the Failover algorithm, this attribute enables failover when all connections in a data source are in use.

- `TestFrequencySeconds="``120``"`

- This attribute controls the frequency at which WebLogic Server checks the health of data sources previously marked as unhealthy to see if connections can be recreated and if the data source can be re-enabled. For more details see Configuring JDBC Multi Data Sources.

## Using Multi Data Sources with Global Transactions

In this configuration, a multi data source "pins" a transaction to one and only one Oracle RAC instance and failover is handled at the multi data source level when a RAC instance becomes unavailable. If there is a failure on a RAC instance before PREPARE, the operation is retried until the retry duration has expired. If there is a failure after PREPARE the transaction is failed over to another instance.

### Rules for Data Sources within a Multi Data Source Using Global Transactions

The following rules apply to the XA data sources within a multi data source:

- All the data sources must be homogeneous. In other words, either all of them must use an XA driver or none of them can use an XA driver.
- If you choose to specify them, all XA-related attributes must be set to the same values for each data source. The attributes include the following:

- `XARetryDurationSeconds`
- `SupportsLocalTransaction`
- `KeepXAConnTillTxComplete`
- `NeedTxCtxOnClose`
- `XAEndOnlyOnce`
- `NewXAConnForCommit`
- `RollbackLocalTxUponConnClose`
- `RecoverOnlyOnce`
- `KeepLogicalConnOpenOnRelease`

Note: If you are not using XA, BEA recommends the use of multi data sources for failover and load balancing across RAC instances, but the XA-specific configuration requirements above do not apply. For more information see Using Multi Data Sources without Global Transactions.

### Required Attributes of Data Sources within a Multi Data Source Using Global Transactions

Each data source within the multi data source should have the following attributes:

- Oracle JDBC Thin driver 10g. For example:

>     <url>jdbc:oracle:thin:@lcqsol24:1521:SNRAC1</url>
>     <driver-name>oracle.jdbc.xa.client.OracleXADataSource</driver-name>

- `KeepXAConnTillTxComplete="true"`

- Forces the data source to reserve a physical database connection and provide the same connection to an application throughout transaction processing until the distributed transaction is complete.
- Required for proper transaction processing with Oracle RAC.

- `XARetryDurationSeconds="300"`

- Enables the WebLogic Server transaction manager to retry XA recover, commit, and rollback calls for the specified amount of time.

- `TestConnectionsOnReserve="true"`

- Enables testing of a database connection when an application reserves a connection from the data source. See Test Connections on Reserve to Enable Fail-Over for more details about this attribute.
- Required to enable failover to another RAC node.

- `TestTableName="``name_of_small_table``"` The name of the table used to test a physical database connection. For more details about this attribute, see Connection Testing Options for a Data Source.

### Sample Configuration Code

Sample configuration code for a multi data source and two associated data sources is shown below.

>     <jdbc-data-source xmlns="http://www.bea.com/ns/weblogic/91"
>       xmlns:sec="http://www.bea.com/ns/weblogic/91/security"
>       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>       xmlns:wls="http://www.bea.com/ns/weblogic/91/security/wls"
>       xsi:schemaLocation="http://www.bea.com/ns/weblogic/91/domain.xsd">
>       <name>oracleRACXAPool</name>
>       <jdbc-driver-params>
>         <url>jdbc:oracle:thin:@lcqsol24:1521:SNRAC1</url>
>         <driver-name>oracle.jdbc.xa.client.OracleXADataSource</driver-name>
>         <properties>
>           <property>
>             <name>user</name>
>             <value>wlsqa</value>
>           </property>
>         </properties>
>         <password-encrypted>{3DES}aP/xScCS8uI=</password-encrypted>
>       </jdbc-driver-params>
>       <jdbc-connection-pool-params>
>         <test-table-name>SQL SELECT 1 FROM DUAL</test-table-name>
>         <profile-type>0</profile-type>
>       </jdbc-connection-pool-params>
>       <jdbc-data-source-params>
>         <jndi-name>oracleRACXAJndiName</jndi-name>
>         <global-transactions-protocol>TwoPhaseCommit
>             </global-transactions-protocol>
>       </jdbc-data-source-params>
>       <jdbc-xa-params>
>         <keep-xa-conn-till-tx-complete>true</keep-xa-conn-till-tx-complete>
>         <xa-end-only-once>true</xa-end-only-once>
>         <xa-set-transaction-timeout>true</xa-set-transaction-timeout>
>         <xa-transaction-timeout>120</xa-transaction-timeout>
>         <xa-retry-duration-seconds>300</xa-retry-duration-seconds>
>       </jdbc-xa-params>
>     </jdbc-data-source>

>     <jdbc-data-source xmlns="http://www.bea.com/ns/weblogic/91"
>       xmlns:sec="http://www.bea.com/ns/weblogic/91/security"
>       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>       xmlns:wls="http://www.bea.com/ns/weblogic/91/security/wls"
>       xsi:schemaLocation="http://www.bea.com/ns/weblogic/91/domain.xsd">
>       <name>oracleRACXAPool2</name>
>       <jdbc-driver-params>
>         <url>jdbc:oracle:thin:@lcqsol25:1521:SNRAC2</url>
>         <driver-name>oracle.jdbc.xa.client.OracleXADataSource</driver-name>
>         <properties>
>           <property>
>             <name>user</name>
>             <value>wlsqa</value>
>           </property>
>         </properties>
>         <password-encrypted>{3DES}aP/xScCS8uI=</password-encrypted>
>       </jdbc-driver-params>
>       <jdbc-connection-pool-params>
>         <test-table-name>SQL SELECT 1 FROM DUAL</test-table-name>
>         <profile-type>0</profile-type>
>       </jdbc-connection-pool-params>
>       <jdbc-data-source-params>
>         <jndi-name>oracleRACXAJndiName2</jndi-name>
>         <global-transactions-protocol>TwoPhaseCommit
>             </global-transactions-protocol>
>       </jdbc-data-source-params>
>       <jdbc-xa-params>
>         <keep-xa-conn-till-tx-complete>true</keep-xa-conn-till-tx-complete>
>         <xa-end-only-once>true</xa-end-only-once>
>         <xa-set-transaction-timeout>true</xa-set-transaction-timeout>
>         <xa-transaction-timeout>120</xa-transaction-timeout>
>         <xa-retry-duration-seconds>300</xa-retry-duration-seconds>
>       </jdbc-xa-params>
>     </jdbc-data-source>

>     <jdbc-data-source xmlns="http://www.bea.com/ns/weblogic/91"
>       xmlns:sec="http://www.bea.com/ns/weblogic/91/security"
>       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>       xmlns:wls="http://www.bea.com/ns/weblogic/91/security/wls"
>       xsi:schemaLocation="http://www.bea.com/ns/weblogic/91/domain.xsd">
>       <name>oracleRACXAMDS</name>
>       <jdbc-data-source-params>
>         <jndi-name>oracleRACMDSJndiName</jndi-name>
>         <algorithm-type>Load-Balancing</algorithm-type>
>         <data-source-list>oracleRACXAPool,oracleRACXAPool2</data-source-list>
>       </jdbc-data-source-params>
>     </jdbc-data-source>

## Using Multi Data Sources without Global Transactions

The following sections describe a configuration that uses Oracle RAC with multi data sources in an application that does not require global transactions.

### Attributes of Data Sources within a Multi Data Source Not Using Global Transactions

Data sources must have the following attributes:

- Oracle JDBC Thin driver 10g. For example:

>      <url>jdbc:oracle:thin:@lcqsol24:1521:SNRAC1</url>
>      <driver-name>oracle.jdbc.xa.client.OracleXADataSource</driver-name>

- `TestConnectionsOnReserve="true"`

- Enables testing of a database connection when an application reserves a connection from the data source. Test Connections on Reserve to Enable Fail-Over on page 4-3 for more details about this attribute.
- Required to enable failover and connection request routing within a multi data source (effectively, failover to another RAC node).

- `TestTableName="``name_of_small_table``"`

- The name of the table used to test a physical database connection. For more details about this attribute, see Connection Testing Options for a Data Source.

### Sample Configuration Code

Sample configuration code for a WebLogic JDBC multi data source and associated data sources is shown below.

>     <jdbc-data-source xmlns="http://www.bea.com/ns/weblogic/91"
>       xmlns:sec="http://www.bea.com/ns/weblogic/91/security"
>       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>       xmlns:wls="http://www.bea.com/ns/weblogic/91/security/wls"
>       xsi:schemaLocation="http://www.bea.com/ns/weblogic/91/domain.xsd">
>       <name>jdbcPool</name>
>       <jdbc-driver-params>
>         <url>jdbc:oracle:thin:@lcqsol24:1521:snrac1</url>
>         <driver-name>oracle.jdbc.OracleDriver</driver-name>
>         <properties>
>           <property>
>             <name>user</name>
>             <value>wlsqa</value>
>           </property>
>         </properties>
>         <password-encrypted>{3DES}aP/xScCS8uI=</password-encrypted>
>       </jdbc-driver-params>
>       <jdbc-connection-pool-params>
>         <test-connections-on-reserve>true</test-connections-on-reserve>
>         <test-table-name>SQL SELECT 1 FROM DUAL</test-table-name>
>       </jdbc-connection-pool-params>
>       <jdbc-data-source-params>
>         <jndi-name>jdbcDataSource</jndi-name>
>       </jdbc-data-source-params>
>     </jdbc-data-source>

>     <jdbc-data-source xmlns="http://www.bea.com/ns/weblogic/91"
>       xmlns:sec="http://www.bea.com/ns/weblogic/91/security"
>       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>       xmlns:wls="http://www.bea.com/ns/weblogic/91/security/wls"
>       xsi:schemaLocation="http://www.bea.com/ns/weblogic/91/domain.xsd">
>       <name>jdbcPool2</name>
>       <jdbc-driver-params>
>         <url>jdbc:oracle:thin:@lcqsol25:1521:SNRAC2</url>
>         <driver-name>oracle.jdbc.OracleDriver</driver-name>
>         <properties>
>           <property>
>             <name>user</name>
>             <value>wlsqa</value>
>           </property>
>         </properties>
>         <password-encrypted>{3DES}aP/xScCS8uI=</password-encrypted>
>       </jdbc-driver-params>
>       <jdbc-connection-pool-params>
>         <test-connections-on-reserve>true</test-connections-on-reserve>
>         <test-table-name>SQL SELECT 1 FROM DUAL</test-table-name>
>       </jdbc-connection-pool-params>
>       <jdbc-data-source-params>
>         <jndi-name>jdbcDataSource2</jndi-name>
>         <global-transactions-protocol>OnePhaseCommit
>              </global-transactions-protocol>
>       </jdbc-data-source-params>
>     </jdbc-data-source>

>     <jdbc-data-source xmlns="http://www.bea.com/ns/weblogic/91"
>       xmlns:sec="http://www.bea.com/ns/weblogic/91/security"
>       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>       xmlns:wls="http://www.bea.com/ns/weblogic/91/security/wls"
>       xsi:schemaLocation="http://www.bea.com/ns/weblogic/91/domain.xsd">
>       <name>jdbcNonXAMultiPool</name>
>       <jdbc-data-source-params>
>         <jndi-name>jdbcDataSource</jndi-name>
>         <algorithm-type>Failover</algorithm-type>
>         <data-source-list>jdbcPool,jdbcPool2</data-source-list>
>         <failover-request-if-busy>true</failover-request-if-busy>
>       </jdbc-data-source-params>
>     </jdbc-data-source>

Note: Line breaks added for readability.

## Using Connect-Time Failover with Oracle RAC

When multi data sources are not an option in your application, you can configure your data sources to use connect-time failover and load balancing. To connect WebLogic Server to multiple Oracle RAC nodes using data sources configured for connect-time failover and load balancing, configure a JDBC data source for each RAC instance in your RAC cluster with the Oracle Thin driver, as described in the sections that follow. Figure B-4 shows an overview of the system.

Figure B-4 Data Source Configuration with Oracle Thin Driver Connect-Time Failover

You can use the Administration Console or any other means that you prefer to configure your domain, such as the `weblogic.Admin` command line utility, the Weblogic Scripting Tool (WLST), or a JMX program.

When connections are created in the data source, the Oracle Thin driver determines which Oracle RAC instance to use. When an application gets a connection, it looks up a data source on the JNDI tree and requests a connection from the data source. The data source delivers one of the available connections from the pool of connections in the data source.

The following sections describe configuration options and requirements:

- Using Connect-Time Failover without Global Transactions
- XA Considerations and Limitations with Oracle RAC
- JDBC Store Recovery with Oracle RAC

## Using Connect-Time Failover without Global Transactions

The following sections describe a configuration which uses Oracle RAC's connect-time failover features to handle connection failures. With this configuration, in some failure cases, the failover time is as long as the TCP timeout, which can be several minutes, depending on your environment.

### Attributes of a Connect-Time Failover Configuration without Global Transactions

To use this configuration, create JDBC data sources in your WebLogic domain with the following attributes.

- Oracle JDBC Thin driver 10g configured for connect-time failover. For example:

>     <url>jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=
>         (PROTOCOL=TCP)(HOST=lcqsol24)(PORT=1521))(ADDRESS=(PROTOCOL=TCP)
>         (HOST=lcqsol25)(PORT=1521))(FAILOVER=on)(LOAD_BALANCE=off))
>         (CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=snrac)))</url>
>     <driver-name>oracle.jdbc.OracleDriver</driver-name>

- `ConnectionReserveTimeoutSeconds="120"`

- Enables application requests for a connection to wait 120 seconds for a connection to become available.

- `TestConnectionsOnReserve="true"`

- Enables testing of a database connection when an application reserves a connection from the data source. See Test Connections on Reserve to Enable Fail-Over for more details about this attribute.
- Required to enable failover to another RAC node.

- `TestTableName="``name_of_small_table``"` The name of the table used to test a physical database connection. For more details about this attribute, see Connection Testing Options for a Data Source.

### Sample Configuration Code

Sample configuration code is shown below.

>     <jdbc-data-source xmlns="http://www.bea.com/ns/weblogic/91"
>       xmlns:sec="http://www.bea.com/ns/weblogic/91/security"
>       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>       xmlns:wls="http://www.bea.com/ns/weblogic/91/security/wls"
>       xsi:schemaLocation="http://www.bea.com/ns/weblogic/91/domain.xsd">
>       <name>oracleRACNonXAPool</name>
>       <jdbc-driver-params>
>         <url>jdbc:oracle:thin:@(DESCRIPTION=
>              (ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)
>              HOST=lcqsol24)(PORT=1521))(ADDRESS=(PROTOCOL=TCP)
>              (HOST=lcqsol25)(PORT=152))(FAILOVER=on)
>              (LOAD_BALANCE=off))(CONNECT_DATA=(SERVER=DEDICATED)
>              (SERVICE_NAME=snrac)))</url>
>         <driver-name>oracle.jdbc.OracleDriver</driver-name>
>         <properties>
>           <property>
>             <name>user</name>
>             <value>wlsqa</value>
>           </property>
>         </properties>
>         <password-encrypted>{3DES}aP/xScCS8uI=</password-encrypted>
>       </jdbc-driver-params>
>       <jdbc-connection-pool-params>
>         <test-connections-on-reserve>true</test-connections-on-reserve>
>         <test-table-name>SQL SELECT 1 FROM DUAL</test-table-name>
>         <profile-type>4</profile-type>
>       </jdbc-connection-pool-params>
>       <jdbc-data-source-params>
>         <jndi-name>oracleRACJndiName</jndi-name>
>         <global-transactions-protocol>OnePhaseCommit
>              </global-transactions-protocol>
>       </jdbc-data-source-params>
>     </jdbc-data-source>

Note: Line breaks added for readability.

# XA Considerations and Limitations with Oracle RAC

When using XA (global transactions) with Oracle RAC, consider the following requirements and limitations.

## Required JDBC Driver Configuration for Use with XA

In this configuration, you must use the Oracle Thin driver connect-time failover to create database connections as described in Using Connect-Time Failover without Global Transactions.

## Oracle 9i RAC XA Requirements

Oracle 9i RAC has the following requirements when using XA.

### A Global Transaction Must Be Initiated, Prepared, and Concluded in the Same Instance of the RAC Cluster

Global transactions must be initiated, prepared, and concluded in the same instance of the RAC cluster. WebLogic Server data sources manage this for you when you set `KeepXAConnTillTxComplete="true"` in the JDBC data source configuration.

Note: WebLogic Server relies on the connect-time failover feature in the Oracle Thin driver to work with Oracle RAC. As described in Oracle's Technical Note 235118.1, the Oracle Thin driver cannot guarantee that a transaction is initiated and concluded on the same RAC instance when the driver is configured for load balancing. As Oracle RAC requires that all database operations inside a global transaction be routed to the same Oracle instance, this known limitation means that you cannot use connect-time load balancing when using XA with Oracle RAC and, therefore, you cannot use a primary/primary RAC configuration.

### Transaction IDs Must Be Unique Within the RAC Cluster

When using global transactions, transaction IDs (XIDs) must be unique within the RAC cluster. However, neither the Oracle Thin driver nor an Oracle RAC instance can determine if an XID is unique within the RAC cluster. Transactions with the same XID can execute SQL code on different instances of the RAC cluster without any exception.

The WebLogic Server Transaction Manager generates unique transaction IDs. However, in some failover scenarios, a transaction can continue on a RAC instance other than the originating instance, which can cause data inconsistencies. See Potential for Inconsistent Transaction Completion (Data Loss) in Some Failure Conditions.

## Known Limitations Wh en Using Oracle 9i RAC with WebLogic Server

The following sections describe known issues and limitations when using XA and WebLogic Server with Oracle RAC.

- Potential for Inconsistent Transaction Completion (Data Loss) in Some Failure Conditions
- Potential for Data Deadlocks in Some Failure Scenarios
- Potential for Transactions Completed Out of Sequence

Note: Some of these limitations are also described in Oracle's bug numbers 3428146 and 395790. Contact Oracle for more information about these issues.

### Potential for Inconsistent Transaction Completion (Data Loss) in Some Failure Conditions

In some failure conditions, when multi data sources are not being used, transaction processing (data changes) that occurred on a RAC instance other than the instance on which a transaction was initiated will be lost without any notification or exception.

For example, consider the following WebLogic Server configuration:

- A WebLogic cluster containing two servers: `server1` and `server2`.
- A JDBC data source `ds1` targeted to the cluster. (Identical instances of the data source are present on all instances in the cluster.)
- A JDBC data source `ds1` is configured to connect to an Oracle RAC cluster with connect-time failover enabled. `RAC1` is set as the primary RAC instance; `RAC2` is set as the secondary RAC instance. Data source `ds1` is targeted to the cluster. (Identical instances of the data source are present on all nodes in the WebLogic cluster.)

In the following scenario, some data changes will be lost:

1. Network connectivity between `server2` and `RAC1` is lost, which causes database connections in `cp1` on `server2` to fail over to `RAC2`. The same data source on `server1` still has connections to `RAC1`.
2. On `server1`, an application starts a transaction and uses a database connection from `cp1` (a connection to `RAC1`) to make data changes.
3. The application invokes an EJB on `server2`, which uses a database connection from `cp1` on `server2` (a connection to `RAC2`) to make data changes.
4. The application completes the transaction on `server1`.

Result: Data changes on RAC1 are committed. Data changes on RAC 2 are ignored. The WebLogic Server transaction manager calls prepare and commit on the resource. In this case, because the data sources have the same name, they are considered to be the same resource, so the calls are made on only one instance of the data source. Because the data sources contain connections to different RAC instances, the data changes are committed on one RAC instance, but the changes on the other RAC instance are lost.

Workaround: Provide redundant network hardware between the WebLogic Server instance and the Oracle RAC instance to avoid the network failure.

### Potential for Data Deadlocks in Some Failure Scenarios

There is a window of time in which transaction IDs are not available across the RAC cluster. Because of this known Oracle limitation, after some failure conditions, some incomplete transactions cannot be properly completed, which can result in deadlocks in the database. To prevent these failure conditions from arising, WebLogic Server provides two configuration attributes that enable XA call retry for Oracle RAC: `XARetryDurationSeconds` and `XARetryIntervalSeconds`. For more information about these configuration options, see Delays During Failover.

### Potential for Transactions Completed Out of Sequence

When using the Oracle DataBase Control, the order of transaction processing is not guaranteed. For example, if you implement a web service that uses DataBase Control do the following transaction sequence:

1. Create a table
2. Insert record 1
3. Insert record 2
4. Insert record 3
5. Select records

If the primary node goes down momentarily after the table is created, it is possible that transactions submitted to the database are performed out of sequence.

## Known Issue Occurring After Database Server Crash

If, while a transaction is being processed, the database server instance crashes after the `PREPARE` operation is complete but before the results of that operation have been written to the transaction log, a `COMMIT` call from a client for that transaction may hang for several minutes and possibly until the TCP timeout period has expired. The window of time in which this might occur is small and the problem occurs rarely. There is no workaround for the issue at this time.

# JDBC Store Recovery with Oracle RAC

If you are using a JDBC Store with Oracle RAC, there are features and limitations to consider that concern Oracle RAC node failover. See the following sections:

- Configuring a JDBC Store for Use with Oracle RAC
- Automatic Retry

## Configuring a JDBC Store for Use with Oracle RAC

The way that a JDBC Store works limits the options you have for configuring one for use with Oracle RAC. You cannot configure a JDBC store to use a JDBC data source that is configured to support global transactions. The JDBC store must use a JDBC data source that uses a non-XA JDBC driver.

A JDBC Store holds on to a connection until that connection fails, at which point it moves on to the next connection and repeats the process. Therefore you cannot implement load balancing with a JDBC Store, including using a load balancing multi data source. You should configure a multi data source for a JDBC store to use the Failover algorithm. For more information about this configuration option, see Using Multi Data Sources without Global Transactions.

## Automatic Retry

JMS has a limited connection retry mechanism which enables it to silently react to the failure of the RAC node that hosts its database connection. If the database has experienced either a minor network 'hiccup' or a RAC database has failed over to another node, the second connection attempt (the retry) will succeed to the next RAC node.

The time within which this retry is attempted and the number of retries attempted are limited to minimize the negative effects that an extended connection retry time could cause. If the database connection remains unavailable for a long period of time, the delay can impede the ability of JMS to properly continue its processing (for example, to maintain proper message ordering). Also, the transaction manager could declare the JMS resource of a transaction to be dead if there is not enough processing progress made within this time period, or out-of memory conditions could arise. There are system-level tuning guidelines that can help minimize the RAC failover time frame which is critical to the success of the automatic retry.

The tight loop on the automatic retry is particularly important when JMS processing occurs with transactions. If an I/O failure occurs in the JDBC Store, the store record is in an unknown state which will put the message itself in an unknown state. To prevent the message from being committed in this unknown state, JMS will mark the transaction associated with the message as a "failedTransaction." Any future attempts by the transaction manager to finishing committing the message will cause JMS to throw a javax.transaction.xa.XAException with an errorCode set to `XAException.XAER_RMERR`. This exception is an indication to the transaction manager that a transient error has occurred in the resource manager (JMS) and that the transaction manager should retry commit processing. The retry logic provides a second attempt to establish the connection before JMS communicates any failure to the upper layer which would translate into an RMERR. If the RMERR is generated, then the only way to recover the message and complete the transaction is to restart WebLogic Server.

The automatic retry logic is currently governed by an option on WebLogic Server as follows:

>     -Dweblogic.store.jdbc.IORetryDelaySeconds=X

Where `X` is the number of seconds that should elapse before the connection to the database is retried. The default is one second. This value is restricted to the range `0` to `15`, and the retry will only be attempted once. If a failure occurs on the second attempt, an exception will be propagated up the call stack and a manual restart will be required to recover the messages associated with the failed transaction.

Note: In the event that an automatic retry attempt is not successful, you must restart WebLogic Server.