Skip navigation.
Downloads | Product Documentation | Support

Concepts Guide

# Introducing BEA AquaLogic Data Services Platform

This chapter provides an overview of BEA AquaLogic Data Services Platform (DSP). It covers the following topics:

- The Extraordinary Cost of Information Fragmentation
- What is Data Services Platform?
- DSP and a Service-Oriented Architecture (SOA)
- Modeling the Enterprise
- Implementing Data Services
- Data Consumers
- Life Cycle of a Data Services Platform Development Project
- Data Service Development Roles

Note: Data Services Platform was initially named Liquid Data. Some artifacts of the original name remain in the product, installation path, and components.

# The Extraordinary Cost of Information Fragmentation

Information resources are often the most important assets an enterprise holds. Information can make or break an enterprise's ability to execute on its essential functions—from day-to-day operations to the marketing and delivery of products or services to strategic planning. The enterprise must be able to access, interpret, and use information as efficiently as possible.

Unfortunately, as enterprises evolve, their data resources tend to become increasingly fragmented. Corporate acquisitions and mergers, new technologies, and changing business strategies can all contribute to a state of data segregation, in which pockets of information become isolated by physical and technical boundaries. The information comes to reside in a variety of places, in databases and files, within applications, or even outside the organization, supplied by business partners, vendors, and clients as Web services. Figure 1-1 illustrates this fragmentation of data.

Figure 1-1 Direct Data Access Applications

As a consequence, the information becomes effectively lost—inaccessible to developers and, in turn, the customers, employees, and decision makers who need it. Developers who do attempt to use the information need to know a daunting variety of data access mechanisms and APIs. The applications are difficult to create and maintain. Their source code tends to be dominated by data management logic, making them intolerant to changes in the data layer.

When changes do occur, the applications must be revised and retested, because their business logic is embedded with data access code. In the worst case, an application must be significantly modified and redeployed as a result of the changes to the data source layer.

When it comes to developing applications, the costs of information fragmentation are quantifiable and significant. By many estimates, up to 70% of the time required to develop an enterprise application goes into data programming.

Other costs may be more difficult to gauge but are certainly as significant—from hampered decision making and poor responsiveness to lost information resources.

# What is Data Services Platform?

Data Services Platform (DSP) is the AquaLogic component for developing and deploying integrated data services. Data services give consumers an easy-to-use, uniform model for accessing heterogeneous, distributed data.

DSP significantly simplifies the task of creating and deploying reusable data services. It has tools and frameworks for rapidly generating data services based on existing data sources and for creating logical data services based on those sources.

From the perspective of the data consumer, DSP presents a sensible, uniform data model for getting and using information. The application simply instantiates a data service and retrieves or updates its data by calling one of the data service's public functions. The data may come from legacy applications, relational databases, Web services, delimited files, XML files, or any source from which a Java application can extract useful data. These data services can be viewed as a layer integrated between the data consumer and the data sources as illustrated in Figure 1-2.

Most data in an organization exists as queryable data or application data. DSP lets you use all data together to compose a logical data model for the organization. The data model represents the business entities relevant to the organization, such as customers, orders, or products.

Figure 1-2 Data Integration Layer Between Data Users and Data Sources

DSP is a virtual data layer in the sense that, except for caching, data is not kept in any DSP or WebLogic component. Instead, data services dynamically retrieve data from the physical data sources. Dynamic access ensures that applications have access to current (that is, real time) information.

A data service represents a unit of information in the data model. Like a Web service, it exposes a public interface in the form of one or more accessor functions. A data service accessor function typically returns data in the form of the data service's datatype. It uses XML and XQuery to acquire, transform, and aggregate data from external sources.

The data is poured into the XML data shape defined as the return type for the data service. Because an individual data service represents a relatively small unit of information, response time from the client's perspective is minimized. Other features, such as caching and query optimization, help to ensure the optimal performance of the data services layer.

DSP insulates data consumers from the complexity of disparate data sources. It encapsulates the details of data integration logic and gives consumers a sensible, coherent model for accessing and updating data. Data access and business or presentation logic are effectively decoupled, resulting in applications that are easier to develop and maintain, without data access plumbing code.

As it does for reading data, DSP gives client applications a unified interface for updating data. DSP allows client applications to modify and update data from heterogeneous, distributed sources as if it were a single entity. The complexity of propagating changes to diverse data sources is handled by DSP.

From the data service implementor's point of view, the task of building a library of update-capable data services is considerably eased by the DSP update framework. For relational sources, DSP can propagate changes to the data source automatically. For other sources or to customize relational updates, you can use the DSP update framework artifacts and APIs to quickly implement customized, update-capable services. Either way, data consumers are isolated from the details of updating the data sources.

# DSP and a Service-Oriented Architecture (SOA)

A Service-Oriented Architecture is an architectural style for an IT environment. In this architecture, business processes are delivered as a set of loosely bound services. Web services provide an implementation of SOA. A web service is a modular, self-describing component that can be used in a platform-independent way.

Traditionally, applications are built as monolithic units. They impose platform dependencies on users and are not easily adaptable to new types of interactions. Web services break these applications into multiple reusable components that can interact with other independent components and services in meaningful ways. For example, instead of having a single financial application, an organization may deploy one or more services that expose the business activities of the application as self-contained, callable processes. That way other systems (such as Human Resource services or applications) can easily leverage financial computing resources. Thus, SOA breaks down barriers between applications. With SOA the IT infrastructure operates in an integrated, organic fashion like a virtual application that spans the organization.

Viewed as a whole, a data service deployment constitutes a data abstraction layer between data users and sources. It normalizes diverse data and exposes a uniform, well-defined interface that data consumers can use to access data. Client developers do not have to know how to connect to a data source, what its native format is, or even where it comes from, whether from one or many underlying sources. The application can simply call a public function of a data service.

SOA is considered a loosely coupled—not completely decoupled—architecture because, while the service provider has no knowledge of the interfaces or business concerns of its consumers, the consumers do include explicit references to the service provider interface, in the form of service calls.

The benefits of loose coupling include simplified data access, reusability, and adaptability. The effect of changes to the data source are localized, requiring only relatively few changes in the data services layer rather than in every application that uses the data source. New services can be exposed and used without requiring extensive changes to existing applications. The result is a data integration layer that is highly adaptive and change tolerant.

DSP extends SOA to the realm of data. With DSP, you can expose information as data services. A data services represents a self contained business entity—such as a customer, product, or order—that is reusable across the organization.

Simply put, DSP disentangles the data provider from the applications that use the data.

Other benefits of DSP include:

- Leverage domain resources. Information "locked away" in legacy systems can be made available to application developers. Applications, and their users, benefit by being able to access and update the entire range of information resources of an enterprise.
- Dramatically reduce coding. With a high-level query language and visual development tools for service implementation, DSP makes it easy to implement a data services layer that lets developers use simple function calls instead of tedious custom code. Data services employ a SQL-like language called the XML Query Language (XQuery).
- Consolidate data management. DSP provides a single point for applying security and caching policies.
- Enable uniform service governance. Service-level agreements can be verified and documented.

# Modeling the Enterprise

DSP enables you to capture and deploy a data model specific for the needs of your organization. A data model typically organizes data in a way that represents business entities. Similarly, in your implementation, you can define the data model that makes sense for your organization. Business entities common to most organizations include, for example, products, orders and customers. A Customer object can have all of the attributes that are relevant to a customer in the deployed environment, such as contact information, shipping preferences, payment information, and so on. The data model can also specify security, caching, and other policies.

Instead of having to contend with multiple sources to acquire the information relevant to a customer, developers can simply use a Customer object from a data service. The policies relevant to the data are applied to the data consistently, without requiring developers to implement them in the various applications they develop.

To help you get from a complex, distributed physical data landscape into a sensible data model, DSP supports a visual, model-driven approach to developing data services. Modeling provides a graphical representation of the data resources in your environment, providing a bird's eye view of a large system, or giving you a way to break a larger problem into smaller pieces.

The result is real-time access to externally persisted data through a logical data model (as illustrated in Figure 1-3) while providing a single point for applying policies, such as security and caching, on information used across applications and services.

Figure 1-3 Logical Data Model

In SOA, services are conventionally thought of in terms of processes. However, most SOA initiatives actually start with data. By its nature, data has broader, more generalized usage than typically does a business process. Data consumers can use the same piece of data in different ways. Once you have developed the data model for your enterprise data, the data types, as encapsulated by data services, can be reused by numerous data consuming applications.

The data model represented by a DSP deployment is captured by metadata. In DSP, metadata serves several purposes:

- Developers can use metadata to determine what data services are available, what functions they provide, where data comes from, and much more.
- Administrators can use metadata for impact analysis, that is, to determine how changes to the underlying data layer affect the data services layer. When changes occur, as they inevitably do, tools for synchronizing source metadata ease the difficulty of applying changes to data services.

The metadata browser, a component of the Data Services Platform Console, provides an easy-to-use interface for searching and browsing through metadata.

# Implementing Data Services

A data service encapsulates the logic for normalizing, transforming, and integrating disparate data. A client uses a data service by calling one of its public functions. A data service can have any number of public functions, each of which returns data in the form of the data shape described by the XML Schema associated with the data service.

The logic encapsulated in a data service is in the form of queries written in the XML Query (XQuery) language. XQuery, based on an emerging standard specification, is a SQL-like language specifically intended for working with structured, XML data. As a declarative language, XQuery lends itself to compiler-based optimization techniques, alleviating data service developers from having to learn extensive rules and techniques for writing optimized XQuery functions.

Those familiar with SQL should find XQuery easy to learn. They have many features in common. DSP provides an XQuery editor that allows you to get started with XQuery by using an easy-to-use, intuitive graphical tool.

# Data Consumers

Once the DSP application has been deployed to a WebLogic Server, clients can use it to access real-time data. DSP supports a number of different types of data clients with these mechanisms:

- Java
- Web services
- Data service control
- JDBC/SQL

Data Services Platform supports a service-oriented approach to data access through Java interfaces (the mediator API and Workshop data service control) or through Web services that act as wrappers for the data services.

As an additional option, the Data Services Platform JDBC driver gives SQL clients (such as reporting and database tools) and JDBC applications a traditional, database-oriented view of the data layer. To users of the JDBC driver, the set of data served by DSP appears as a single virtual database. Note that, given the two-dimensional view of data inherent in SQL, SQL access is supported only for data services that provide a flat view of data.

# Life Cycle of a Data Services Platform Development Project

The following are the basic steps for developing, deploying, and maintaining a Data Services Platform implementation:

1. Establish project requirements. This entails identifying the data resources that are available, business logic requirements, security needs, access information, and caching requirements.
2. Import metadata from available sources. Once you have chosen the data resources you want to include in your implementation, you can define them as data sources by creating data source profiles for them. Importing metadata for the sources automatically creates the physical data services that are the building blocks of your data services layer and stores the connection information for the source.
3. Model the data access layer. A model is a graphical representation of the data services in your environment and the relationships between them. Modeling helps you plan and document your data services implementation. Data services can be created and modified directly from a model diagram.
4. Implement logical data services. Logical data services provide a sensible view of the information resources that are available. They can transform, filter, and aggregate data, as well as prescribe constraints on the data, security, caching and other properties. For details see "Creating Data Services" in the Building Queries and Data Views.
5. Identify update requirements. For relational sources, update capabilities are automatically available through SDO. For other sources and for situations where you need to implement custom transaction rollback logic or other processes, you can use Java to customize the update routines provided with Data Services Platform. Test your update logic and ability to deploy the application.
6. Deploy the data services. For more information, see Deploying Data Services Platform Applications in the Administration Guide.
7. Configure cache and security settings.
8. Create the Data Services Platform client applications.
9. Maintain and monitor the data services layer. DSP includes tools for evaluating the impact of changes to the data source layer and for synchronizing data services with the modifications.

# Data Service Development Roles

DSP facilitates a team-based approach to application development. Instead of developers having to understand how to connect to and use numerous data sources—along with the business or presentation logic that needs to be implemented once the data is acquired—development tasks can be divided in ways that naturally correspond to the roles that often exist in an organization:

- Data Architects know about the desired business entities to be created and the data sources that are required. They have a deep understanding of the data, underlying schema, and relationships across the various data sources. They create the schemas and Data Services and update logic used by the Application Developers.
- Application Developers create client applications that use the services of DSP to access updateable, real-time information.
- System Administrators for a Data Services Platform implementation perform the following types of tasks:

- Deploy Data Services Platform in WebLogic domains.
- Configure access to data sources.
- Set up and configure the Data Services Platform caching.
- Implement security using the WebLogic Server security features, such as configuring users, groups, and defining security roles.
- Monitor Data Services Platform operation, tune performance, and provide support.