IBM Data Connect service for Bluemix

In the Command Data Shaping options area of the text description, is it possible to use a tabular format with a layout like: operation | description….

IBM Data Connect provides a lightweight, cloud-based set of data refinement and access services. In this section, you learn about:

  • Capabilities of IBM Data Connect
  • Common tasks that you can do with IBM Data Connect
  • Enterprise connectivity with Secure Gateway

IBM Data Connect capabilities

The IBM Data Connect service in Bluemix allows you to identify relevant data, transform the data to suit your needs, and load it to a system for use. In IBM Data Connect, you begin by finding the data that you want to work with from data sources like IBM Cloudant NoSQL DB and IBM dashDB. You use metrics to better understand your data quality and identify areas to improve.

To improve the data quality, you work with a sample of the data and apply shaping actions such as sorting, filtering, and joining. You can apply the actions to the full data set and load the data to destinations such as Cloudant NoSQL DB. Actions are defined and processed through Activities. After an Activity is created, it can be used to set up recurring shaping and copy actions and the Activity control panel includes a display of the current run status.

Common data shaping operations

After a data set is connected to, Data Connect operations can be applied at the table level or the column level. Several shaping operations are supported. Here are some of the supported operations:

Blank out – Converts the string in the column to null.

Ceiling of number – Returns a number’s nearest integer of greater value. Example: The ceiling of 2.31 is 3. The ceiling of -2.31 is -2.

Change the timestamp value – Changes the timestamp in a column by a specified number of years, months, days, hours, minutes, seconds, and milliseconds. Each part of the timestamp can be independently advanced or set back. You can modify the timestamps in the selected column by up to 101 years earlier or later.

Collapse white spaces – Collapses multiple white spaces to a single white space. White space characters are spaces, tabs, line feeds, carriage returns, and form feeds.

Concatenate string – Appends a user-specified string to the value in a selected column.

Connecting to on-premises data

The IBM Data Connect service works with the Secure Gateway service to establish connectivity to enterprise on-premises data sources and targets. You need to create an instance of the Secure Gateway service in your organization and then install the Secure Gateway client on an on-premises computer that has free network access to all your on-premises data sources.

 

Related links

Getting started with Data Connect