Business Intelligence Lowdown

Maintaining Data Consistency

Though firms are giving the highest priority to the quality, consistency, and integrity of data these days, the recovery of data during disaster management manoeuvres still poses a problem. This is because data that is held on standby for emergency purposes is usually not consistent with the data in the production database. The release of Veridata 1.0 promises to ensure consistency and eliminate discrepancies between an organization's standby and production databases. The product comes from the transactional data management (TDM) vendor, GoldenGate Software, and compares source and target data in relational databases to ensure that they are identical. Biz Intelligence Pipeline reports:

Veridata has three components: Veridata server, client agent(s) and the Web/command interface. The server processes comparisons of data served by client agents, which reside on the database servers and connect to the production and standby data sources. Users configure, execute and report on comparisons using a command-line interface or the simple Web interface.

Effective Data Management

Data is the lifeblood of any organization. But to serve a useful purpose, it has to be accurate, consistent and properly managed. A good strategy to manage your data quality will help in optimizing profitability and costs. Effective data management should encompass data profiling, data quality, data integration, data augmentation, and data monitoring. The data should be scanned for errors, discrepancies and duplication, which should then be corrected and rectified. Data from various sources should be integrated and then enhanced. The data thus gathered and merged should be monitored from time to time to prevent inconsistencies and redundancies from creeping in.

Cognos Ties up with Similarity Systems

With data quality being a top priority for business intelligence professionals, the tie-up between Athanor from Similarity Systems and Cognos 8 from BI corporation Cognos seems to make perfect sense. Athanor will provide users methods to monitor data quality and deal with data problems at their source. Data quality metrics from Similarity Systems can be scrutinized using Cognos 8. Business Intelligence Pipeline reports:

Data quality remains a serious challenge for many BI practitioners. Most surveys of business intelligence professionals find poor data to be a top impediment to the generation of meaningful analytics.

BI and Data Quality - Hand in Hand

Data quality technology is silently emerging as a necessary complement to business intelligence (BI) applications. The need for clean data when running BI was well established when SAS Institute bought out data quality vendor DataFlux Corp. to complement its BI products.

The LexisNexis Group, which uses DataFlux's schemes and verification processes to eliminate data overlapping and duplication and to project a single view of its customers from all data sources available, has seen benefits from the combined training materials and documentation of DataFlux and SAS.

Data quality issues have a way of cropping up in a BI system, which means that data quality tools need to be in place to process the data before it goes into the decision engine, says Keith Gile, principal analyst with Mass.-based Forrester Research Incorporated, on Search Data Management. He adds that though the SAS approach __ that everything, including modeling, extract, transform and load (ETL) tools, and analysis, comes under the umbrella of BI __ is not necessarily shared by everyone, for companies looking at a wider data management strategy, data quality is vital.

Data Quality goes from Bad to Worse

In a poll conducted by Business Intelligence Pipeline in October 2004, readers were asked to rate the overall quality of their enterprise data for analytical purposes, as excellent, good, fair or poor. The responses were not too spectacular; around 19 percent awarded an "excellent" rating to their data quality, nearly 12 percent said it was "good", over 50 percent said their data was of "fair" quality, and only 19 percent rated their data quality as "poor".

A similar poll conducted towards the end of 2005 has shown the previous year's results in good light. An astounding 30 percent rated their data quality "poor", which Business Intelligence Pipeline deems, "often worthless for BI purposes". Only 4 percent said their data quality was "excellent". The number of "fair" ratings also slipped to 43 percent. The only consolation was in the "good" category, with 23 percent of readers voting that their data quality was good.

Keeping data safe and maintaining operational security

Data quality has to match up with the exacting standards that governing organizations set, particularly, financial institutions, where collecting and studying loss data is a high priority activity. Loss data is important with respect to Operational Risk and the development of regulatory capital requirements. Even though data loss is a big risk that financial institutions face, it can be controlled. The first step is to educate the staff, be it executive, managerial, IT, etc. Back-up mechanisms, disaster-recovery plans, and, proper training for the end users is essential. Human error is frequently responsible for data loss, which can be categorized as occurring due to not following procedures or lack of education. For a better understanding of Operational Risk, financial institutions should share the loss data and have a common set of metrics and definitions. Operational Risk Management involves data tracking, collection, and reporting so that the key risk indicators are identified with the tracking and monitoring of historical and current data. Businessintelligence reports:

However, the main challenge for the banks in the coming months will be to spot and locate the required data and build up a data storehouse or data warehouse for storing the ready data for analysis. Moreover, banks must ensure effectual integration of different risk types and accurate computation of the various risk measures.

Read More: Measuring Operational Risk: The Data Challenges