Last week, we hosted a webinar with IDC analyst Carl Olofson that examined the results of an InterSystems survey of IT professionals highlighting the cost of slow data transfers on businesses. The study found that an overwhelming percentage of organizations (75 percent) believe untimely data has inhibited business opportunities (check out our blog post recapping the results).
The Challenge
According to the study, conducted by IDC, the issue of untimely data begins with organizations using extract, transform and load (ETL) and Changed Data Capture (CDC). These technologies slow businesses’ ability to conduct real-time data analysis, hindering efficiency as a result. In fact, 27 percent of survey respondents indicated that slow data has negatively affected productivity and agility, and 25 percent report it negatively impacts their ability to conduct analysis.
Taking a closer look at where ETL and CDC struggle, the study exposed that nearly two-thirds of data moved via ETL was at least five days old by the time it reached an analytics database. With CDC, the survey found that on average it takes 10 minutes or more to move 65 percent of CDC data into an analytics database. For organizations that make decisions based on live data, this is problematic as it requires the ability to blend analytical queries into transactions in real-time.
The Solution
Many organizations today face the repercussions of having unique, independent data architectures for transaction processing, for their analytics engine and integration. In fact, most survey respondents have more than five production transactional databases, and over 60 percent have more than five analytical databases. While this was considered a best practice not too long ago, it has created unnecessary complexity and increased total cost of ownership.
This research also shows that the demand to consolidate database operations by combining transactional and analytic capabilities into a single DBMS solution, with blended features, is higher than ever. In fact, there is an even higher need to consolidate data and application integration into a single data platform. The study itself revealed 37 percent of respondents are evaluating new database technologies, and more than 90 percent think blended features (transaction processing, analytics and integration) are very valuable. The benefits of a unified data platform go well beyond the reduced cost of the technology itself. Fewer integrated parts lends itself to easier maintenance, development and interoperability with other systems.
The bottom line is this: eliminating latency between transaction processing and identifying insights with analytics requires a modern data architecture. We need to rethink how to develop applications that can support conditional transaction processing – meaning before committing to a decision, we must use analytics to guide the next best action in real-time, initiating a workflow that interoperates between legacy, SaaS applications and new business processes.
To download a recording of “The Data Dilemma: The Challenge of Managing Data in Enterprise IT” webinar, click here.
