What is Data Management?

Data management refers to the set of practices, techniques, and tools for managing the storage and access to enterprise data assets while ensuring security and governance. While this definition is highly general, and data management sub-domains are full topics in themselves, it does encircle the fact that data management as a practice has grown exceptionally complex alongside those business use-cases that are incorporating ever more greater volumes and varieties of data. Due to this phenomenon, for many enterprises, it simply is not possible to conduct operations without a highly tuned data management system to collect, track, organize, and deliver the information that is critical to business processes.

Understanding Data Management

Enterprise data management practices and tools aim at streamlining data operations while applying a layer of business logic, typically to address policy driven domains like data governance, and security, all ultimately designed to provide the freshest and most relevant business intelligence and usefulness from the company’s data assets made appropriate for every data user. Data usefulness and insight for the end user, inside and outside the organization, is the ultimate purpose for data management.

The general challenge is as organizations grow, and their data needs increase, data infrastructure often becomes patched together from many data sources and vendor technologies. Typically this is unavoidable because of the many data domains that a company needs to track (e.g. accounting, CRM, ERP).

These patchworks introduce complications such as data siloing which can wall off data and lead to lack of data insights, or difficulty in maintaining performance while complying with data requirements. Moreover, trying to reduce the complexity by replacing these patchworked data infrastructures with singular central systems can be difficult and risky. Least of which is the threat of downtime or maintaining data integrity and accuracy throughout a migration. A better alternative is to unify the complexity with integration tools that employ automation or machine learning, such as using a Connected Data Platform, and which establish a single source of real-time data, or “a single source of truth” that is reliable, relevant, and fresh.

The discipline of data management can be divided into two levels, a technical layer, and a non-technical layer. On one level, raw data is collected, transferred, processed, analyzed, and stored. But more and more the second layer, the non-technical level, is prioritized as the location where business users come into contact with the data efforts of the technical layer. At this level, business personnel using higher level dashboards can quickly glean insight into their everyday workflows.

Data management contains multiple subdomains, each specialized and complex. These subdomains broadly represent the priority disciplines in modern data management today.

• Data Governance: Addresses both the macro level, political nature of data and how data should be governed, and the micro level, corporate data responsibility to ensure political requirements are enforced within the company.
• Data Architecture: Concerns the models and policies that govern data collection and storage.
• Data Modeling and Design: Modeling and design bridges the abstraction of data models to physical databases. It must be ensured that structures are representative of what they are modeled for, and that structural configurations do not curtail further data developments as systems grow.
• Database and Storage Management: An operational domain, DB and storage management lays out processes for actively managing and maintaining database infrastructure.
• Data Security: Data security concerns the protection of data assets against threats that can change, destroy, or steal data. This differs from data protection which aims at preventing loss or corruption to data as it is handled.
• Reference and Master Data Management: Reference and Master Data Management (MDM) is essential for multi-source, multi-system, multi-location organizations to help them coordinate across systems and business lines. MDM increases the contextual depth for operational and transactional data.
• Data Integration and Interoperability: Integration and interoperability concerns how data is transformed from one system to another; integration refers to the use of middleware to translate data from one system to another, interoperability addresses real-time data exchange without middleware.
• Document and Content Management: Document and content management encompasses the process of cataloging and storing business documents.
• Data Warehousing and Business Intelligence: Warehousing and intelligence is a prominent aspect of data management, wherein data is mined, analyzed for insights, and then stored in data warehouses, data marts, and other systems for access by data scientists and end data users.
• Metadata Management: Metadata is data about content data. Like catalogs, taxonomies, metadata management aims at summarizing and potentially drawing high level conclusions of content data.
• Data Quality: Data quality management is concerned with presenting the “most fit data for its intended use”. Though “quality” is subjective, and since data is digital, greater accuracy and precision is achieved by ensuring data is complete, accurate, consistent, valid, unique, and properly integrated. Ultimately, data suitability must be judged beyond these dimensions by users of the data.

Benefits of Data Management

Adhering to sound data management practices, businesses have the ability to understand the entirety of their data. Achieving full data control grants many benefits, but the overarching benefit of data management is to improve customer experience through the mastery and operational control of a company’s data assets. For each organizational case this entails a specific and unique setup, and however that is achieved, the measure of a properly executed data management apparatus bestows the following benefits.

• Improved Data Quality and Accuracy: Effective data management can improve data quality and accuracy by mapping the entire data landscape. With checks and policies, automated systems can ensure that data rules are followed and data remains in its most accurate and useful forms.
• Eliminated Data Redundancy (Data Silos): Data silos are a clear example of potential negative data redundancy—where the same data can exist in multiple storage locations. This can cause weakness in company data management, and open opportunities for data to become inconsistent, unreliable and meaningless. And while system redundancy is a useful technique used in preserving data, or scaling operations, data management aims at reducing redundant data and the operational.
• Reduced Time and Cost: Data management requires upfront costs to implement. This may deter some companies and push them into finding another cheaper workaround for their data problems, perhaps with existing software within the company techstack. This could be a fruitful endeavor, however, long term success relies on anticipating future bottlenecks that consume resources. For example, implementing lesser solutions when leadership is intent on company growth can set teams up for multiple challenges, like 1) struggling to get useful insight out of data systems, 2) mounting pressure on weak apps demanding teams to migrate to better solutions, and 3) disorganized data operations that build blind spots within the company knowledge bank guiding companies to make decisions with stale, absent, or wrong information. A tuned company wide data management system, while requiring upfront resources to set up, helps to organize data so that teams can be informed and anticipate and plan reducing time and costs.
• Guaranteed Data Compliance: Data compliance and the protection of personal information is a business reality today that are accompanied by stiff penalties for their violations. Over the backdrop of data explosion, compliance becomes cumbersome, difficult, and costly. The powerful tools in data management leverage automation to overcome the challenge of compliance.
• Improved Security: Data management supports improved security of company information and the vital personal information they are responsible to protect. Security can be built into a data management platform, however, data security platforms (DSP) that can be integrated may offer more advanced data encryption, intrusion detection, visibility and monitoring, and other tools that make resolving errors and breaches more easily.
• Informed Business Decisions: Data management solves the age old desire of business leaders to understand their business entirely with measurable data that is honest and accurate and obtain a true picture of performance. Data management tools do this by first bringing together an organization’s entire data ecosystem, using disparate sets of data to understand greater context, and then synthesizing actionable insights.
• Single Source of Truth Reliability: Data management supports the single-source of truth (SSOT) approach to maintaining a unified system. An SSOT is the aggregation of all data sets, or essentially the de-siloing of data in a company, usually using integration technology, like master data management (MDM) systems. Decision makers can then take a holistic view of their performance and make data-driven decisions.

Types of Data Management

Data management is a broad topic, as noted above by the existence of several subdomains. Because there are many variations, it’s important for companies to understand their end data needs, as well as to understand their organizations data lifecycle, or the flow of data through different stages affected by established policies to manage their data. More specifically, the data pipeline is the path that data will take “physically” through that lifecycle, e.g. a data source like a digital ad platform can be connected in series to a marketing platform that finds insights from ad data.

Below are several types of data management components that traditionally appear in company data management strategies.

Data Processing

Data processing is the “ingestion” stage of data. During data processing data scientists will gather data, typically electronically, but also mechanically or manually depending on the data sources. For legacy companies with their records stored on paper in filing cabinets, data processing may mark the beginning of their digital transformation.

Data processing is an important and critical step to safeguard against ingesting erroneous or inaccurate data. Clearly, because these results form the foundation of any insights discovered later, errors in this foundation will most assuredly lead to poor decision making. The fix is to pay attention to technical requirements as well as the big picture of how data will be made to contribute to company success.

Data Modeling

Data modeling is often defined in two ways: it is the creation of representative models that accurately depict data structures, a software engineering function; and, the process of creating visual representations of the data in those structures, useful for end users to quickly digest information.

Conceptual data models may look like this, possibly representing sources, process, and attributes:

Sometimes data modeling is interchanged with data visualizations, however their purposes are different. Whereas a data model abstracts and represents the structure of data in the system, the following infographics developed by BBC represent data modeled or visualized for end users.

Data Warehouses

Data warehouses are massive storage options for enterprises, and essentially the technology that allows data to be mined and analyzed. They can be off-site, or on-premise, or popularly in the cloud. Its main purpose is to store massive amounts of data to be analyzed, cataloged, and then presented to users. Not to be confused with data lakes or data marts, data warehouses store structured data, while data lakes store unstructured raw data. Data warehouses typically store a refined version of data pulled from the much larger data lake, as if it were dipping a ladle in to pull out relevant data and then analyze it. A data mart is much more like a data warehouse in that it presents a very refined version of data, typically for use by non-technical people. For example, a single data mart may only serve the sales department, and therefore only contain relevant sales information for producing reports.

Data Architecture

As it sounds data architecture is concerned with the infrastructure of data management systems, and data architects are responsible for translating business requirements and devising a technical solution. Generally speaking, there are three stages data architects move through, 1) the conceptualization of a solution that represents all of the businesses entities, 2) the extension of logic to map the concept and how entities are related, and 3) the establishing of physical infrastructures and data mechanics that realize the solution.

A three stage model for data architecture belies the complexity of many data management systems. In architecting solutions, technical and physical concerns are important but merely manifestations of the larger more complex conceptualization of factors that influence successfully meeting business needs. For this reason, data architects are at a senior level, and have the real-world experience to anticipate and plan for contingencies that may not be understandable at lower levels.

Data Security

Data security concerns the protection of data assets against threats that can change, destroy, or steal data. This differs from data protection which aims at preventing loss or corruption to data as it is handled. Data security must be addressed at multiple levels, and multiple points, including but not limited to hardware, software, user devices, data governance policies, admin controls, storage and backups.

More specifically, data must be guarded in these circumstances.

• Where you store your data. Do you store on third-party services or in the cloud, and where are they physically located?
• How you access your data. Think all the ways data enters and exits your system, through webapps, company devices, or potentially worse employee owned devices.
• When you access your data. A statistic that crops up is that 95% of cyberattacks are allowed because of human error. This may be because users are accessing on insecure networks, or have more privileges than are necessary providing opportunity to access sensitive information.
• When data is transferred. As much as the advent of machine-to-machine data transfers is a boon to business efficiency, it is still a potential weak spot. Securing these lines with encryption is standard.

Examples of Data Management

Data management examples abound. This is because data management practices can service very specific domains. The following data management examples illustrate how diverse domains these systems serve.

Product Data Management (PDM)

PDM systems focus on managing product information useful for engineers and designers. They can access this information, like product specs, version control, change orders, bills of materials, vendors/suppliers, schematics, etc., through dashboards. And then share the data with other systems that can leverage product data into other wider operations.

Customer Relationship Management (CRM)

CRMs can be critical to the success of most any business that relies on large numbers of customers with developed relationships. They help sales and marketing teams track personal data, sales leads, sales conversions, revenue data, offers and subscriptions, renewals, etc. Furthermore, they can just as easily track client communications and historical information, essentially tracking the relationships of the company with clients. Combined with say PIM systems, and Enterprise Resource Planning systems, companies can ascertain highly accurate

Product Information Management (PIM)

PIM systems, not to be confused with product data management systems but to complement them, use select portions of product data combined with other systems. For example, by combining with marketing systems PDM data can be channeled to printers, websites, social media, marketplaces, advertising channels, digital marketing channels, or partners. Combining PDMs and PIMs helps manufacturers manage thousands of products.

Master Data Management (MDM)

MDM systems are like umbrella systems, intent on integrating all the data systems under a company’s banner. From the MDM, teams have the tools and workflows to unified multiple data sources, automate their data interactions, and ensure that data integrity is maintained.