by Sachin Rane, on Jan 10, 2020 5:19:09 PM
Estimated reading time: 2 mins
Big Data is presenting challenges as well as opportunities. Data lakes are forward-looking data storage solutions to the traditional dilemma of storing and using high volume, multi-dimensional data and information for enterprise-wide collaboration.
Enterprises usually invest in data lake solutions when they have operational complexity, high operational costs, and multi-protocol analytics. If not, then you are good to go with a traditional data warehouse or a data mart.
Benefits offered by a data lake
A data lake solution helps you build a massive collaborative space around multi-structured data. It thereby offers multiple business benefits:
- Data ingestion: Captures the data in its “as-is” form, be it structured, semi-structured, or un-structured, and does not need to create a schema prior to data capture.
- Data democratization: Allows users to analyze the data as and when required without having to overly rely on the data science team.
- Data discovery: Enables storage of years of data, which is usually discarded on use, and allows exploring, testing, refining as well as extrapolating of data points by engaging predictive and prescriptive analytics.
- Data analytics: Allows exploration of old and new data sources and zero down on key variables that hint at better business performance.
- Collaboration: Breaks down the silos and offers a mammoth distributed architecture, where data resides and can be used whenever it is required.
- Pattern identification: Allows to explore data relationships and identify new patterns in the data points harbored within the colossal data set thus unlocking value in near real-time.
Best practices for data lake implementation
4 best practices to build a Data Lake:
- Culture shift
- Metadata
- Training
- Gate keeping
Enterprise-wide collaboration is the basis of building a data lake . The following key points enable you to establish a strong foundation towards a functional data lake:
- Culture shift: Inculcate organizational discipline and employ a conscious culture shift towards self-service analytics, insights-driven thinking and building enterprise collaboration. In absence of which a data lake is just a “white elephant”.
- Metadata: Build your data lake on a foundation of metadata if you want to turn your data into action. Otherwise, you are at the risk of turning the data lake in to a data swamp, which is of no use.
- Training: Invest in data skills and trainings to develop the right skill-set and organizational culture. You have to lead by example. Failing which you are risking the return to the era of dependency on programmers.
- Gate keeping: Institute agile governance. You can delete the data sets which seem to be not utilized for more than two years.
In summary
You can manage without a data lake if you are still working in a low complexity business environment and can do with two-dimensional analytics. With the global market orienting itself towards increasingly complex transactions and enterprises generating multi-structured data, you need to expand your scope of data management and analytics. Having said that, the enterprise data management can be a winning ball game only if you evolve the data lake deployment around an insights-driven culture, metadata, training investments, and agile governance. Otherwise, you risk the mistake of creating a massive data swamp.
