What does best practice architecture look like for your Azure Data Lake in your organisation? How have you implemented RBAC/ACL to ensure data is made available only to those required to consume?
258 views1 Comment
Sort by:
Content you might like
I have complete knowledge of what data my company collects, who has access to it, and the reasons we collect and store it.
Strongly agree9%
Agree52%
Neutral16%
Disagree20%
Strongly disagree1%
Other (please specify)
My ESG team uses a dedicated application to consolidate indicators for external reporting. This system manages a workflow where a designated owner submits data and a validator approves it. However, the process is heavily manual and lacks system integration. With upcoming regulations like CVM 193 in Brazil, the pressure for accurate, auditable, and timely ESG data is intensifying, and our current tool is being questioned. Question: What strategies or platforms have you implemented to automate the extraction of ESG data from various source systems (EHS, HR, ERP, etc.) into a centralized reporting tool?
I work for a company with geographically dispersed operations, and we are facing significant challenges with our data collection and reporting processes for HSE (Health, Safety, and Environment). We collect a variety of environmental data (e.g., water quality indexes, equipment training compliance) from multiple sites. Our current data collection application has critical governance gaps. When an indicator shows a deviation, we struggle to trace its origin because the system lacks a clear audit trail. We cannot confidently verify who entered the data or when, which undermines the reliability of our HSE indicators. Furthermore, our data collection forms are rigid. Any request to add or modify fields requires our development team, leading to significant delays and hindering our ability to adapt to new requirements quickly. Questions: 1. How do you ensure robust data governance for HSE indicators? What tools or best practices have you found effective? 2. Have you successfully empowered your business or HSE teams to create and modify data collection forms without developer dependency? If so, what low-code/no-code platforms or solutions do you recommend?
What do you see as the single greatest threat to ethical data use in your industry?
Short-sighted focus on profit18%
Lack of holistic view of data pipelines42%
Regulations not keeping up19%
Unclear ownership of responsibility for ethical data use16%
Limited view of potential impact of unethical data use5%
Something else? Comment below
I'm working on something like this right now. What we recommend often is the kind of gold-silver-bronze layer approach of the data lake. Bronze is your raw data, gold is the end consumer and silver is some kind of hybrid of the two. And yes, we have designed our backend RBAC/ACL's for each of those layers. So within each layer, the data organized in a taxonomy that is tagged to particular security access for individual user groups. The gold layer has broader accessibility because it's been fully managed and secured. Silver and bronze have a more specific user group because they need more training and more sophistication in that using the tools. But again, it's organized by subject area or data domain. And this for this particular client has to be split between regulatory and non-regulatory - and certain subsets of that data set more tightly controlled. So in that RBAC model, we've got this kind of split based on data lake area, domain and data classification.