How should we approach the problem of excessive data collection?

Data & Analytics Business Intelligence Data Warehouse +3 more

334 views1 Upvote9 Comments

CEO in Software, 11 - 50 employees

I started the idea of multi-tenancy for IoT, realistically multi-tenancy for data back in 2016. The basic idea is, we need to find the right way to get the maximum value out of the infrastructure we're building, and thereby not create even more sets of data about the same stuff. I have no idea if this is even possible, but I've used a similar model for infrastructure design and build in the past. What if you could work with manufacturers from an application standpoint to define data value prioritization and retention models that applied to specific operational environments like shop floor or manufacturing machines, to where you could apply a policy that could be defined for you. While it sounds great, the reason I think it would never work is that there's never been a time where somebody has said, "Well, can you be 100% certain that I'll never want to go back and look at that data?

3 Replies

CTO, 11 - 50 employees

Yeah. I agree with you. With regard to the multi-tenancy thing, Mark, you are brilliant and you foresaw a real problem.

CEO in Services (non-Government), Self-employed

There is a lot of information to digest in that, but I love the idea of the multi tenancy because it involves, in a commercial operation, supply chain, inventory control, GPS and the location of products in a line, all of the quality control that goes with it, all of this acquired metrics.

CEO in Software, 11 - 50 employees

Simon, you are way too kind to me. You should run your compliments by my wife before you post. She'll set you straight. :)

CTO, 11 - 50 employees

Ultimately, if you think that saving data is a commodity, let somebody do it. As long as you focus on the procedures for deriving intelligence, you’re good. So you hand it off to somebody else to store it.

1 2 Replies

CEO in Services (non-Government), Self-employed

Now that you have voice, you have chat, you have all of these capabilities in taking Edge devices very close to a location. Keep what you need to keep for historical value on a server in a cloud somewhere, wherever, but give me my instant decision.

CEO in Services (non-Government), Self-employed

Separate compute and store the #Seagate model...

CIO / Managing Partner in Manufacturing, 2 - 10 employees

How do you define excessive? I would start with a different question, what is the business value of the data and how is it going to be used? By starting there you can determine the purpose and value, and maybe that data is no longer excessive?

IT Director and Software Producer in Software, 11 - 50 employees

It’s about sifting through the ever-expanding mountain of data and reading out meaningful content.

More is NOT better. More is just more.

One of my preferred techniques is combining multiple pieces of data into single indexed results — i.e. making more into less and building digestible meaning.