What end user tools/platforms are you giving to employees to analyze millions, possibly billions of rows of data? Excel is not strong enough to ingest and do analyzation of this amount of data.
Sort by:
As far as end User tools than can run only in the users workstation probably none besides of power BI for end users, but for this amount of data you are looking more at a Big Data analysis toll, in our company we use Apache Hive, is a data warehouse infrastructure built on top of Hadoop. It provides a SQL-like query language called HiveQL. The choice of the most suitable option depends on factors such as specific requirements, budget, existing infrastructure, and expertise within the organization (end users skills). It's important to assess the capabilities, scalability to determine the best fit for your needs
I agree with Lance, it depends on their skill set. Generally, I've given employees Qlik or Power BI accounts for end user data anyalysis (i.e. employees who are not programmers). For the more technical staff who know how to code, I've given them python, and PyCharm is a nice tool for using python.
We've had this sort of challenge for years in lifescience and it's always a bit of a conundrum due to the skillset of employees. At the high end of data science and engineering, who are often as qualified if not more so than IT on data systems, they will get SQL/database access and often special platforms to do such analysis. We've even carved out dedicated compute and cloud environments for them, to minimize blast radius, but also to empower them to conduct their work. In the lowest tier of data expertise we create usual dashboards with IT analysts to provide the custom work for them. The middle ground is the hardest, folks often want to "just use excel" and while technically capable, they usually don't have the statistical background to make the correct data decisions. But we still allow them excel and other stats tools (R, SAS, prism, etc) however we will often connect them with the data science team(s) to make sure they're doing the analysis correctly.
I would recommend taking a look to the product Rows: https://rows.com/