I have a calculation and data storage heavy Excel file. About 90k rows of raw data, about 2500 calculations (some of which are dependent on other formulas), about 20 presentation worksheets (with a lot of VLOOKUP, INDEX/MATCH across the whole column )
I have simplified some of the data and now pull from a database rather than Excel files.
I can create the raw data via fuzzy database join (join using LIKE) but poor database performance (due organizational cheapness) makes this presently unviable (the concept works)
What I a more interested in is a way to simply the mass of formulas and calculate at database level. But I don't want to have to write some 2500 complex stored procedures
Power BI and Tableau look promising but the number of calculations and dataset size makes this somewhat unworkable. And the result is more reporting than visualization.
This feels like is should be solvable using a rules engine but the various rules engines I've looked at don't really cover this sort of area.
The challenge is that making changes to formulas necessitates a recalculate which takes time so changing and development is a slow process
Ideas?