Every now and then we see a request for the OLAP Council benchmark, last updated in 1998; since nobody else seems to have all the materials in one place, we have decided to put them here as a service.
- The specification can be downloaded here.
- The data generation program is here.
While it may be somewhat dated now, this is still the only such benchmark there has been. Our friends Erik Thomson and George Spofford of DSS Labs produced it for the OLAP Council, which was an industry association of many of the leading OLAP companies. Thomsen and Spofford, who worked with us for a while, have maintained a unique focus on theory in an industry not noted for its enthusiastic support for theory.
The diversity of OLAP products ensured that results of benchmark runs could be — and were — conditioned and spun in various ways by the sponsoring companies. Nonetheless, the benchmark continues to represent a set of basic tasks which OLAP products can be expected to perform, and which can be measured and timed.
The OLAP Council is no longer, and with it, the function of certifying benchmark auditors, so this is hardly a current project. We are making available on our site the benchmark specification, and also a little program that generates data files to run the specification.
August 1st, 2006
Let me start this post by saying that Vector Space has no involvement or business interest in DQS, a startup with a product in beta. But when we heard about it from a former colleague, we thought it was worth a deeper look. Perhaps you’ll find it interesting too.
DQS proposes a new way of bringing together the information resources of large enterprises.
To understand why we’re interested in DQS, we must peek at the dark side of IT. Consider the following common scenario:
Our consultants go to a meeting with a major client.
“I need to analyze corporate operations,” says the customer. “Sales, cost of sales, profitability by channel, that sort of thing, month by month.
“Our company makes 123 different products, it sells them in 54 regions, with 37 sales channels, and produces them at 19 plants. Although it’s a lot of information, we’re already capturing it all in our systems. All I need you to do is pull it together and put it on my computer screen so I can work with the big picture. Can you do it by the end of the month?”
One might imagine every serious enterprise already has this information. After all, how does anyone run a company without knowing what they make on each product? Changes in sales for their channels? Seasonal and regional variations in demand?”
One might also imagine that such a facility is easily created.
But imagination would be wrong, on both counts.
Companies are run without this vital information every day.
And applications that draw information from all over the enterprise pose daunting challenges, because of the way IT systems are typically structured.
The information archipelago
Every large company has many systems. Just a little digging usually shows that corporate information resources reside in a patchwork of purpose-built systems.
- There are systems for different business functions: a sales system, a production system, another for inventory;
- Local offices create their own systems to meet their special needs, especially where the company operates in several countries;
- Where there have been mergers and acquisitions – which is to say at most large companies — legacy systems of the merger parties continue for years, resulting in further division even within the same function. So it is not unusual to find multiple sales systems, multiple production systems within one company.
To make things worse, these systems run on different hardware using different operating systems and different database technologies. Even where they run on software from the same vendor – SAP, for example — implementation can be different and incompatible. And in many cases, similar concepts – product, for example – can be represented by different codes, or tracked at different levels of detail.
Each of the incompatible legacy systems embodies years of staff effort, and represents a complex study in its own right, leading to major staffing headaches. Someone working on one system can only move to another after a significant period of retraining and apprenticeship.
Integration can be shockingly difficult.
Bridge-building vs. consolidation
For analysis – and for every other enterprise function — communication among systems is a constant issue. Some IT departments spend a majority of their resources building bridges connecting their data archipelago, especially in the wake of a merger.
Analytics consultants like ourselves spend no end of time (and customer money) working on what we call ETL – Extraction, Transformation and Loading – in other words, systems reconciliation.
Seeking a detour around the data swamp, many companies initiate a new master system, designed to replace the many existing systems. But since the old systems are needed for ongoing business, a new system is – at least in the short run – simply one more system for the stew.
Mega-system risks
Building a new comprehensive system is also very risky. Big-ticket items costing in the many millions of dollars, such facilities typically require a large dedicated staff and multiple years to complete. To achieve success, they require impeccable design and execution, and a very stable development team.
Even under the best of circumstances, enterprise systems projects only begin to display measurable results towards the end. But they can only succeed with a reliable, unfailing, continuous commitment by top management and investors over long months and years. Impressive persuasive powers are required to maintain such a commitment years in advance of results.
A new system of this size is critically vulnerable to management changes to further mergers and acquisitions and to the financial fortunes of the organization. And the costs of such projects can depress the bottom line dramatically.
The DQS idea
The founders of DQS have a different idea. A great deal of useful business logic and experience is embodied in the well-tested and highly-debugged legacy systems of every company. The problem is not the systems, which typically work very well, but their mutual incompatibility.
What if, instead of building bridges system by system, we designed a method for systems to communicate in general? After all, the incompatibilities between systems are not endless – they can be enumerated and solved in a general way.
What if each computer in the enterprise could be retrofitted with a program whose only purpose is to communicate with the others, using a common communication protocol? And what if these communication links could be made extremely easy to access, so that the information resources of the enterprise appear to reside on one virtual system even though they really live on multiple systems?
John Walsh, Chairman and Chief Strategy Officer wrote to me in a recent email:
We are in beta now with our enterprise version of what we refer to as an Optimized Service Oriented Architecture. We have spent the last four and a half years patenting and developing the technology. We were granted a special patent reserved for technologies that are “in the national interest”. We were told by the patent office that only 10 of these patents have been awarded in past 40 years. We have several other patents pending.
The technology enables you to quickly create an inventory of re-usable objects – data structures, software, and processes – that can be used in designing, testing and implementing applications and processes. The technology is language and operating environment agnostic. We do not rely on XML or Web Services standards.
We have a run time environment that creates a linked network out of disparate computers.
The interface allows users to collaborate and share distributed data, software, processes and computers without regard to geography.
Our goal is to “industrialize” Information Technology. Our models are the manufacturing and construction industries in which products are designed and then assembled from components built by hundreds or thousands of different vendors located all around the world.
This is the premise of DQS. It looks promising to us, and we’re curious to see how it plays out. We’ll bring more to you as we learn more ourselves – stay tuned.
June 27th, 2006
