"Make or Buy" in BI? The Truth is: "Buy" Provides the Better Solution – Faster and Cheaper.
In Business Intelligence (BI), the question of "Make or Buy" seems to be much more present than in most other software categories for some reason. While self-built solutions are clearly a minority in ERPs, CRMs, Marketing Clouds, and other operational systems, they seem to be the preferred option in BI for a variety of companies - at least until they painfully experience for the first time how their self-built BI project is failing, after spending six or seven figure amounts and months or even years of time, as well as hiring a diverse workforce that is difficult to deploy to other tasks.
Why is that? Why is it, for most companies, not as plausible in the context of BI as it is in the context of ERP and other categories, to buy standardized (and customizable) software that is much faster and cheaper to implement than DIY - and made by real professionals? Why are reporting, analysis and data-driven work requirements classified as so much more individual than those of operational core processes - or in other words: Why are we as a manufacturer of standardized BI software so much less trusted when it comes to building customization options than ERP manufacturers, for who it is taken for granted? Why do so many companies, due to this misjudgment, choose a self-built stack plus visualization tools like Tableau, Power BI or Qlik, and not realize that the supposedly gained freedom of design is simply the completely generic nature of those tools, which therefore provide methodical added value at most, but not any added value in terms of best practice tools and content? Many questions that I cannot really answer, even today. In this article, I would like to discuss the fundamental issue of "Make or Buy" instead, before I devote myself to the related topic of individualization claims next time. I hope to be able to contribute some discussion material to the rather unquestioned "Make" trend in BI.
4 Inconvenient Truths About Self-built BI Setups
To live up to the name of my blog, let me jump right into it: with four inconvenient truths that cause most companies to sooner or later be haunted by their decision to "make" their own BI solution.
1 – It will take years. 90% of the data magic lies below the colorful frontend surface - concept, development and testing of the complete setup (see below) will take at least one to two years in all likelihood.
2 – It will be terribly expensive. Since there are no scaling effects with your own solution, you will easily end up spending more than € 2 million on technology and people over a period of five years (quite possibly much more than that depending on the size of your company).
3 – You carry the full risk. You will have to invest heavily in staff and infrastructure - which means a high capital commitment, even if the project fails.
4 – You will end up with a mediocre solution. Because the technical challenge is immense, self-built BI solutions tend to end up as pure collections of ETL processes - if you're lucky, with Excel and a visualization tool as a frontend. Quite certainly, there will be no easy-to-use tools for the most important user group of any modern BI solution: the operational teams (see also my article No One Uses Your BI System).
Yes, these are hard truths, but unfortunately that does not make them less true. Let me explain why that is.
Why is it so Hard to Build a Good BI Solution Yourself?
The answer to this question is comprised of many aspects - I would like to summarize the six most important one here. In the diagram, you can see where these are located within a classic DIY BI stack (processing layers plus visualization and/or Excel as "frontend".
1 – Users usually do not really know what they need - or they want whatever they can think of. Either way, requirement gathering and, above all, prioritization, is complicated (and will remain so over time).
2 – A true solution frontend with easy-to-use tools for the operational teams is a key requirement of modern BI - but requires a lot of expertise (business and tech) as well as high manpower (long-term!) to implement yourself. De facto, therefore, hardly any self-built BI has such.
3 – The more comprehensive the setup, the greater the challenge of keeping the central data warehouse performant enough to enable queries at all required levels - detailed and aggregated.
4 – Bringing complex data into a consistent, sustainable data model is a key task that requires expertise and ongoing maintenance - especially as data models continue to grow over time as emerging source systems and requirements increase.
5 – Developing ETL processes from multiple systems to a single data warehouse is not only challenging, but requires continuous work, as source systems and output requirements are constantly changing and growing.
6 – Building integrations into source systems is a one-time effort - but keeping them running requires continuous maintenance.
Why are BI Projects so Tedious and Expensive?
The answer to this question is also divided into several aspects - I will try to give the most compact answer possible with the help of the following diagram. In advance: Of course, the "Make" presentation is very generalized, in truth, there is a wide range in terms of both costs and project duration, depending on company size and project ambitions. Our example is near the bottom of this range. The "Buy" representation, in turn, is equally variable depending on the provider; because I can classify the price of minubo, the solution of my own company most accurately, and, of course, can recommend it to every company in retail and manufacturing, also in terms of solution scope and quality, I have used it for the "Buy" comparison.
"Make" and "Buy" in a five-year comparison:
Click on image to enlarge
- Costs in comparison: Actually, it is obvious that the costs for a self-built BI solution have to be much higher than for a purchased standard software solution. With purchased solutions, the concept and development is already complete, and the associated costs are covered by a large number of customers, whereas, the full cost burden must be absorbed by those who choose the self-built option. This not only means high personnel costs for reinventing the wheel in project and development (i.e., business and technology), but also costs for a dedicated tech stack from hosting via ETL and data warehouse to visualization tools and more or less additional costs for external consultancy. In terms of tech costs, a factor adding to the significantly lower "Buy" costs is that the cost of the "Buy" provider's central tech stack are not only shared by all customers, but, in particular for Cloud Vendors such as minubo, are so severely economized due to scaling effects that the tech cost per customer is only a fraction of what a proprietary tech stack would pay.
- Time to productive use in comparison: The decisive factor is here almost more obvious than with the cost - a finished solution does not have to be built first, but only implemented. In the case of minubo, this means a simple onboarding, which is limited to the connection of the ready-to-use solution to the source systems of the customer. Since this primarily takes place via standardized interfaces, such onboarding takes just a few weeks, with little to no effort on the part of the customer. Thus, while the self-built option is first conceptualized, developed, and tested, usually with several delays due to ever-changing prioritization due to the in-house project agenda (or budget surprises within the BI project itself), while undergoing lengthy trial-and-error processes, because internal teams usually have less technical expertise and experience than specialized providers, a purchased solution like minubo can be productive after just one to two months. Everything that is desired for this quickly implemented (and usually covering at least 80-90% of the customer requirements) base in terms of customization and extension of the solution scope, can be subsequently implemented in parallel with its use in production.
So, a closer look reveals not only where the particular challenges of the "Make" approach in business intelligence lie, but also the immense gap between "Make" and "Buy" costs, as well as the respective time until productive use becomes visible.
Yes, Even the "Make" Approach has Arguments on its Side
Even if I myself am a solution provider, I can certainly understand individual aspects of the reasoning of the "Make" lobby. In the following table, I summarized the advantages and disadvantages of both approaches (here again I used minubo on the "Buy" page as an example):
Click image to enlarge
It becomes apparent that the advantages of the "Make" approach lie, above all, in the top three aspects: coverage of requirements, control over the setup, and in-house competence building. It becomes equally clear, however, that at least minubo as a specific "Buy" alternative also offers concrete added value in these aspects. In terms of requirements coverage it is even far ahead until the “Make” variant can catch up (if at all, because the realities of implementation usually differ enormously from the Make-a-Wish feature list from the requirements collection). Therefore, in view of this comparative list, I can only repeat my unbelieving questions from the beginning of this article: Why on earth is the "Make or Buy" question regarding BI still being asked by so many companies? ... and, moreover, answered with "Make!" by a lot of them (up to the crashing failure of the projects in, according to my personal perception, at least 80% of the cases)? If anyone can explain that to me, feel free to contact me at email@example.com.
"Make and Buy" - Always Worth Considering
At the end of this article (despite the early "Buy" plea above), one important aspect must not be overlooked: the "Make and Buy" scenario. In the context of business intelligence and analytics, "Make and Buy" is, in my view, always an option worth considering; especially if the DIY BI does not need to be created from scratch, but already exists. An according scenario might look like this (again with minubo as the "Buy" example):
We see that a) both stacks are fed from partly different, partly common data sources, i.e., contain data warehouses with different contents; that b) the two data warehouses can feed each other with these different, cleanly modeled datasets, and that c) the simple visualization and/or Excel frontends of self-built BI are complemented by the more comprehensive and user-friendly frontend of minubo (with standard and possibly individual tools). In addition, d) operational systems are fed from the minubo database, which is built also for that use case, enabling efficient, intelligent processes and automation. Overall, this scenario (depending on the setup) may, for example, have the following four advantages:
- Do not lose time: Be up and running with minubo quickly as you begin to develop your own solution.
- Use what's already there: In-house BI is often restricted to the ERP system from a data-scope perspective and (as a result of the increased usage complexity of the frontends for other user groups) is generally aimed primarily at Analysts and Controllers. Keep what works for them and complement it with what your broader operational user base needs.
- Combine finance and business perspectives: minubo does not focus on Finance requirements, whereas in-house BI often does - with a combined setup you bring both worlds together.
- Stay cost-effective: Don’t do the heavy lifting in data processing yourself, but focus on complementing what you cannot buy more cost-effectively.
Conclusion and Outlook
So far on the "Make or Buy" question - I hope I have been able to explain why the "Buy" path in the context of Business Intelligence should hardly be less obvious for every company than in the context of operational systems such as (e) Commerce platform, ERP or CRM. The advantages of a purchased BI solution are too obvious - and they can be customized or extended by the vendor or supplemented by DIY components as needed. In conclusion, I would like to make it very clearl: Stop making - buy!
If you opt for the "Make" approach in your BI initiative nevertheless, I have two recommendations for you:
- The minubo Commerce Intelligence Blueprint on the basic approach to a BI project and its key success factors.
- The “Commerce Reporting Standard” initiative by minubo and Project A, which, with its Transaction Metrics Matrix, provides a jump start in terms of the data model.
If you would like to exchange views on the "Make or Buy" question in person, feel free to contact me at any time - simply by sending a message to firstname.lastname@example.org. The next article, following on from today’s article, will then deal specifically with the topic of individualization.
If you would like to stay informed about this and other articles, just sign up for the automatic email update - there will be a new post approximately once a month.
In Business Intelligence (BI), the question of "Make or Buy" seems to be much more...