|
A number of inter-related trends
are currently shaping the data warehousing and business intelligence
market:
- warehouse sizes are increasing, thanks to technological
advances;
- warehouse updates are occurring more frequently, sometimes
in near real time;
- costs are falling;
- vendors are consolidating;
- tools and applications are becoming more integrated;
- more off-the-shelf applications are becoming available.
Perhaps the two most significant drivers of change are the
explosive growth of the Internet and the entry of Microsoft
as a major player.
Microsoft’s
market entry
Microsoft now offers an OLAP server and an ETL
tool (called the Data Transformation Service) bundled into
its flagship database product, SQL Server 7.
Whilst take-up has been slow, the impact has been significant:
- The established business intelligence vendors are facing
strong price competition, and are responding by developing
value added analytic applications, and offering end-to-end
solutions i.e. consultancy.
- This has motivated considerable consolidation and integration:
several database vendors, including Oracle, are adding OLAP
support to their main product. Also, several OLAP and ETL
vendors have merged or been acquired.
- Data warehousing is now perceived as affordable by medium
sized organisations.
Impact
of the Internet
Widespread adoption of web-enabled applications has had 3
distinct effects:
- Almost all new warehouse implementations are aiming to
deliver information via the web, initially to in-house staff
via an intranet, and increasingly to external users via
an extranet or even the Internet. This trend is driving
down the cost per user for business intelligence software,
and leading to convergence with tools for building enterprise
information portals.
- The growing number of e-commerce applications provides
a new source of detailed customer information. Many organisations
are now creating data "web-houses" to store this,
and a variety of tools are available for clickstream
analysis.
- There is growing interest in the exchange of information
between partner organisations, with a view to streamlining
processes and providing a better customer experience. Examples
include the e-government initiative, and steps towards greater
supply chain integration in the manufacturing and retail
sectors.
Tanler’s
book is a useful reference for anyone interested in building
web-enabled warehouse applications.
Technological
advances
There is a clear trend towards ever larger data warehouses.
This is being driven by the desire to analyse more detailed
information, in particular customer behaviour, and to do so
in real time. Technological advances in several fields are
acting as enablers:
- Dedicated hardware for high volume, rapid access disk
storage and management;
- Development of parallel
processing architectures;
- Enhanced decision support facilities within relational
database engines, e.g. SQL Server 7.0 and Oracle 9.0;
- Proliferation of robust warehouse design techniques e.g.
dimensional
modelling.
Integrated
applications
As more detailed data are included in the data warehouse
environment, so possibilities to utilise this information
to improve operational decision making multiply. Examples
include questions such as these:
- Should this insurance policy be renewed?
- What other products should be offered to the customer
now on the line?
- Should this customer be allowed an increased credit limit?
These decisions would probably be based on some analysis
of historic data from the warehouse, combined with current
data in a transaction processing system, and are commonly
required in customer
relationship management systems.
Integrating the use of the data warehouse into operational
systems in this way is increasingly a requirement, and is
sometimes referred to as closed loop decision processing (see
closed
loop applications). Implementing this kind of application
is extremely complex, and requires careful consideration of
the overall systems architecture.
|
OVERVIEW
