Data Warehousing

  • Middleware - ODBC, OLE, OLE DB, DCE, ORBs, and JDBC.
  • Data base connectivity - ODBC, JDBC, OLE DB, and others.
  • Data management - ANSI SQL and FTP.
  • Network access - DCE, DNS, and LDAP.

Regardless of what standards they support, major data warehousing tools are meta data-driven. However, they don’t often share meta data with each other and vary in terms of openness. “So research and shop for tools carefully,” advises Thornthwaite. “The architecture is your guide. And use IT advisory firms, like GartnerGroup, META Group, Giga, etc.”

How detailed does a data warehouse architecture need to be? The question to ask is this: Is this enough information to allow a competent team to build a warehouse that meets the needs of the business? As for how long it will take, the architecture effort will grow exponentially as more people are added for its development (i.e., it becomes “techno-politically complex”), and more complex the resulting system needs to be (i.e., “functionally complex”).

Like almost everything in data warehousing, an iterative process is best. You can’t do it all at once because it’s too big—and the business won’t wait. Also, Thornthwaite says, the data warehouse market isn’t complete yet. So begin with high leverage, high-value parts of the process. Then, use your success to make a case for additional phases.

Conclusions

To sum up, the benefits of having a data warehouse architecture are as follows:

  • Provides an organizing framework - the architecture draws the lines on the map in terms of what the individual components are, how they fit together, who owns what parts, and priorities.
  • Improved flexibility and maintenance - allows you to quickly add new data sources, interface standards allow plug and play, and the model and meta data allow impact analysis and single-point changes.
  • Faster development and reuse - warehouse developers are better able to understand the data warehouse process, data base contents, and business rules more quickly.
  • Management and communications tool - define and communicate direction and scope to set expectations, identify roles and responsibilities, and communicate requirements to vendors.
  • Coordinate parallel efforts - multiple, relatively independent efforts have a chance to converge successfully. Also, data marts without architecture become the stovepipes of tomorrow.

Thornthwaite recommends that companies align with business requirements but to be practical. He also emphasizes the importance of keeping up with advances in the data warehouse industry. Finally, remember that there is always an architecture: implicit or explicit, “almost in time” or planned. Experience shows that the planned and explicit ones have a better chance of succeeding.

Source

“From Bauhaus to warehouse: Understanding data warehouse architecture requirements,” presented by Warren Thornthwaite at DCI’s Data Warehouse Summit, held December 8-10, 1998 in Phoenix, AZ.

References

1. Star schema designs produce a large fact table and many, smaller dimension tables which extend the different aspects of the facts. By processing the dimension tables first, fewer of the detailed records in the fact table need to be scanned to complete the query. (From Interactive Data Warehousing, by Harry Singh. Prentice-Hall, 1999, p.163.)

2. Normalization is the process of removing all model structures that provide multiple ways to know the same fact; a method of controlling and eliminating redundancy in data storage. (from Interactive Data Warehousing, by Harry Singh. Prentice-Hall, 1999, p.444.)

Pages: 1 2 3 4 5

Latest Blog Entries

Full Blog »
  • June 21, 2013

    A History of the HMIS & Cloud Computing

    Posted by Erica Harrison

    In May 1729 Philadelphia passed a law allowing citizens to deport local homeless persons.1

    In 1873, Civil War veterans suffered derogatory slurs such as “bums” and “tramps” after railroad giant Jay Cooke & Company shut its doors, eliminating the main source of employment for soldiers post Civil War.2
    Views on homelessness, and the services provided to those experiencing homelessness, [...]

  • May 22, 2013

    Business Process Automation: The Role of Customized Software

    Posted by Erica Harrison

    There is no argument that IT has significantly enhanced business productivity. However, by doing so, IT has in turn increased the volume and diversity of IT-dependent services. This dramatic increase in demand for efficient and complex IT solutions has stretched IT infrastructures to the limits, especially in regards to scalability. Despite the seemingly automated characteristics of IT services, many IT processes are still manual and thus susceptible to human error. These errors can bring business operations to a halt, regardless of the size of the organization.

    There is a solution however; one that can drastically reduce or even eliminate human error while also reducing costs and improving productivity. This solution comes in the form of Business Process Automation (BPA)

  • March 19, 2013

    API: The Solution to Centralized & Coordinated Assessment

    Posted by Erica Harrison

    Although homelessness in America has declined by 5.7 percent since 2007, the amount of homeless persons has remained relatively unchanged, decreasing only slightly since 2011.[1]

    This lack of reduction is certainly not due to lack of effort. CoCs around the nation have significantly boosted their efforts to fight homelessness in their communities. Instead, this slow rate of decline is due in part to economic reasons, but largely due to data communication barriers among and within CoCs nationwide.

  • Read All