Deloitte Director Greg Szwartz outlines the following six things you need to consider when approaching a big data integration project.
There are many decisions to be made when considering the information architecture design as it relates to big data storage/analysis. These include choosing either commodity or special-purpose hardware; relational or non-relational data stores; virtualized on-premise servers or external clouds; in-memory or disk-based processing; uncompressed data formats (quicker access) or compressed (cheaper storage). Companies also need to decide whether or not to shard split tables by row and distribute them across multiple servers to improve performance. Other choices to be made include either column-oriented or row-oriented as the dominant processing method and hybrid platform or greenfield approach.