Headquartered in Milpitas, California, Solectron Corporation is a leading electronics manufacturing services (EMS) company. It provides global manufacturing and integrated supply chain services so that its original equipment manufacturer (OEM) customers in the electronics industries can focus on their core competencies and gain a competitive edge by turning to Solectron for important, but non-core activities. Solectron's services include collaborative design, lean manufacturing and fulfillment, and post-manufacturing services. With operations on five continents and locations in more than 15 countries, Solectron incorporates the principles of lean manufacturing and Six Sigma throughout its operations.
RUNNING SUCH A VAST MANUFACTURING NETWORK and global supply chain brings a formidable data management challenge. When Solectron found itself struggling to retrieve and coordinate business-critical information, it turned to Sun Microsystems Inc. and Sun iForce Partner Eakins Open Systems for guidance in developing an improved global data warehouse solution and infrastructure robust and scalable enough to cope with its enormous data inputs.
Kevin Dillon, account executive from Sun, explains, "Solectron faced similar challenges other companies face in the industry. At what point is it no longer cost effective to manage increased business demands on older technologies? Solectron presented us with a challenge based on the needs of its business, and we were able to demonstrate an alternate solution that met its requirements. The Sun account team along with the resources of the Sun reference architecture team, the iForce Business Intelligence Data Warehousing (BIDW) Global Competency Center team and our iForce partner were able to propose a compelling architecture that would consolidate Solectron's infrastructure, consolidate the storage behind the new technology and meet the requirements of the global data warehouse." Steve Doss, director of sales and marketing from Eakins Open Systems, adds, "By consolidating different technologies onto one higher performance system, Solectron is also able to reduce management and data center costs."
"We have plants throughout the world, from Guadalajara, Mexico, to Timisoara, Romania, to mainland China," says Senthil Rajamanickam, enterprise integration leader at Solectron. "When I joined Solectron, the company was in the midst of a reorganization. In January 2003, Mike Cannon joined Solectron as president, CEO and as a director on the company's board. Cannon revamped Solectron's organizational structure so that the company could deliver one face to its global customers and business partners. Before his arrival, Solectron's business units operated independently with little coordination of operations."
The decentralized approach was expensive and inefficient. Solectron realized that in order to remain competitive, it needed to consolidate its data management onto a new data infrastructure.
"So one of the biggest business problems we had was to gain global visibility of data from all of our manufacturing and service locations," Rajamanickam says. "We were doing a lot of batch processing, gathering data from different countries and generating reports to different parts of the organization. From the time a transaction happened, it would take one to two days to be visible globally to the users."
TO ACCOMPLISH THIS GOAL MORE QUICKLY, the company realized it could leverage a data warehouse already in place in the Global Material Services organization, which controlled Solectron's materials supply chain. Rajamanickam says that this unit had developed certain script-based mechanisms to gain visibility into the operations so it could better manage the supply chain, including pulling data from across the world to better manage inventories, materials and other business processes.
The Global Material Services data management infrastructure had a three-tier architecture. The data warehouse consisted of IBM DB2 running on an IBM RS/6000 server. The reporting layer was BusinessObjects running SQL Server on four Windows systems. The ETL (extract, transform and load) layer was Informatica and Oracle database running on a Sun Fire 3800 server.
While it was state of the art when it was first put in place more than four years ago, this infrastructure was struggling with capacity constraints and scalability limitations. "The architecture that existed could not scale beyond a certain point, and we were experiencing bottlenecks," Rajamanickam says. "We regularly had to tell our users that we could not get them the data they required because of the bottlenecks."
SOLECTRON NEEDED TO FIND a more robust hardware platform, architect a new data warehouse for performance, scalability and flexibility, and reduce the complexity of administration. At any one point throughout the life of the project, an IT team, comprising as many as 20 members from Solectron application and infrastructure services, was tasked with implementing this global data warehouse. The team began a review of the data warehouse architecture, with a commitment to not only improving the data warehouse scalability, but also fulfilling other present and future needs of Solectron's IT environment.
"When our team began its review, we learned that the data warehouse was also being forced to double as a makeshift enterprise application integration [EAI] solution. That's when we really became curious, because data warehousing and EAI are two opposite ends of the spectrum of information systems," Rajamanickam says. "Real-time integration is an important goal - but we discovered that the wrong tools and wrong architecture were being used to solve the problems of information latency and information integration for applications."
The consequences of the wrong approach became harsher as the data burden on the system increased. "We were loading close to 90 gigabytes of data every day [into the Global Data Warehouse], and we were about to add more data elements and more sites. The depth and breadth of the data integration were increasing rapidly, and with 24x7 operations as colleagues logged in from the Americas, Asia/Pacific and Europe, the Middle East and Africa, there was only a small window of time during which the data warehouse could be brought down for loading purposes. We were running into very difficult systemic capacity constraints. During the data warehouse loading, users were diverted to a lower capacity system that contained data from the previous day," says Rajamanickam. "The process for our users to access their data was getting longer, and our users were getting more and more upset that we had the data but were not able to deliver it to them," he adds.
THE FIRST CHALLENGE FOR THE SOLECTRON team was to find the right hardware platform on which to consolidate all existing databases and data management applications. "We had to eliminate the multiple hardware platform and move to one system. The top priority for us was the platform's ability to handle tremendous amounts of I/O, because data warehousing systems are basically I/O-bound rather than processor-bound. Also, we wanted to remain in Unix, so we compared Unix servers from multiple vendors. Through analyst reports and technical studies, we determined that the Sun Fire architecture gave us the highest amount of I/O throughput by way of its backplane architecture," Rajamanickam says.
"The specific choice was the Sun Fire 12K server, running the Solaris 8 Operating System, because it is the most scalable and highly available hardware architecture, while remaining still an open system," Rajamanickam adds. "We have two system boards in each domain so that even if one system board fails, another system board will be able to run it, albeit at reduced capacity. That is our assurance against failure. In the last year of its operation, the previous architecture had outages every six weeks or so. Since we implemented the new architecture, we have not had any unscheduled downtime," he says.
"One of the issues that we had with our previous architecture was that the BusinessObjects environment was running on an Intel server, and the administrator had to regularly reboot the operating system. This was a very painful process because anyone with outstanding queries would be rudely and abruptly terminated," Rajamanickam says. "With our new architecture running the Solaris 8 Operating System, we have never had to reboot. The best thing about Solaris 8 is that it is - as any good operating system should be - invisible," Rajamanickam says.
THE SUN FIRE 12K ARCHITECTURE BROUGHT another advantage into play for Solectron's data consolidation efforts. It can be easily partitioned into multiple domains, each essentially acting as an independent machine. As Solectron worked to migrate data from three separate platforms into a single data warehouse, it created a domain for each platform on the Sun Fire 12K server (plus two more for backup). "Also, Sun has the reputation and track record in providing scalability and delivering on performance and availability. Sun consultants' professionalism impressed us. In addition to implementing the architectural design, they performed a SunReady Availability Assessment, Application Readiness Service and RAS Profile that were part of a High Availability Service Pack for the Sun Fire 12K server. The Sun consultants also transferred a lot of knowledge to us through mentoring. They went out of their way to help us out. It's a discretionary effort that no hardware/software can replace."
The need for consolidation also drove the decision to replace the legacy IBM DB2 database that had been the core of the original data warehouse with Oracle9i databases. Rajamanickam says, "The choice of Oracle made it easier to pump large amounts of data back and forth from one system to another."
Explains Rajamanickam, "At the Sun iForce BIDW Global Competency Center in Menlo Park, California, we tested a mechanism for moving data from one machine to another machine, with both machines sharing the same storage area network (SAN) without actually moving data - just moving meta data using Oracle's transportable tablespace mechanism. This is a great capability in the Oracle database, and it really helped our situation." Rajamanickam adds, "It is also one reason we moved from IBM DB2 to Oracle. IBM DB2 at that point in time didn't have any equivalent capabilities."
PRIOR TO THE PROOF-OF-CONCEPT at the Sun iForce BIDW Global Competency Center, Sun professionals presented to Solectron the iForce Enterprise Data Integration (EDI) Reference Architecture with Informatica as a solution to its challenges. Sun then proposed three architectural designs to Solectron and worked with the company's IT team to select the right one. Based on experience and best practices from the EDI Reference Architecture, the Sun team was able to help Solectron to set up a highly scalable data warehouse in days instead of months while minimizing costs. "The proof-of-concept was accomplished, and it worked just as advertised," Rajamanickam says. "Now when we import the meta data into the data warehouse server, we have about 600 gigabits of data all moved, and this takes us only 5 to 12 minutes per day. It is nearly transparent to the end users, and there is no downtime."
The next step was to address Solectron's external storage strategy. "Sun and Eakins Open Systems suggested that we review the storage options so that we could snapshot data from one application at the storage level and then be able to move that data to other applications and to the data warehouse," Rajamanickam says.
"We were mostly accustomed to direct-attached storage. The lack of scalability and flexibility was an issue," notes Rajamanickam. Eakins and Sun recommended moving from direct-attached storage to a storage area network. Solectron opted for the Sun StorEdge 6320 SAN system with Sun StorEdge Enterprise Storage Manager 1.2, Sun StorEdge Availability Suite and Sun StorEdge Traffic Manager and Sun Management Center 3.5 software. "The Sun StorEdge 6320 system was very suitable for us because this storage was going to be used by other systems in addition to the data warehouse. We could configure this particular machine to cache performance in a way that suits the data warehouse or configure it differently for other purposes. We moved from a storage capacity of not more than 400 gigabytes of raw data with the old architecture to 12 terabytes of raw data on the new system," Rajamanickam says. Sun StorEdge Remote Response service has been activated to provide for off-site monitoring of Solectron's storage environment.
THE NEWLY ARCHITECTED DATA WAREHOUSE, currently at approximately 2 terabytes, has already more than doubled in capacity, and Rajamanickam expects that over the next two years it will double again and possibly even triple. "In the past, we served only materials-related information. Now we are also serving accounts receivable, accounts payable, executive dashboard information and we have also built a complete human resources data warehouse on this infrastructure. In addition, we have added a lot of finance users, especially those who are looking to fulfill Sarbanes-Oxley types of requirements." Rajamanickam says that BusinessObjects alone draws 130-140 simultaneous users querying the data warehouse. Counting other applications and power users, the number of concurrent users rises to about 220. "With the help of Sun and Eakins Open Systems, we have added a tremendous amount of additional capability to our data warehousing infrastructure," Rajamanickam notes.
This re-architecting project came in on time, and under budget by 10 percent. The project was approved in June 2003, started in September 2003 and met the go-live deadline of January 5, 2004. "It is very clear," Rajamanickam says, "that the data warehouse we have would no longer exist if we had continued to run into bottlenecks. The infrastructure was outdated, and if we hadn't gone for this capacity upgrade, our ability to deliver timely data would have been severely compromised. We would have started missing critical data elements that are necessary for customer wins and regulatory compliance."
"I BELIEVE THE COMPANY COULD NOT RUN as effectively as it runs today without this new data warehouse," he adds. "The team did an outstanding job. Not only did we achieve our goals with this solution, but we also achieved benefits beyond our initial scope, including our ability to deliver Sarbanes-Oxley compliance information to our executives, as well as to have a complete global view of our employees and a global view of the various enterprise performance metrics," he says.
The Solectron IT team is pleased with the performance and productivity of the new architecture. "It is my experience that not all capabilities advertised by vendors work in complex environments. However, in the case of Sun, everything worked tremendously well - just as advertised. Thanks to Sun and Eakins Open Systems, we have accelerated our data reporting mechanisms, increased the performance of our data warehouse by 200 percent, reduced the footprint in our data center by 60 percent and reduced our operational costs by 40 percent."
Register or login for access to this item and much more
All Information Management content is archived after seven days.
Community members receive:
- All recent and archived articles
- Conference offers and updates
- A full menu of enewsletter options
- Web seminars, white papers, ebooks
Already have an account? Log In
Don't have an account? Register for Free Unlimited Access