REVIEWER: Roberto Monge, chief architect at TransGaming Inc.
BACKGROUND: TransGaming, Inc. is the global leader in the deployment and distribution of electronic entertainment across multiple platforms. TransGaming works with the industry's leading developers and publishers to enable and distribute games for Smart TV set-top boxes, Mac computers and Linux/CE platforms. TransGaming is headquartered in Toronto, Canada, with offices in Ottawa and Atlanta.
PLATFORMS: The product was not installed locally. We leveraged Apica’s data center in order to run repeatable loads.
PROBLEM SOLVED: We recently launched GameTreeTV, an on-demand gaming service that runs on Smart TV. The new service provides a library of on-demand family-friendly games that run directly on their existing set-top box and controller. As we prepared for our release with the European broadband provider, Free, our operations team moved to independently verify and test that our systems could handle the expected television spike traffic levels. Unlike traditional websites, the traffic would be instantaneous – as soon as the operator turned on the gaming channel. In short, we needed to be 100 percent sure the service we were offering worked under high concurrent loads.
PRODUCT FUNCTIONALITY: We selected the RightScale Cloud Computing Management Platform to manage our cloud infrastructure. Apica, a RightScale partner, was brought in to handle system testing. With RightScale and Apica, we were able to achieve more than 400 percent improvement in performance and concurrency testing, when compared to our initial tests. The Apica data allowed us to easily tune the systems for high-spike loads. Apica’s ability to drive load from its European cloud agents provided valuable insight on the response times that European users would be experiencing. Using the Apica LoadTest data, RightScale was able to configure different quantities and machine sizes, which allowed us to optimize server sizing and costs for the projected load. This level of analyst and reconfiguration would have been difficult to perform with non-cloud deployments and without Apica’s real-world testing experiences and scenarios. A high-concurrency testing followed this step, which identified a number of bugs that weren’t earlier identified from our normal stress-testing process.
STRENGTHS: Apica LoadTest technology was able to drive highly concurrent instantaneous loads in a repeatable manner. Since it’s offered via SaaS, we didn’t have to provide hardware or do software installations, which was a big benefit. We gained great insight from Apica’s experienced testing team. Our overall experience with Apica has been very positive.
WEAKNESSES: In order to get the benefit of Apica’s high-performance data center, minimum reservation is required. I would like to be able to continually use Apica’s systems as needed in an on-demand basis without having to reserve large time blocks.
SELECTION CRITERIA: Apica LoadTest was selected because of its ability to expertly simulate actual set-top box user scenarios in complex scripts and for its ability to drive load from multiple geographies, particularly in Europe, where our first customer was located. We also zeroed in on Apica for its deep technical experience with high-volume caching systems, like Varnish. We initially began with some homegrown load testing tools alongside open source tools like httperf, ab and JMeter. We were able to drive a good amount of load, but the analysis and management of the data and load test system began to take time away from our core product development. We decided to partner with someone so that we could focus on improving the system.
DELIVERABLES: Over a three-week period, we were able to achieve more than 400 percent improvement in performance and concurrency testing with RightScale and Apica, when compared to our initial testing results. Additionally, we were able to identify anticipated spike loads and plan and tune our systems accordingly. The analysis and data provided by Apica identified specific configurations and code that when fixed and restarted, showed significant improvement in concurrency. And finally, reporting was great. Reports were very graphical, easy to read and easy to understand.
VENDOR SUPPORT: The Apica team worked side by side with our team. They worked long hours and helped us overcome some difficult issues. Their committed test team made the load testing almost fun.
DOCUMENTATION: Apica’s load test team ran the testing so we didn’t have to directly understand their load generator tools. The reports after the load tests were easy to understand.
Register or login for access to this item and much more
All Information Management content is archived after seven days.
Community members receive:
- All recent and archived articles
- Conference offers and updates
- A full menu of enewsletter options
- Web seminars, white papers, ebooks
Already have an account? Log In
Don't have an account? Register for Free Unlimited Access