Seven years ago, I developed a little analytic app in R on my notebook to monitor the performance of the stock market. The program reads/downloads performance data refreshed daily from about 25 different stock index portfolios starting in 1995. From the index-level data, I calculate daily percent changes that lead to growth of $1 calculations for each portfolio over any time period. The app then produces charts in R comparing side-by-side performance of indexes for selected time frames – e.g. 2015 year to date, from the end of the recession to the present, the 20 years starting in July 1995, etc. All index and daily percent change calculations are ultimately written to CSV files and Tableau workbooks for additional processing.
Once a year I revisit the code and without fail make changes/enhancements. At times, especially early on, the updates were to streamline clunky code, at other times to add new capabilities, and finally to redo sections with the latest R features. An example of the former is development of more efficient and readable R data download; examples of the later include migrating the source code to RMarkdown and introducing data.table as the driving R data structure.
Despite the current carnage in the market, I continue to run the program daily and recently completed my annual code review to determine what maintenance to do this Fall. After seven years, I must admit to being a tad disappointed with my work, wondering why I hadn't made the changes I'll make this year – cutting half the program elapsed runtime and making the graphs easier to understand – a long time ago. Yet when all is said and done, the app has served me well in its lifetime, sourcing answers to countless queries on market performance over the last 20 years. My annual updates are simply maintenance to a useful application. Maybe I'll do something entirely new in 2016.
More Blogs From Steve Miller
> R for Data Science with Handley Wickham
> Predictive Analytics and Data Science: Same or Different
>Politics and Analyics, Oh My
I thought about the app and its maintenance recently as my company, Inquidia Consulting, completed an engagement. A customer had brought us in to take a look at a six-year-old analytics application developed on a platform in which we're experts. The point of consulting departure was a “broken” system, the result of “bad design” and a “horrible choice of platform”. All problems, it seemed, derived from those bad choices. Our charter was to first perform triage on the critical patient, then to treat the life-threatening wounds in order of severity. Finally, if there was any time remaining, we were to help them architect a next-generation application built on a replacement software platform.
As we peeled back the system onion, we began to see some patterns. First, several of the goblin-causing problems attributed to the analytics platform were in fact the result of bugs introduced elsewhere in the system chain. Second, the current analytics app actually works pretty well – running fine 90% of the time and meeting most needs.Yes, there are features it doesn't have, as would be expected of any system at age six, and performance could be better. And third, the new developers never invested in learning the analyics software, instead finding it easier to criticize from afar and work with tools they knew.
In the end, after several months of hard work, it appears normalcy has returned. The existing app continues to work reasonably well and will not be retired for at least six months, with support provided by an engaged internal team. In addition,the next generation replacement is concurrently in design, with an anticipated new visual face but a data integration body adapted from the successes of the existing platform. It seems the customer now understands the simultaneous needs of both maintenance and new application development. Much like a certain consultant.
Register or login for access to this item and much more
All Information Management content is archived after seven days.
Community members receive:
- All recent and archived articles
- Conference offers and updates
- A full menu of enewsletter options
- Web seminars, white papers, ebooks
Already have an account? Log In
Don't have an account? Register for Free Unlimited Access