Generally, organisations need to look at two different ways to incubate ML practices i.e. whether to handle and manage supervised data and learning or to manage un-supervised data and learning.
“Based on these two areas, complete business and tech strategy need to be implemented. This requires lots of understanding of business statistics and data science algorithms, hardware, scale and extensible architecture. This also calls for distributed systems, Big Data, hadoop ecosystem and microservices understanding, where ML systems are continuously getting stream of data from the source and keep learning and predicting outcomes. Already many E-comm and social media companies are utilizing these for customer experience and thus revenue maximization”, Saurabh Chandra, IIM-A (Ex- Engineering Leader, Amazon) said in discussion with ETCIO.
Chandra talks about top challenges CTOs face while designing ML & Analytics framework:
Real challenge faced by CTO’s today is – migration from old system to new one where in traditional system; scale, extensibility at this stage to enable Analytics and ML was not conceptualized.
Second is to keep up the pace of business alongside transformation. Third is re-skilling the workforce and create a culture of learning as Big data and microservices space is entirely new and governed by open-source technologies where continuous learning ability is a key to sustain.
Fourth is to influence and engage business to have a buy-in to take route of transformation, which means the project prioritization really need to be vetted thoroughly and in limited time and capacity both BAU (business as usual) as well as transformation charter need to be executed.
Fifth, to change the culture of organisation where speed and change is key else though you may have data but you will not be able to clock benefits of the same and lastly, to be attuned with regulations and policies around the globe as whole GDPR across world is catching up attention.
Sixth and last, leader herself/himself need to be very technically savvy to be one of the architects of the system and thus need to possess very high end expertise, and hands on knowledge on analytics and ML subject coupled with Open source and MicroServices technology.
Chandra shares some tips & tricks to follow while deploying ML & Analytics systems:
The first and foremost is to understand “what’s the problem one wants to solve” and if it is not about scale or extensibility then one must think twice as investment to tap big data technologies is huge both in terms of cost and time. Once this part is clear then one needs to understand the finer nuances of business objectives and take an approach whether to move towards null approach or CSF (Critical Success Factor) approach or not. Post one is very clear on business requirements and approach then comes study of existing monolithic applications and re-architect phase wise for migration. It is unlike that you can shut down the existing system and start working on new. Lots of tech excellence and dive deep is needed in this aspect because the challenge is to minimize redundancy of tech work as making redundancy zero is highly unlikely.
Then is to take decisions around build or buy products. Mostly for scale and extensible, investing in build is critical but in case one is not a technology company then buy should be the approach. No one way is the right way to say rather is governed by the nature of business and what future one is carving out for the business.
Post this analysis, then comes techstack shortlisting such as Java , Go lang, Hadoop, NoSQL DB’s or relational DB, Kafka, Spark, ELK monitoring stacks to name a few. And hence comes the decision about skills, resourcing , budgeting and timing. If it’s a global company having footprint across the world then there’s benefit as one can think about multi-country strategy to attract talent and increase the speed by follow the sun model.
Read more about developing successful use case of ML & Analytics system in the Indian retail sector here at The Dilemma To Design- “India Retail Analytics, ML System”: IIM-A Research.
Credit: Google News