Thinking about AI and ML? You’ll need more than data!
Enterprises embarking on the AI and ML journey shouldn’t rely solely on data scientists to see the bigger picture.
Data scientists play a critical role in realising the potential of Artificial Intelligence (AI) and its various subsets like Machine Learning (ML), both of which wouldn’t exist without the capability to analyse and draw meaningful insights from large volumes of data.
However, enterprises embarking on this journey often make a common mistake. They rely solely on data scientists to deploy these technologies and attribute any failures to them. Data scientists are actually just one part of the bigger picture when it comes to building and executing a successful AI and ML implementation.
Successful cognitive application in production is not just a data science problem but also a software engineering problem that involves several key roles. What’s needed is a phased approach that uses different disciplines and collaborative processes within the organisation, ensuring everything from the creation of the models to the real-world testing and integration into the business processes is handled successfully.
The essential players are referred to as the 4Ds:
- data scientists to leverage data and create models
- designers to work on UI and UX
- developers to develop the right software or application that applies the models to business processes
- DevOps to manage and update the infrastructure, integrations, deployments and model management.
Ensure you get your AI and ML foundations right
Understanding the need, alongside enabling machinery, to collect data is the first critical step for organisations that are developing their AI strategy. The next step is to get the right teams in place. The DevOps team who manage the infrastructure will work to collect and store the data.
Then, data scientists will take over to analyse data at a deeper level and build predictive models. All this takes place within a research environment, whereby data scientists identify the best way to leverage the data, followed by creating models that help make predictions and, ultimately, decisions.
This process becomes more complex when there is a larger volume of data. When it comes to Industrial Internet of Things (IIoT), in particular, where each and every machine produces data per the millisecond, the role of data scientists in developing ML-enabled analytical models is pivotal. AI and ML depend on data analytics models, which in turn depend on data scientists’ work. That is either actual human or ML-powered systems that can replicate analysis on a much larger scale.
So, how do you build upwards?
Once the models are in place, it is important for organisations to start thinking about the three other Ds and how they can work with the data scientists to deliver an AI strategy. At this second phase, these models need to be deployed, managed and updated within a dynamically changing scenario, resulting in them ultimately becoming productionised.
This means testing the models within the actual production process, in order to leverage value from the data, which can then be integrated back into business processes. Organisations need to understand that this is a software engineering problem, and not just a data science problem, as is often thought.
Productionising data requires a lot of planning and strategising on how data is analysed, how data analytics models are applied and updated, and how the predictions and insights coming from them are going back into business processes and how the results will be consumed by the end users.
For this to happen successfully, developer teams need to be involved to work on developing the right software or applications. Software and apps will be the platform through which the data models will move to the production process and reach the end-user.
They are a form of aggregated knowledge and intelligence, which in order to be useful, need to be streamlined and fed back into the business operations. But for this to be successful, the design team need to ensure that the software or app has the right user interface (UI) that makes it usable and appealing to the end user.
So, the input of that team and its role in defining the user experience (UX) can make or break the overall project.
Finally, the DevOps team is responsible for managing and updating the production systems, ensuring that the whole process throughout its lifecycle, from design to development and production, runs as smoothly as possible.
Success is all about collaboration
Innovative technologies like AI and ML can completely transform an organisation and hugely improve business processes. They, however, require investment that goes beyond the finance department. Organisations that are looking to deploy these technologies should understand that they need to have a holistic view of their strategy, one that goes beyond the role and responsibility of a single team.
Lack of understanding, alongside an inability to cater for the need to move beyond the research and into the production stage and engage the developer, designer and DevOps teams, is a key reason why most AI and ML projects fail before they are fully realised.
It is time to change our mindset on how these should be deployed, and ensure this is a shared project across the organisation.
Ruban Phukan, VP, Progress
