The amount of data feeding into our lives has become unmanageable and overwhelming.
My own daily communications consist of a huge volume of emails, messages, and texts. Going through them is intensively time consuming, when only a minority of items are important or even relevant. If I could get someone to simply go through my Slack messages and say: “you can ignore 80% of these, here are the three things that are a priority, and here are some items we couldn’t figure out” it would significantly increase my productivity and make me a more effective leader.
In the enterprise world this is happening with data. There is so much data coming from different sources that many people are trying to do data consolidation. A sophisticated AI that could analyse it reliably would save so much time. But it’s important to understand what AI really is and the limits of what we can do. While we hear AI as a buzzword every day, we’re still in the earliest infancy of the technology and its potential to evolve our work and life.
Understanding what AI is
We need to understand what is and what isn’t AI. For example, a digital assistant like Alexa isn’t AI, it’s just voice recognition (at least at this stage). AI is essentially a bunch of algorithms. If you apply them to the right data, you can get good output. As the algorithms are refined, hopefully it continues to adapt and get better. But we’re still so far away from this.
For example, when it comes to my work messages volume issue, there are tools and virtual assistants that work through it. But they’re not yet reliable enough to trust completely – just consider the important emails that get auto-filtered into your Spam folder. We still have to manually check our Spam folders just to ensure that important things aren’t missed
Machine learning doesn’t always make machines more intelligent
The holy grail is machine learning – algorithms that can improve over time and become increasingly more useful and reliable as they take in more and more data. But this depends on the data being good. Bad data can actually un-train machines. I’ve got an app on my phone that guesses the words you’re trying to use by using swype gestures. It has a self-learning algorithm that supposedly makes it more accurate over time. But while it started out well, it got “dumber” the more I used it. Eventually I had to delete and reinstall it, to reset, and it became much better again.
We’re just not there yet with refining algorithms. The challenge is that to have good AI, you need to train it with high quality data. Data needs to be consistent, clean and accessible. Bad data has been described as“enemy number one” for machine learning. Data cleaning is very labour intensive – data scientists estimate it takes up 80% of their time – and even if you start out with the right data, it could be compromised by bad data later on.
Automation doesn’t create job losses
“I don’t want AI to take over my job” is a phrase that’s uttered by many in IT, and while there’s fear that automation and AI will make human beings redundant, it’s a long way off from creating job losses. As an example, if you’ve got 100 operations engineers in an organisation, much of their time is spent filtering through issues, sifting through data to find patterns and optimisations – not to mention managing priorities from management, engineering and even customers. If AI could remove 60% of that noise, does that not mean that you need 40% less operations engineers? No. On the contrary, these highly skilled employees can start focusing on more strategic work rather than time-consuming administrative tasks.
What AI does is make people’s jobs more effective, freeing us all up to be more innovative. Devs don’t want to spend all their time triaging. Instead of looking through 300 alerts every morning, AI can help to prioritise them. Instead of waiting for someone to complain, AI can help detect unusual activity or changes. Instead of sifting through thousands of lines of code and hundreds of containers/servers, AI can help get to the root cause faster.
AI means a lot of different things to many people. It’s a space of continuous innovation, but it’s also still in its infancy. While we would like for it to be smarter and faster, we have a lot of work to do in terms of collecting, storing, and managing the data required for the machine learning models to be more effective.
Myk Shaforostov is Senior Director of Sales Engineering – ANZ at New Relic
This article was first published by Business IT