One of the most popular questions
that I often get asked by the stakeholders is about the timelines required for
a ML model to finish development. I will try to address the subtlety of this topic
in this writeup.
AI development is a
unique scenario where you are expected to deliver an innovation. It is a
special case where the resource required is uncertain. And hence it is
sometimes very difficult to understand when & where to “STOP”.
When I talk to businesses
one of the questions, what I stress about the most, is for them to define what
an “MVP” solution is to them. That is with what minimum accuracy or maximum
error rate the AI solution would still be useful for their business.
If you are investing on
AI use cases one of the concepts, I would recommend you understand is – AI
resourcing & diminishing return. Please look at the graph below –
So, what I suggest to
the AI investors are if you haven’t reached an MVP by the point of maximum
return, “STOP”. For example – By the end of PoMR
if the model is still with an error rate of 30%, and that is something that does
not work for your business, may be AI cannot solve this for you. Maybe it needs
a completely different approach to solve this. Whatever is the case, deploying
more resource is not the solution.
Driving from my
experience with all the AI&ML use cases I have worked for almost a decade
now, a general thumb rule which I recommend is – The accuracy or error rate,
that you get at the end of 3 months is your Point
of maximum return. You should reach an MVP by then. Beyond that it
should be fine tuning or customizing to specific business needs. By then if it it’s
still miles apart from your business objective, may be its time to pull the
plug.
This is again an
opinion piece, and these has been my experience. Will be glad to hear how the
journey has been for you.