The Complete Picture: Understanding the Full Software Procurement Lifecycle

 If you regularly respond to Requests for Proposals (RFPs), you've likely mastered crafting compelling responses that showcase your solution's capabilities. But here's something worth considering: RFPs are just one piece of a much larger puzzle.

Like many professionals, I used to focus solely on the RFP itself - until I realized how much happens before and after that document gets issued. Understanding this complete lifecycle doesn't just make you better at responding to RFPs; it transforms how you approach the entire sales process.

1. Request for Information (RFI): The Discovery Phase

Before any RFP exists, organizations typically begin with an RFI (Request for Information). Think of this as their research phase - they're exploring what solutions exist in the market without committing to anything yet.

Key aspects of an RFI:

• Gathering market intelligence about available technologies

• Identifying potential vendors with relevant expertise

• Understanding current capabilities and industry trends

Why this matters: When you encounter vague or oddly specific RFPs, it often means the buyer skipped or rushed this discovery phase. A thorough RFI leads to better-defined RFPs that are easier to respond to effectively.

Real-world example: A healthcare provider considering AI for patient records might use an RFI to learn about OCR and NLP solutions before crafting their actual RFP requirements.

2. Request for Proposal (RFP): The Formal Evaluation

This is the stage most vendors know well - when buyers officially outline their needs and ask vendors to propose solutions.

What buyers are really doing:

• Soliciting detailed proposals from qualified vendors

• Comparing solutions, pricing, and capabilities systematically

• Maintaining a transparent selection process

Key to success: Generic responses get lost in the shuffle. The winners are those who submit tailored proposals that directly address the buyer's specific pain points with clear, relevant solutions.

3. Proposal Evaluation: Behind Closed Doors

After submissions come in, buyers begin their assessment. This phase combines:

Technical evaluation: Does the solution actually meet requirements?
Financial analysis: Is it within budget with no hidden costs?
Vendor assessment: Do they have proven experience and solid references?

Pro tip: Even brilliant solutions can lose points on small details. Include a clear requirements mapping table to make evaluators' jobs easier.

4. Letter of Intent (LOI): The Conditional Commitment

When a buyer selects their preferred vendor, they typically issue an LOI. This isn't a final contract, but rather a statement that says, "We plan to work with you, pending final terms."

Why this stage is crucial: It allows both parties to align on key terms before investing in full contract negotiations.

For other vendors: Don't despair if you're not the primary choice. Many organizations maintain backup options in case primary negotiations fall through.

5. Statement of Work (SOW): Defining the Engagement

Before work begins, both parties collaborate on an SOW that specifies:

• Exact project scope (inclusions and exclusions)

• Clear timelines and milestones

• Defined roles and responsibilities

The value: A well-crafted SOW prevents scope creep and ensures everyone shares the same expectations from day one.

6. Purchase Order (PO): The Green Light

The PO transforms the agreement into an official, legally-binding commitment covering:

• Payment terms and schedule

• Delivery expectations and deadlines

• Formal authorization to begin work

Critical importance: Never start work without this formal authorization - it's your financial and legal safeguard.

7. Project Execution: Delivering on Promises

This is where your solution comes to life through:

• Development and testing

• Performance validation

• Final deployment

Key insight: How you execute often matters more than what you promised. Delivering as promised (or better) builds the foundation for long-term relationships.

8. Post-Implementation: The Long Game

The relationship doesn't end at go-live. Ongoing success requires:

• Responsive support and maintenance

• Continuous performance monitoring

• Regular updates and improvements

Strategic value: This phase often determines whether you'll secure renewals and expansions. It's where you prove your commitment to long-term partnership.

Why This Holistic View Matters

Understanding the complete procurement lifecycle enables you to:

• Craft more effective proposals by anticipating the buyer's full journey

• Develop strategies that address needs beyond the immediate RFP

• Position yourself as a strategic partner rather than just another vendor

Final thought: When you respond to an RFP, you're not just submitting a proposal - you're entering a relationship that will evolve through all these stages. The most successful vendors understand and prepare for this entire journey, not just the initial document.

The Journey Beyond Learning: My Year at IIM Lucknow

A year ago, I embarked on a journey at IIM Lucknow, driven by the pursuit of professional growth. I sought knowledge, expertise, and a refined understanding of business dynamics. But as I stand at the end of this transformative chapter, I realize I am leaving with something far greater—a profound evolution of my spirit, character, and perception of life.

What began as a quest for professional excellence soon unfolded into a deeply personal and spiritual exploration. The structured curriculum, case discussions, and strategic frameworks were invaluable, but what truly shaped me was the realization that growth is not just about skills—it’s about resilience, patience, and self-discipline. And nowhere was this lesson more evident than in a simple yet powerful idea: “I can think, I can wait, I can fast.”

The Wisdom of Siddhartha: The Lessons We Often Overlook

Hermann Hesse’s Siddhartha tells the story of a man in search of enlightenment. When asked about his abilities, Siddhartha humbly states:
“I can think, I can wait, I can fast.”
At first glance, these may seem like ordinary statements. But as I reflected on them, I saw their profound relevance—not just in spiritual journeys but in our professional and personal lives as well.

Thinking: The Power of Deep Contemplation

In an environment as intense as IIM, quick decisions and rapid problem-solving are often celebrated. But I realized that the true power lies in the ability to pause, reflect, and analyze beyond the obvious. Critical thinking is not just about finding solutions—it is about questioning assumptions, challenging biases, and understanding perspectives beyond our own. The ability to think deeply is what sets apart great leaders from the rest.

Waiting: The Strength in Patience

Patience is an underrated virtue in a world that demands instant results. IIM taught me that waiting is not about inaction—it is about perseverance. There were times when ideas took longer to materialize, when failures felt discouraging, when the next step seemed uncertain. But waiting allowed me to develop resilience, to trust the process, and to realize that true success is not immediate—it is earned over time.

Fasting: The Discipline to Endure

Fasting is not just about food—it is about the ability to withstand hardships and resist temptations. In the corporate world, in leadership, and in life, there will be moments of struggle, of deprivation, of difficult choices. The ability to endure, to sacrifice short-term pleasures for long-term goals, is what defines true strength. At IIM, I learned to push beyond my comfort zone, to embrace challenges with determination, and to understand that true discipline is the key to transformation.

More Than an Institution—A Journey of Self-Discovery

IIM Lucknow was not just an academic experience; it was a crucible that shaped my mind, spirit, and character. I came seeking professional advancement, but I left with something far deeper—an understanding of what it means to be a better human being.

Beyond business models and strategy decks, I learned that the greatest asset is self-awareness, the greatest skill is patience, and the greatest success is inner peace.

A heartfelt thanks to Professor Neerja Pande, whose guidance in communication not only refined my professional skills but also enlightened us with a path of spirituality and wisdom, leading to profound personal and professional growth.

As we strive for excellence in our careers, let us not forget to nurture the qualities that make us better individuals—the ability to think, to wait, and to fast. Because in mastering these, we master not just our professions but our very existence.

This is not just my story—it is a reminder for all of us, and a lesson we must pass on to the next generation.

The Evolution of AI Assistants: From Generic to Personalized Recommendations

In the world of AI, the difference between a generic bot and a personalized assistant is like night and day. Let me walk you through the journey of how AI assistants are evolving to become more tailored and intuitive, offering recommendations that feel like they truly "know" you.

The Generic Bot: A One-Size-Fits-All Approach

The first bot we’ll discuss is a generalized AI assistant built on generic data. It’s designed to provide recommendations and answers based on widely available information. While it’s incredibly useful, it has its limitations. For instance, if you ask it for a restaurant recommendation, it might suggest popular places but won’t consider your personal preferences. The responses may vary slightly depending on how the question is phrased, but fundamentally, the recommendations remain the same for everyone.

This bot is a great starting point, but it lacks the ability to adapt to individual users. It doesn’t know your likes, dislikes, or unique needs. It’s like talking to a knowledgeable stranger—helpful, but not deeply connected to you.

The Personalized Bot: Tailored Just for You

Now, let’s talk about the second bot—a fine-tuned, personalized assistant. This bot is designed specifically for an individual, taking into account their preferences, habits, and even past interactions. For example, if the user is a vegetarian, the bot will recommend vegetarian-friendly restaurants without being explicitly told each time. It remembers the user’s preferences and uses that information to provide highly relevant recommendations.

This level of personalization makes the bot feel like a close friend who truly understands you. It’s not just an assistant; it’s a companion that grows with you, learning from your interactions and adapting to your needs.

The Value of Personalization in AI

The shift from generic to personalized AI assistants represents a significant leap in technology. Here’s why it matters:

1. Relevance: Personalized bots provide recommendations that align with your unique preferences, making them far more useful.
2. Efficiency: By knowing your preferences, the bot can save you time by filtering out irrelevant options.
3. Connection: A personalized assistant feels more intuitive and human-like, fostering a stronger bond between the user and the technology.

The Future of AI Assistants

As AI continues to evolve, we can expect more assistants to move toward personalization. Imagine a world where your AI assistant not only knows your favorite foods but also understands your mood, anticipates your needs, and offers support tailored to your personality. This is where AI is headed—a future where technology feels less like a tool and more like a trusted companion.

Final Thoughts

The journey from generic to personalized AI assistants highlights the incredible potential of AI to transform our lives. While generic bots are useful, personalized assistants take the experience to a whole new level, offering recommendations and support that feel uniquely yours. As we continue to innovate, the line between technology and human-like understanding will blur, creating a future where AI truly knows and cares about you.

Thanks for reading, and here’s to a future filled with smarter, more personalized AI!

Optimizing Azure Document Intelligence for Performance and Cost Savings: A Case Study

    As a developer working with Azure Document Intelligence, optimizing document processing is crucial to reduce processing time without compromising the quality of output. In this post, I will share how I managed to improve the performance of my text analytics code, significantly reducing the processing time from 10 seconds to just 3 seconds, with no impact on the output quality.

Original Code vs Optimized Code

Initially, the document processing took around 10 seconds, which was decent but could be improved for better scalability and faster execution. After optimization, the processing time was reduced to just 3 seconds by applying several techniques, all without affecting the quality of the results.

Original Processing Time

• Time taken to process: 10 seconds

Optimized Processing Time

• Time taken to process: 3 seconds

Steps Taken to Optimize the Code

Here are the key changes I made to optimize the document processing workflow:

1. Preprocessing the Text

Preprocessing the text before passing it to Azure's API is essential for cleaning and normalizing the input data. This helps remove unnecessary characters, stop words, and any noise that could slow down processing. A simple preprocessing function was added to clean the text before calling the Azure API. Additionally, preprocessing reduces the number of tokens sent to Azure's API, directly lowering the associated costs since Azure charges based on token usage.

def preprocess_text(text):
    # Implement text cleaning: remove unnecessary characters, normalize text, etc.
    cleaned_text = text.lower()  # Example: convert to lowercase
    cleaned_text = re.sub(r'[^\w\s]', '', cleaned_text)  # Remove punctuation
    return cleaned_text

2. Specifying the Language Parameter

Azure Text Analytics API automatically detects the language of the document, but specifying the language parameter in API calls can skip this detection step, thereby saving time.

For example, by specifying language="en" when calling the API for recognizing PII entities, extracting key phrases, or recognizing named entities, we can directly process the text and skip language detection.

# Recognize PII entities
pii_responses = text_analytics_client.recognize_pii_entities(documents, language="en")

# Extract key phrases
key_phrases_responses = text_analytics_client.extract_key_phrases(documents, language="en")

# Recognize named entities
entities_responses = text_analytics_client.recognize_entities(documents, language="en")

This reduces unnecessary overhead and speeds up processing, especially when dealing with a large number of documents in a specific language.

3. Batch Processing

Another performance optimization technique is to batch multiple documents together and process them in parallel. This reduces the overhead of making multiple individual API calls. By sending a batch of documents, Azure can process them in parallel, which leads to faster overall processing time.

# Example of sending multiple documents in one batch 

documents = ["Document 1 text", "Document 2 text", "Document 3 text"] 

batch_response = text_analytics_client.analyze_batch(documents)

4. Parallel API Calls

If you’re working with a large dataset, consider using parallel API calls for independent tasks. For example, you could recognize PII entities in one set of documents while extracting key phrases from another set. This parallel processing can be achieved using multi-threading or asynchronous calls.

Performance Gains

After applying these optimizations, the processing time dropped from 10 seconds to just 3 seconds per execution, which represents a 70% reduction in processing time. This performance boost is particularly valuable when dealing with large-scale document processing, where speed is critical.

Conclusion

Optimizing document processing with Azure Document Intelligence not only improves performance but also reduces operational costs. By incorporating preprocessing steps, specifying the language parameter, and utilizing batch and parallel processing, you can achieve significant performance improvements while maintaining output quality and minimizing costs by reducing token usage.

If you’re facing similar challenges, try out these optimizations and see how they work for your use case. I’d love to hear about any additional techniques you’ve used to speed up your document processing workflows while saving costs.

Building BloomBot: A Comprehensive Guide to Creating an AI-Powered Pregnancy Companion Using Gemini API

Solution approach for BloomBot

1. Problem Definition and Goals

Objective:

• Develop BloomBot, an AI-powered chatbot tailored for expecting mothers to provide:
  • Pregnancy tips
  • Nutrition advice by week
  • Emotional support resources
  • A conversational interface for queries

Key Requirements:

• AI-Powered Chat: Leverage Gemini for generative responses.
• User Interface: Interactive and user-friendly chatbot interface.
• Customization: Adapt responses based on pregnancy stages.
• Scalability: Handle concurrent user interactions efficiently.

---

2. Architecture Overview

Key Components:

1. Frontend:

   • Tool: Tkinter for desktop GUI.
   • Features: Buttons, dropdowns, text areas for interaction.

2. Backend:

   • Role: Acts as a bridge between the frontend and Gemini API.
   • Tech Stack: Python with google.generativeai for Gemini API integration.

3. Gemini API:

   • Purpose: Generate responses for user inputs.
   • Capabilities Used: Content generation, chat handling.

4. Environment Configuration:

   • Secure API key storage using .env file and dotenv.

---

3. Solution Workflow

Frontend Interaction:

• Users interact with BloomBot via a Tkinter-based GUI:
  • Buttons for specific tasks (e.g., pregnancy tips, nutrition advice).
  • A dropdown for selecting pregnancy weeks.
  • A text area for displaying bot responses.

Backend Processing:

1. Task-Specific Prompts:
   • Predefined prompts for tasks like fetching pregnancy tips or emotional support.
   • Dynamic prompts (e.g., week-specific nutrition advice).
2. Free-Form Queries:
   • Use the chat feature of Gemini to handle user inputs dynamically.
3. Response Handling:
   • Parse and return Gemini's response to the frontend.

Gemini API Integration:

• Models Used: gemini-1.5-flash.
• API methods like generate_content for static prompts and start_chat for conversational queries.

---

4. Implementation Details

Backend Implementation

Key Features:

1. Pregnancy Tip Generator:
   • Prompt: "Give me a helpful tip for expecting mothers."
   • Method: generate_content.
2. Week-Specific Nutrition Advice:
   • Dynamic prompt: "Provide nutrition advice for week {week} of pregnancy."
   • Method: generate_content.
3. Emotional Support Resources:
   • Prompt: "What resources are available for emotional support for expecting mothers?"
   • Method: generate_content.
4. Chat Handler:
   • Start a conversation: start_chat.
   • Handle free-form queries.

Code Snippet:

class ExpectingMotherBotBackend:
    def __init__(self, api_key):
        self.api_key = api_key
        genai.configure(api_key=self.api_key)
        self.model = genai.GenerativeModel("models/gemini-1.5-flash")
    
    def get_pregnancy_tip(self):
        prompt = "Give me a helpful tip for expecting mothers."
        result = self.model.generate_content(prompt)
        return result.text if result.text else "Sorry, I couldn't fetch a tip right now."

    def get_nutrition_advice(self, week):
        prompt = f"Provide nutrition advice for week {week} of pregnancy."
        result = self.model.generate_content(prompt)
        return result.text if result.text else "I couldn't fetch nutrition advice at the moment."

    def get_emotional_support(self):
        prompt = "What resources are available for emotional support for expecting mothers?"
        result = self.model.generate_content(prompt)
        return result.text if result.text else "I'm having trouble fetching emotional support resources."

    def chat_with_bot(self, user_input):
        chat = self.model.start_chat()
        response = chat.send_message(user_input)
        return response.text if response.text else "I'm here to help, but I didn't understand your query."

---

Frontend Implementation

Key Features:

1. Buttons and Inputs:
   • Fetch pregnancy tips, nutrition advice, or emotional support.
2. Text Area:
   • Display bot responses with a scrollable interface.
3. Dropdown:
   • Select pregnancy week for tailored nutrition advice.

Code Snippet:

class ExpectingMotherBotFrontend:
    def __init__(self, backend):
        self.backend = backend
        self.window = tk.Tk()
        self.window.title("BloomBot: Pregnancy Companion")
        self.window.geometry("500x650")
        self.create_widgets()

    def create_widgets(self):
        title_label = tk.Label(self.window, text="BloomBot: Your Pregnancy Companion")
        title_label.pack()

        # Buttons for functionalities
        tip_button = tk.Button(self.window, text="Get Daily Pregnancy Tip", command=self.show_pregnancy_tip)
        tip_button.pack()
        
        self.week_dropdown = ttk.Combobox(self.window, values=[str(i) for i in range(1, 51)], state="readonly")
        self.week_dropdown.pack()
        nutrition_button = tk.Button(self.window, text="Get Nutrition Advice", command=self.show_nutrition_advice)
        nutrition_button.pack()

        support_button = tk.Button(self.window, text="Emotional Support", command=self.show_emotional_support)
        support_button.pack()

        self.response_text = tk.Text(self.window)
        self.response_text.pack()

    def show_pregnancy_tip(self):
        tip = self.backend.get_pregnancy_tip()
        self.display_response(tip)

    def show_nutrition_advice(self):
        week = self.week_dropdown.get()
        advice = self.backend.get_nutrition_advice(int(week))
        self.display_response(advice)

    def show_emotional_support(self):
        support = self.backend.get_emotional_support()
        self.display_response(support)

    def display_response(self, response):
        self.response_text.delete(1.0, tk.END)
        self.response_text.insert(tk.END, response)

---

5. Deployment

Steps:

1. Environment Setup:
   • Install required packages: pip install tkinter requests google-generativeai python-dotenv.
   • Set up .env with the Gemini API key.
2. Testing:
• Ensure prompt-response functionality works as expected.
• Test UI interactions and Gemini API responses.

6. Monitoring and Maintenance

• Usage Analytics: Track interactions for feature improvements.
• Error Handling: Implement better fallback mechanisms for API failures.
• Feedback Loop: Regularly update prompts based on user feedback.

Amazon's Digital Transformation: A Journey of Innovation and Expansion

 

Amazon.com is a well-known online ecommerce trading platform that has transformed the way we shop. Its Marketplace platform, with superior data analytics and logistics efficiency, has enabled it to create a pseudo perfectly competitive market, optimize and reduce the cost of products, and offer personalized recommendations to customers.

 

Amazon's Core Competency: The Marketplace Platform

Amazon's business model is driven by data and technology, allowing it to innovate and transform existing business models in multiple industries and domains. Its core competency lies in finding service improvement opportunities in business models and driving innovation using technology and data.

 

Amazon's Next Growth Engine: AWS

In the short term, Amazon's next growth engine is its cloud computing services, AWS. AWS is a critical part of Amazon's core business, as it can process and churn massive amounts of data and drive innovation using technology and data. AWS also generates a large portion of Amazon's revenue and profit.

 

Amazon's Expansion to Growing Economies

As Amazon expands to growing economies, it faces competition from regional competitors who have a better understanding of the local market and may receive better support and protection from local governments. To succeed, Amazon needs to learn the local markets better, possibly by partnering with or acquiring local businesses, and adapting to the government and regulatory policies in different regions.

 

In conclusion, Amazon's digital transformation journey shows a company that is constantly innovating and expanding, using data and technology to drive growth and success.

 

Copilot & GPT for product managers & owners

 

GPT, or Generative Pretrained Transformer, is a type of AI language model that can generate human-like text. This technology has the potential to revolutionize the way we create content, making the process faster, easier, and more efficient.

I am a Product owner & manager, and in my role -

·         I need to create a lot of contents fast,

·         Analyze a lot of data.

·         Need to find documents fast in my system.

·         And need to discover documents to refer across the organization.

·         Churn out POCs very quickly.

And for all these, GPT has been God send for me. And thanks to my organization they have made MS Copilot available to me. Microsoft 365 Copilot internally utilizes GPT-4.

Do not consider this as the ultimate reference, but some of the ways I have used GPT & Copilot are.

·         Using Copilot to create meeting summaries, sharing it with the team, and keeping for personal reference.

·         Creating reference documents on a particular subject – by downloading transcriptions of all previous meetings recorded and combining them with all notes and creating a summarized document from them and then finally manually structuring them.

·         I rarely have recently created a document on MS doc from scratch without getting a draft from MS doc Copilot.

·         Created notes by summarizing multiple documents to create information from large data and have made them consumable for wider audience.

·         Created presentations, slides & organized slides from unstructured documents and have made them consumable for wider audience.

·         You can create your own chat bot by putting your tabular data on an excel table and then use it for question answering using Copilot over excel.

·         I have discovered documents across the organization for reference using Microsoft 365 Copilot.

·         I have used Microsoft 365 Copilot to ask questions & find documents while searching for particular contents within my personal Organization One drive.

·         Being an AI product owner, I have particularly used Copilot to demonstrate the concept of Sentiment analysis over MS words & excels which I never thought could have been possible.

I think Copilot on an average has saved 30-45 minutes ~ improved 5-7% of my productivity, on a daily basis.

Evaluating the success of an AI&ML use case

              Data science team has finished development of the current version of the ML model & has reported an accuracy or error metric. But you are not sure how to put that number in context. Whether that number is good or not good enough.

                In one of my previous blogs, I have addressed the issue of AI investment and how long before the business can know if the engagement has some potential or it is not going anywhere. This blog can be considered an extension of the above-mentioned blog. If you haven’t checked it out already, please visit : https://anirbandutta-ideasforgood.blogspot.com/2023/07/investment-on-developing-ai-models.html

                In my previous blog, I spoke about the KPIs like Accuracy and Error as thumb-rules to quickly assess on the potential success of the use case. In this blog, I will try to add more specificity or relativeness to it.

                Fundamentally, to evaluate the latest performance KPI of your AI&ML model, there are 3 ways you can go about it, in independence or in combination.

Consider human level performance metric.

For AI use cases which has the primary objective of replacing human effort, this can be considered the primary success metric. For example, if for a particular process the current human error stands at 5%, and the AI can have less or equal to 5% error rate, it can be determined a valuable model. Because AI with the same error rate, bring along with it - smart automation, speeding up the process, negligible down-time etc.

Example: Tasks which needs data entry can easily be replicated by AI. But the success criteria for AI does not need to be 100% accuracy for adoption, but just must match the accuracy which the human counterpart was delivering, to be adopted for real word deployment.

Base Model metric

In use cases for which problem areas getting addressed are more theoretical in nature, or the discovery of the business problem that can get addressed is in progress, its best to create a quick simple base model and then try to improve the model with each iteration.

For example: Currently I am working on a system to determine if a content is created by AI or not. For the lack of any past reference based to which the accuracy can be compared, I have taken this approach to determine the progress.

Satisfying & optimizing metric

We outline both, a metric that we want the model to do as good as possible (we call this optimizing metric) while also meeting some minimum standard which makes it functional and valuable in real life scenarios (we call this satisfying metric)

Example: For Home Voice Assistant, the optimizing metric would be the accuracy of a model hearing exactly what someone said. The satisfying metric would be that the model does not take more than 100 ms to process what was said.

AI TRUST & ADOPTION – THE METRICS TO MONITOR

 

Trust is critical to AI adoption. With more deployment of next generation of AI models, building trust on these systems becomes even more vital and difficult. For example, although with the amazing capabilities Generative AI, LLMs are delivering, it brings along with it the trouble of it being larger, complex, and opaque than ever. This makes identification of the right metrics and continuously monitoring and reporting them imperative.

Below are some of the most critical metrics that every organization & business should be continuously monitoring and have the capability to report them as and when necessary.

DATA

•       Date of instances

•       Date processed.

•       Owner & steward

•       Who created it?

•       Who funded it?

•       Who’s the intended user?

•       Who’s accountable?

•       What do instances (i.e., rows) represent?

•       How many instances are there?

•       Is it all of them or was it sampled?

•       How was it sampled?

•       How was it collected?

•       Are there any internal or external keys?

•       Are there target variables?

•       Descriptive statistics and distributions of important and sensitive variables

•       How often is it updated?

•       How long are old instances retained?

•       Applicable regulations (e.g., HIPAA)

 

MODELS

•       Date trained.

•       Owner & steward

•       Who created it?

•       Who funded it?

•       Who’s the intended user?

•       Who’s accountable?

•       What do instances (i.e., rows) represent?

•       What does it predict?

•       Features

•       Description of its training & validation data sets

•       Performance metrics

•       When was it trained?

•       How often is it retrained?

•       How long are old versions retained?

•       Ethical and regulatory considerations

 

BIAS remains one of the most difficult KPI to define & measure. Hence, I am excited to find some measures which can contribute measuring presence of BIAS in some format.

•          Demographic representation: Does a dataset have the same distribution of sensitive subgroups as the target population?
•          Equality of opportunity: Like equalized odds, but only checks the true positive rate.
•          Average odds difference: The difference between the false positive and true positive
•          Demographic parity: Are model prediction averages about the same overall and for sensitive subgroups? For example, if we’re predicting the likelihood to pay a phone bill on time, does it predict about the same pay rate for men and women? A t-test, Wilcoxon test, or bootstrap test could be used.
•          Equalized odds: For Boolean classifiers that predict true or false, are the true positive and false positive rates about the same for sensitive subgroups? For example, is it more accurate for young adults than for the elderly?
•          Average odds difference: The difference between the false positive and true positive
•          Odds ratio: Positive outcome rate divided by the negative outcome rate. For example, (likelihood that men pay their bill on time) / (likelihood that men don’t pay their bill on time) compared to that for women.
•          Disparate impact: Ratio of the favorable prediction rate for a sensitive subgroup to that of the overall population.
•          Predictive rate parity: Is model accuracy about the same for different sensitive subgroups? Accuracy can be measured by things such as precision, F-score, AUC, mean squared error, etc.

But considering all the above, we must be very sensitive and be cognizant of the business & social context while identifying our above mentioned “sensitive group.”

By no means, it is a exhaustive list, but only a start towards a safer & fairer digital ecosystem. I will try my best to consolidate new information.

 

Thanking dataiku, some information collected from dataiku report: How to build trustworthy AI systems.