AI Governance: Artificial Intelligence Model Governance

What are the 4 Key Principles of Responsible AI?

Because AI governance is about protecting both organizations and the customers they serve, defining the key ethical principles of responsible AI is a helpful way to guide policy. In addition, because as a society we’re still in the early stages of AI development and new AI projects emerge daily we must learn from past mistakes and adjust accordingly.

Sometimes machine learning algorithms introduce unintended results as seen in these high-profile cases.

Microsoft Tay

After its launch, Microsoft’s Tay Twitterbot quickly gained 50,000 followers and created more than 100,000 tweets. But after only 24 hours of machine learning, Tay turned into a PR nightmare with some of its offensive tweets and had to be taken offline.

COMPAS Recidivism Algorithm

COMPAS, is a commonly used software in the US to guide criminal sentencing. It was exposed by ProPublica to be racially biased. Black defendants were twice as likely to be misclassified as white offenders.

Apple Card Bias

Tech magnates David Heinemeier Hanson and Steve Wozinak called out Apple for discrimination when their spouses were offered substantially lower credit limits despite having a shared banking and tax history.

Facebook Campaign Ads

Facebook sparked a public outcry that charged the company with putting profits before people and democracy after the company refused to police political ads on its AI-driven network.

These are just some examples of many. None of these companies set out with bad intentions, but these cases show the importance of AI governance and active monitoring.

The Four Key Principles of Responsible AI

Have Empathy

For the Microsoft example, it was Tay’s lack of empathy that caused the issue. The bot was not engineered to understand the societal implications of how it was responding. There were no guardrails in place to define the boundaries of what was acceptable and what might be hurtful to the audience interacting with the bot. The natural language processing error led to a big headache for the company.

Control Bias

AI algorithms make all decisions based on the data at their disposal. In the case of COMPAS, although the developers had no intention of creating a racist AI, the bias it uncovered was a reflection of the bias that exists in the natural world justice and sentencing system. Companies need to regulate machine learning training data and evaluate the impact to catch bias that might have been unintentionally introduced.

Provide Transparency

With negative publicity, it can be a challenge to convince consumers that AI is being applied responsibly. The Apple Card issue really wasn’t that Apple’s decision-making was biased; it was that Apple customer service was unsure how to answer the customer’s concerns. Companies must be proactive about certifying their algorithms, clearly communicating their policies on bias, and providing a clear and transparent explanation of the problem when it occurs.

Establish Accountability

Facebook took a lot of heat for its refusal to hold itself accountable for the quality and accuracy of the information being shown in its ads. Regulation around technology issues is always a few years behind the problem, so regulatory compliance isn’t enough. Companies must proactively establish and hold themselves accountable to high standards to balance the great power AI brings.

How Should AI Governance be Measured?

You can’t manage what you don’t measure. Lack of properly measuring AI models puts organizations at risk. So the question becomes which measures are important? In order to answer that, an organization must be clear on their definition of AI governance and who in the organization is accountable, and what their responsibilities are.

Many measures or metrics for AI governance will be standardized for all organizations through government regulations and market forces. Organizations also need to consider other measures that will support their strategic direction and how the company operates on a daily basis. Some essential facts and data-driven KPIs organizations should consider include:

Data

Measure for the lineage, provenance, and quality of the data.

Security

Data feeds around model security and usage. Understanding of tampering or improper usage of AI environments is critical.

Cost/Value

Define and measure KPIs for the cost of data and the value created by the data and algorithm.

Bias

KPIs that can show selection bias or measurement bias are a must. Organizations need to monitor bias through direct or derived data continuously. It will also be possible to create KPIs that measure information on ethics.

Accountability

Get clarity of individual responsibilities and when they used the system, and for what decisions.

Audit

The continuous collection of data could form the basis for audit trails and allow third parties, or the software itself, to perform continuous audits.

Time

Measurements of time should be a part of all KPIs, allowing for a better understanding of the model over specific time periods.

These are just some of the KPIs for organizations to consider. The sooner measurements are in place, the better they can evolve for a particular organizations’ environment and goals and be incorporated into software. AI governance should be and will likely be a mandatory part of all AI strategy and machine learning environments.

What are the Different Levels of AI Governance?

Level Zero: No AI Governance

At level zero, each AI development team uses its own tools, and there are no documented centralized policies for AI development or deployment. This approach can provide a lot of flexibility and is common for organizations just getting started with AI.

However, it comes with potential risks once the models are deployed into production. Because there is no framework, it’s impossible to evaluate risk. Companies working at level zero have a difficult time scaling AI practices. Hiring more data scientists does not lead to a ten-fold increase in AI productivity because of too many inconsistencies.

Level One: AI Governance Policies Available

Many organizations have already established some level of AI governance but have not developed a fully mature AI governance framework. Most companies are around level two or three, and with a little help, could have a fully mature AI governance system that’s fully automated, saving their organization a substantial amount of resources

Level Two: Create a Common Set of Metrics for AI Governance

This level builds upon level one by defining a standard set of acceptable metrics and monitoring tools to evaluate models. This brings consistency among all AI teams and enables metrics to be compared to different development lifecycles.

A common monitoring framework is introduced that allows everyone in the organization to interpret the metrics the same way. This reduces risk and improves transparency to better make policy decisions or troubleshoot reliability if issues arise. Companies operating at level two usually have a central model validation team upholding the policies laid out by the enterprise during the validation process. Level two is where organizations start to see productivity gains.

Level Three: Enterprise Data & AI Catalog

Level three leverages the metadata from level two to ensure all assets in a model’s lifecycle are available in an enterprise catalog with data quality insights and provenance. With a single data and AI catalog, the enterprise can trace the full lineage of data, models, lifecycle metrics, code pipelines and more.

Level three also lays the foundation for making connections between the numerous versions of models to enable a full audit. It also provides a single view to a CDO/CRO for a comprehensive AI risk assessment. Organizations at this level are able to clearly articulate risks related to AI and have a comprehensive view of the success of their AI strategy.

Level Four: Automated Validation and Monitoring

Level four introduces automation into the process to automatically capture information from the AI lifecycle. This information significantly reduces the burden on data scientists and other role players, freeing them from manually documenting their actions, measurements, and decisions.

This information also enables model validation teams to make decisions on an AI model, as well as allowing them to leverage AI-based suggestions. At this level, an enterprise can significantly reduce the operations effort in documenting data and model lifecycles. It removes risks from mistakes along the lifecycle in terms of metrics, metadata, and versions of data or model being excluded.

Companies at level four start to see an exponential increase in productivity as they’re able to consistently and quickly put AI models into production.

Level Five: Fully Automated AI Governance

Level five builds on automation from level four to automatically enforce enterprise-wide policies on AI models. This framework now ensures that enterprise policies will be enforced consistently throughout every model’s entire lifecycle. At this level, an organization’s AI documentation is produced automatically with the right level of transparency through the organization for regulators and customers.

This level enables the team to prioritize the riskiest areas for a more manual intervention.
Companies here can be highly efficient in their AI strategy while maintaining confidence in their risk exposure.

Why You Should Care About AI Model Governance?

As demonstrated by the case examples in this article, many times AI models are simply not making the right decisions. Even if a model is trained correctly, over time, it will experience drift – it will change. Because of the inevitable drift, you need a way to monitor and capture that change so you can see what changed and make adjustments to the model.

Although most of model governance today focuses on model risk management around compliance, there’s an emerging trend towards social responsibility. Although the issues are not in violation of any government regulations, the financial fallout from poorly behaving models generates bad press and significantly impacts organizational bottom lines.

If these companies had been using an advanced production risk management governance tool like Datatron, they could have avoided costly AI errors and public embarrassment and financial loss.

What To Look For In An AI Governance Solution?

The best AI model governance solutions focus on simplifying the entire process of monitoring and compliance assurance while not causing significant disruptions to existing workflows. They allow key stakeholders in the organizations to have visibility into what the models are doing at all times.

The first line of defense starts with the data scientist, whose job is to ensure model validation using some kind of explainability tool, which is necessary from a developmental perspective but not good enough for actual production validity. The second line of defense helps with the production aspect by monitoring elements while the model is working. The third and most sophisticated line of defense has the capability to provide a detailed audit report that shows what the models are doing with only the output data.

Model Governance Checklist

When choosing an AI governance solution, choose a platform that goes beyond just compliance control. All enterprise-level businesses need to be confident about their model inventory, model development, and model management practices.

Make sure your AI monitoring and governance solution offers the following:

• A dashboard that quickly gives you an overview of your AI model’s health in real-time
• An overall health score that provides intuitive and easy-to-understand measures of all models operating in your system at the model and BU/LOB level
• Bias, drift, performance, and anomaly detection
• Alerts for when a model varies from its pre-defined performance thresholds
• Custom metrics to align with your organization’s KPI metrics
• Activity log & audit trail
• An agnostic tool that can work with the various popular ML development platforms
• Integrate with existing infrastructure, including databases, software, and more

See the Datatron Solution