Apr 15, 2024
At the time when artificial intelligence is being talked about even on public transportation, and serious authorities are bellowing about AI’s imminent takeover of humanity, it is natural also for business to be interested in AI solutions. Probably most companies are now wondering how to create an AI-based product or at least implement some AI-based solution. Every company wants to be „powered by AI.“
And here comes the fundamental question – do we really need AI in our company?
A common (misleading) image of AI is the perception of a magic glass ball, which may be asked a question like this: what do I need to improve in my company to be successful? Well, unfortunately, it does not work that way. That is why I strongly encourage you to consult your ideas with an AI specialist before you start shopping.
If you are convinced that you want to go in this direction, a solid business analysis should be conducted and a number of aspects specific to the purchase of AI systems should be considered. Your purchasing department will have something to do!
Business Perspective of Buying AI
AI systems are a special form of IT systems, so some aspects of their purchase will be similar. Without going too much into details, consider the following:
- Alignment with business objectives: When considering the purchase of an AI system, it’s critical to ensure that the technology aligns with your business objectives. This means the AI solution should not only support your current goals but also enhance your business’s overall strategy. The added value comes from the AI system’s ability to streamline operations, improve decision-making, and provide insights that were previously unattainable. It’s important to identify specific goals and outcomes that the AI system is expected to achieve and ensure they are in line with the long-term vision of your business.
- Cost-benefit analysis: The financial aspect of implementing an AI system is a crucial consideration. Conducting a thorough cost-benefit analysis helps in understanding the return on investment (ROI) of the technology. This analysis should take into account not only the initial costs associated with purchasing and installing the AI system but also the ongoing expenses related to maintenance and updates. It’s important to weigh these costs against the potential benefits, such as increased efficiency, reduced operational costs, and improved customer satisfaction. Remember, some AI systems can be significantly expensive to develop and maintain, so it’s vital to ensure that the expected benefits justify these costs.
- Integration capabilities: The ability to integrate the AI solution with your existing systems and infrastructure is essential for smooth operation and maximal efficiency. Seamless integration ensures that there are no disruptions to current processes and that the AI system complements and enhances existing workflows. Before making a purchase, verify that the AI solution is compatible with your current IT environment and that it can be easily integrated with other tools and platforms your business utilizes.
- Scalability and flexibility: As your business grows and evolves, so should your AI system. Scalability and flexibility are important characteristics to consider when purchasing an AI solution. The system should be able to handle increasing amounts of data and users without significant performance degradation. Additionally, it should be adaptable to changing business needs and capable of evolving with advancements in AI technology. Consider both the physical infrastructure and software capabilities to ensure that the system can scale and adapt over time.
- Vendor reliability and support: Finally, the reliability and support offered by the AI system vendor are vital. Since AI systems are often complex and unique, reliable customer support and service are essential. Evaluate the vendor’s track record, focusing on their stability, reputation, and the quality of customer service provided. It’s important to select a vendor that offers timely and effective support to address any issues that may arise. Consider the potential risks of the vendor suddenly disappearing or failing to provide adequate support, as this could leave your business facing unresolved challenges with the system.
Artificial Intelligence’s Law Regulations
The great public interest and controversy surrounding AI has prompted measures to regulate the this field. The current regulatory landscape for AI systems is complex and varies significantly by region, focusing on several key areas including transparency, human agency, accountability, technical robustness, diversity, privacy, and social well-being.
In the European Union, the AI regulation is being shaped by the Artificial Intelligence (AI) Act, which was formally adopted by European Parliament on March 13. 2024. The AI Act introduces a risk-based classification system, dividing AI technologies into categories from prohibited to minimal risk, applying different levels of regulatory requirements based on the perceived risk levels. Depending on the risk level, different formal and technical obligations are imposed on the system provider.
The AI Act defines a number of terms, such as what is classified as an AI system, that have a bearing on which category a system will be classified into. This, in turn, will affect obligations and restrictions, the violation of which carries very severe financial penalties. It is therefore worth familiarizing yourself with the details of the regulations, as ignorance can cost you dearly. Importantly, what is decisive for the applicability of the regulations contained in the AI Act is that the AI system is made available within the EU, not the location of the supplier. Thus, EU regulations will have an impact on AI systems developed around the world, well, unless some company can afford to be absent from the European market.
The United States has a more fragmented approach, with sector-specific laws rather than a broad federal framework. However, recently the Biden-Harris Administration has issued an Executive Order on AI, focusing on safety, privacy, and innovation, which serves as a guideline for AI development and utilization. Congress has introduced various legislative proposals aimed at addressing AI’s impact on national security, consumer privacy, and ethical standards. Additionally, individual states have been actively developing their own attitude to AI.
Globally, countries like Brazil, China, Singapore, and South Korea are developing their own AI governance models, ranging from principles-based approaches to more detailed regulatory frameworks. In addition to national recommendations, there are also recommendations from various international organizations, such as the Organization for Economic Co-operation and Development (OECD), the World Economic Forum (WEF), and even UNESCO. These mainly deal with ethical aspects, but there are also more specific recommendations related to the process of purchasing AI systems (EU, WEF). This global variation poses challenges for international businesses, which need to navigate a patchwork of AI regulations that can impact competitiveness, innovation, and legal compliance.
Given the dynamic and diverse legal environment surrounding AI, it is difficult to give anything more concrete and universal advice than: „Know the law, because Ignorantia iuris nocet (Latin: Ignorance of the law harms)“
Data, the foundation of AI systems
Data is a treasure trove of knowledge – about the world, about your business, about your company, about your customers. Sometimes it is very sensitive business data or legally protected personal information that you would not want to share. You can draw a lot of interesting conclusions from the data that will be useful to your business, and AI systems can be taught to do this automatically. However, to do this, AI systems need to learn from sample data, so-called training data, and you need to test the system’s actions on the next batch of data. Hence the need for AI systems to use large amounts of diverse data, and this entails many things to consider:
- Data sources: The foundation of any AI project is the data it uses for learning. Identifying and sourcing the right data is crucial—whether it’s internal data, publicly available open-source data, or synthetic data created to simulate real-world scenarios. Each source has its strengths and pertinence to different AI applications, and the reliability and relevance of these sources directly impact the performance and accuracy of AI systems.
- Data Ownership and Licensing: Before deploying AI systems, it’s essential to clarify who owns the data at various stages—prior to processing, during, and after AI processing. Data ownership and licensing rights determine how data can be used and shared. Ensuring you have the right to use your data with AI systems, and understanding how data ownership changes, if at all, after AI has processed this data, are critical legal considerations.
- Limitations of use: Clearly defining how your data can be used by AI systems and any limitations on its use protects your business interests and ensures compliance with legal and ethical standards. This might involve specifying the purposes for which the AI system can use the data or restricting the use of sensitive or personal information.
- External data: In many cases, external data sources may be required to enrich the AI model’s learning process or to generate synthetic data. The cost of acquiring or generating this data, alongside considerations for its relevance and quality, must be factored into the project planning.
- Data management: The AI system provider’s data management practices are pivotal in safeguarding the integrity and security of your data. This encompasses how data is collected, stored, processed, and protected. Effective data management practices are essential for maintaining data quality and compliance with regulations.
- Data privacy: Complying with data privacy laws such as the General Data Protection Regulation (GDPR) in the European Union or the California Consumer Privacy Act (CCPA) in the United States is non-negotiable. AI systems must be designed to handle personal data ethically and securely, ensuring individuals‘ privacy rights are respected.
- Access: Determining who has access to the data used and generated by AI systems is crucial for maintaining control and security. In the event of a data breach, understanding the liability and having protocols in place for rapid response is essential for mitigating risks and legal repercussions.
- Exit strategies: At the conclusion of a contract with an AI system provider, clear agreements should be in place regarding how data will be handled. This includes whether data will be destroyed, how it will be securely transferred back to you, or if it remains with the provider, under what conditions and protections.
Ethics, Sustainability, Security
AI systems can operate on very sensitive data, e.g. personal, medical, biometric data, and can also automatically make various decisions that have a major impact on individual people, e.g. in recruitment processes or credit ratings. Therefore, it is extremely important to attach a particularly prominent role to data security and the entire system.
One of the widely discussed concerns is related to the ethics of AI. If the objectives set for an AI system do not align with ethical norms or societal values, the system might achieve its goals in ways that are harmful or unethical. It may be also the result of biases present in the training data which leads to discriminatory or unfair outcomes. The opaque nature of many AI algorithms, known as the „black box“ phenomenon, makes it difficult to understand and rectify their decision-making processes. Insufficient testing across diverse scenarios can result in AI systems that do not perform ethically in unanticipated situations. To focus on more specific aspects, please refer to the below points:
- Ethical Principles: Ensuring AI operations adhere to ethical guidelines means embedding values like fairness, accountability, and transparency within the AI’s decision-making processes. This involves setting clear ethical standards during the AI’s development phase and ensuring these principles guide the AI’s behavior, preventing outcomes that could be considered unjust or prejudicial.
- Privacy and Data Protection: Respecting user privacy involves implementing data handling practices that comply with regulations such as the GDPR, which emphasizes consent, data minimization, and the right to erasure. AI systems should be designed to protect personal information, using encryption and anonymization techniques where appropriate to safeguard user data from unauthorized access or breaches.
- Sustainability: Addressing the environmental impact of AI involves optimizing the energy efficiency of data centers and utilizing green technologies to reduce the carbon footprint associated with training and operating AI systems. This might include selecting energy-efficient hardware or using renewable energy sources to power AI operations.
- Transparency and Explainability: Making AI systems‘ decisions understandable to users involves developing models that can be easily interpreted, or providing detailed documentation that explains how decisions are made. This could mean avoiding overly complex models when simpler ones suffice or developing tools that visualize how inputs are transformed into outputs.
- Security Measures: Protecting AI systems from cyber threats requires comprehensive security protocols, including regular vulnerability assessments, the use of secure coding practices, and the implementation of access controls and encryption. AI systems should be resilient against both external attacks and insider threats, ensuring data integrity and system availability.
- Human Oversight: Incorporating human judgment into AI systems involves creating mechanisms for human intervention in critical decision-making processes, especially where there are significant consequences for errors. This might include setting thresholds for when an AI’s decision should be reviewed by a human or developing interfaces that facilitate human-AI collaboration.
- Accountability: Establishing accountability in AI systems means having clear policies on who is responsible for the AI’s decisions and actions, including mechanisms for addressing grievances or harms that arise from AI behaviors. This involves documenting decision-making processes and maintaining logs that can be audited.
- Continuous Monitoring: AI systems should be subject to ongoing scrutiny to ensure they continue to operate as intended and do not develop harmful biases or vulnerabilities over time. This could involve regular re-assessment of the AI’s performance against ethical, security, and operational benchmarks.
- Collaboration and Engagement: Building AI systems that are ethical, secure, and sustainable requires engaging with a broad range of stakeholders, including those who might be impacted by the AI’s deployment. This means soliciting feedback from users, consulting with experts in ethics and law, and working with communities to ensure the AI aligns with societal values and norms.
Summary
The procurement of AI systems underscores the critical role of procurement departments in ensuring that these technologies align with organizational needs and ethical standards. These departments are pivotal in navigating the complexities of AI solutions, ensuring that they not only meet operational requirements but also adhere to relevant regulations and ethical guidelines. Compliance with regulations, such as data protection laws and industry-specific standards, is crucial in the procurement process to mitigate risks associated with privacy, security, and ethical concerns. Ultimately, the effective procurement of AI systems requires a collaborative effort across departments to ensure that the solutions are beneficial, compliant, and ethically sound.