Abstract

Machine Learning (ML) tools have become ubiquitous with the advent of Application Programming Interfaces (APIs) that make running formerly complex implementations easy with the click of a button in Business Intelligence (BI) software or one-line implementations in popular programming languages such as Python or R. However, machine learning can cause substantial socioeconomic harm through failures in fairness. We pose the question of whether fairness in machine learning should be regulated by the government, as in the case of the European Unions (EU) initiative to legislate liability for harmful Artificial Intelligence (AI) and the New York City AI Bias law, or if an industry standard should arise, similar to the International Standards Organization (ISO) quality-management manufacturing standard ISO 9001 or the joint effort between ISO and the International Electrotechnical Commission (IEC) standard ISO/IEC 27032 standard for cybersecurity in organizations, or both. We suggest that regulators can help with establishing a baseline of mandatory security requirements, and standards-setting bodies in industry can help with promoting best practices and the latest developments in regulation and within the field.

The ease of incorporating new machine learning (ML) tools into products has resulted in their use in a wide variety of applications, including medical diagnostics, benefit fraud detection, and hiring. Common metrics in optimizing algorithm performance, such as algorithm Accuracy (the ratio of correctly predicted to total number of attempted predicted), do not paint the complete picture regarding False Positives (algorithm incorrectly predicted positive) and False Negatives (algorithm incorrectly predicted negative), nor do they quantify the individual impact of being mislabeled. The literature has, in recent years, created the subfield of Machine Learning Fairness, attempting to define statistical criteria for group fairness such as Demographic Parity or Equalized Opportunity, which are explained in section II, and over twenty others, described in comprehensive review articles like the one by Mehrabi et al (2021).1 As the field of ML fairness continues to evolve, there is currently no one standard agreed upon in the literature for how to determine whether an algorithm is fair, especially when multiple protected attributes are considered.2 The literature on which we draw includes computer science literature, standards and governance, and business ethics.

Fairness criteria are statistical in nature and simple to run for single protected attributesindividual characteristics that cannot be the basis of algorithm decisions (e.g., race, national origin, and age, among other individual characteristics). Protected attributes in the United States are defined in U.S. federal law and began with Title VI of the Civil Rights Act of 1964. However, in cases of multiple protected attributes it is possible that no criterion is satisfied. Furthermore, oftentimes a human decision maker needs to audit the system for compliance with the fairness criteria with which it originally complied at design,3 given that a machine learning-based system often adapts through a growing training set as it interacts with more users. Moreover, no current federal law nor industry standard mandates regular auditing of such systems.

However, precedents exist in other industries for both laws and standards where risk to users exists. For example, in both the U.S. (the Federal Information Security Modernization Act [FISMA], the California Consumer Privacy Act [CCPA]) and EU (the General Data Protection Regulation [GDPR]) laws protect user data. In addition, the industry has moved to reward those who find cybersecurity bugs and report them to companies confidentially, for example through bug reward programs like Googles Vulnerability Reward Program. In this article, we propose that a joint effort between lawmakers and industry may be the best strategy to improve the fairness of machine learning systems and maintain existing systems so that they adhere to fairness standards, with higher penalties for systems that pose greater risks to users.

Before elaborating further on existing regulations, we will briefly summarize what ML fairness is and illustrate why it is a complex problem.

ML fairnessand AI fairness more broadlyis a complex and multidimensional concept, and there are several definitions and metrics used to measure and evaluate fairness in ML-based systems. Some of the most common definitions include:456

The fairness criteria should equally apply to procedural and distributive aspects.9 Other approaches are also possible; among others, the use of balanced accuracy (and its related measures)10 should be explored.

These definitions of fairness often require trade-offs, as optimizing for one may negatively impact another.11 Chouldechova (2017)12 showed that it is not possible for three group-fairness criteria to be satisfied at once, so determining the appropriate fairness metric for a specific AI system depends on the context, the domain, and the societal values at stake. This is a decision that human designers of an ML system need to make, ideally prior to the systems release to actual users. Involving several stakeholders, including users of the ML system, in deciding the fairness metric is helpful to ensure the end-product aligns with the systems ethical principles and goals.13 There is an extensive literature in ethics and psychology revolving around principles of procedural fairness.141516 Throughout the literature, procedural fairness has been broadly understood as perceived fairness of the methods used to make the decisions,17 and involves high-level principles such as correctability (if a decision is perceived as incorrect, the affected party has a mechanism to challenge it), representativeness, and accuracy (in the field of ML algorithms, this means the algorithms rely on valid, high-quality information18), among others (for a detailed explanation of procedural fairness applied to ML see Footnote 17). These principles are found in regulations; for example, the correctability principle is found in GDPR privacy law as a right to rectification and in the right to object found in both CCPA and GDPR.19

Given the rapid advances in machine learning, we recognize that legal frameworks by nations or local governments may be more difficult to develop and update. Thus, we propose first looking at industry standards, which can be used to incentivize companies to perform better while also collaborating on developing standards. Certification of a product to a standard can provide customers with a signal of quality and thus differentiate among ML-based solutions in the market.

Companies who are early adopters of a novel standard of ML fairness may be able to use that standard to gain market share as well as establish a competitive advantage compared to newcomers. For example, a company that invests early on in an auditing team for its ML system may produce more transparent software, which could be apparent to the discerning customer and thus appease concerns customers might have regarding the use of their data. Industry collaboration organizations such as Institute of Electrical and Electronics Engineers [IEEE] have developed standards for recent technologies with the help of leaders in the industry that have resulted in benefits for customers. For instance, the IEEE 802 wireless standards provided a foundation for the now-widespread Wi-Fi technology and, in the early days, provided a badge to signal to customers purchasing computers that the manufacturer complied with the latest Wi-Fi standard. Current updates to the standard enable a new wave of innovation including in areas of indoor mapping. The same incentives for standardization advocated by organizations like IEEE and ISO in manufacturing quality management, electrical safety, and communications may apply to ML.20

In the following section, we include some additional background information on standards and parallels to the cybersecurity standards that could serve as a reference for developing standards for ML fairness.

Standards are guidelines or best practices developed by industry and federal regulators in collaboration to improve the quality of products and services. Although they are often voluntary, organizations choose to adopt them to signal their commitment to security and quality, among other reasons. Some of the most prolific and widely adopted standards in the technology sector are the International Organization for Standardization (ISO) / International Electrotechnical Commission (IEC) 27000 series, the National Institute of Science and Technology (NIST) Cybersecurity Framework, and the Center for Internet Security (CIS) Critical Security Controls.

While standards are recommended, regulations are enforced. Procedures and standards must be regulatorily compliant.2122 Standards are often included by reference in U.S. federal regulations,23 which means that publishing of regulations in the Federal Register and the Code of Federal Regulations (CFR) by referring to materials already published elsewhere is lawful, as long as those materials, like international standards documents, can be accessed by the public. Since some of the standards of organizations like ISO can be hundreds of pages long and are accessible in electronic form, this approach is sensible. Regulations are legally binding rules imposed by governments or regulatory authorities that are mandatory and specify penalties, fines, or other consequences for non-compliant entities. Examples of cybersecurity and privacy regulations include the GDPR in the European Union, and the CCPA and FISMA in the United States.

The following are criteria for evaluating the use of standards and regulations:

Considering these factors, a combination of standards and regulations may be the most timely and effective approach in many cases. Regulations can establish a baseline of mandatory ML security and ethics requirements, while standards can provide guidance on best practices and help organizations stay current with the latest developments in the field.

In the United States, the process of developing cybersecurity standards often involves collaboration among the federal government, industry stakeholders, and other experts. This collaborative approach helps ensure that the resulting standards are practical, effective, and widely accepted.

One key example of this is the development of the NIST Cybersecurity Framework. NIST, an agency within the U.S. Department of Commerce, plays a key function in developing cybersecurity standards and guidelines, yet is not endowed with regulatory enforcement capabilities. NIST often solicits input from industry stakeholders, academia, and other experts to ensure that its guidance is comprehensive and current. Following the issuance of Executive Order 13636, Improving Critical Infrastructure Cybersecurity, in 2013, NIST was tasked with developing a framework to help organizations better understand, manage, and communicate cybersecurity risks.24 To do so, NIST engaged in an open, collaborative process that included the following:

These examples demonstrate the value of public-private partnerships in the development of cybersecurity standards. By involving industry stakeholders in the process, the federal government can help ensure that the resulting standards are practical, widely accepted, and effective in addressing the challenges posed by cybersecurity threats.

Regulations have also played a significant role in governing cybersecurity and privacy and have often been used to set mandatory requirements for organizations to protect sensitive information and ensure privacy.

General Data Protection Regulation (GDPR) European Union: GDPR, implemented in 2018, is widely recognized as one of the most comprehensive data protection regulations worldwide. It requires organizations to protect the personal data of EU citizens, ensuring privacy and security. GDPR has been effective in raising awareness about data protection and pushing organizations to improve their cybersecurity posture and has been seen as setting a global standard25 for user privacy. A research report on GDPR impacts published by the UK government found that GDPR compliance had resulted in more investment in cybersecurity by a majority of surveyed European businesses and that organizations generally prioritized cybersecurity following the new regulations. However, challenges include the complexity of the regulation as well as high compliance costs (further details can be found here).

Health Insurance Portability and Accountability Act (HIPAA) United States: HIPAA, passed in 1996, sets standards for protecting sensitive patient data and requires health care organizations to implement cybersecurity measures for electronic protected health information (ePHI) . Although HIPAA has been successful in improving the safeguarding of patient data, it has faced criticism for being overly complex for patients who may assume it applies to contexts where it may not offer protections (such as in health mobile apps) and for the fact that patients and caregivers of patients can have difficulty accessing necessary records.26 Furthermore, when cybersecurity breaches of private health data occur, it may be difficulty for consumers to know what options they have for recourse. The law as was written in 1996 may require updating in the face of rapidly evolving cybersecurity threats.

California Consumer Privacy Act (CCPA) United States: Implemented in 2020, the CCPA grants California consumers specific data privacy rights, such as the right to know what information is stored, as well as an option for a consumer to opt-out of data sharing. The CCPA has been praised for empowering consumers and raising the bar for privacy protection in the United States. For companies that have customers in California and other states, the CCPA has resulted in a standard for consumer privacy rights that will likely be applied by companies to other states and create dynamics that contribute to shaping the U.S. privacy regulatory framework.27 However, the CCPA does face criticism for several issues including its complexity: the nine exceptions to the right of consumers to delete data may not give consumers the protection that they expect; the burden it places on businesses; and the potential conflicts with other state or federal privacy regulations.28 Some lessons may be drawn from the current complexity of privacy laws to regulation of algorithmic fairness: Consumers may not have the time to read every opt-out notice or legal disclaimer and understand on the spot what rights they may be giving up or gaining from accepting terms of service.

Federal Information Security Management Act (FISMA) United States: Signed into law in 2002, FISMA requires each federal agency to develop, document, and implement an agency-wide program to provide information security for the information and systems that support the operations and assets of the agency. Although FISMA has led to improved cybersecurity within federal agencies, it has been criticized for being overly focused on compliance rather than continuous risk management and for not keeping pace with the evolving threat landscape. FISMA does establish generally applicable principles of cybersecurity in government, such as requiring the NIST to establish federal information processing standards that require agencies to categorize their information and information systems according to the impact or magnitude of harm that could result if they are compromised, which are codified in NIST standards. Broad principles like these are sensible for businesses to adopt as well (for example, at the business-unit or organization-unit levels).

Although regulations can be effective in setting mandatory requirements for cybersecurity and raising awareness, they may also face challenges such as complexity, high compliance costs, outdated provisions, and potential conflicts with other regulations. To address these issues, policymakers should consider periodically reviewing and updating regulations to ensure they remain relevant and effective in the face of rapidly evolving cybersecurity threats. Additionally, harmonizing regulations across jurisdictions and providing clear guidance to organizations can help alleviate some of the challenges associated with compliance. Existing cybersecurity regulations may be a template for ML fairness regulations, as well.

A similar approach can be applied to ML ethics, where regulations can set a legal framework and minimum requirements for ethical ML development and deployment, while standards can provide detailed guidance on best practices, allowing for flexibility and adaptation to new technological advancements.

Until recently, there have been no comprehensive AI-specific regulations in the United States. However, there have been efforts to establish guidelines, principles, and standards for AI development and deployment, both by the U.S. government and by various organizations and industry groups. Some examples include the following:

Executive Order on Maintaining American Leadership in Artificial Intelligence (2019): Issued by the White House in 2019, this order aimed to promote sustained investment in R&D and enhance the United States global leadership in AI. Although it did not establish specific regulations, it directed federal agencies to create a national AI strategy and develop guidance for AI development and deployment.

Defense Innovation Board (DIB) AI Principles (2019): The DIB is an advisory committee to the U.S. Department of Defense (DoD). In 2019, it released a set of ethical principles for AI in defense, covering areas such as responsibility, traceability, and reliability. Although they are not legally binding, these principles provide a basis for the ethical deployment of AI within the DoD.

Organization for Economic Co-operation and Development (OECD) AI Principles (2019): In 2019, the OECD established a set of standards for use of AI that are respectful of human rights and democratic values.

NIST AI Risk Management Framework (2023): NIST has been active in AI research and standardization efforts, focusing on topics such as trustworthy AI, AI testing and evaluation, and AI risk management. In 2023, NIST published the AI Risk Management Framework (AI RMF 1.0), which aims to provide a systematic approach to managing risks associated with AI systems. This framework, once finalized, could serve as a foundation for future AI standards and guidelines in the United States.

Partnership on AI (PAI) Tenets: The PAI is a multi-stakeholder organization that brings together industry, academia, and civil society to develop best practices for AI technologies. The partnership has published a set of tenets to guide AI research and development, including principles such as ensuring AI benefits all, prioritizing long-term safety, and promoting a culture of cooperation.

Industry-specific guidelines and standards: Several organizations and industry groups have developed their own guidelines and principles for AI development and deployment within specific sectors, such as health care, finance, and transportation. For example, PrivacyCon is an annual conference to bring together industry, government, and academia that serves the development of such guidelines in 2020 the theme of the event was health data. In a recent article, Accentures global health industry lead provided some best practices for generative AI in healthcare which are likely just a beginning of guidelines as generative AI grows in adoption in that industry. Bain and Company put together design principles for the use of generative AI in financial services, again likely a topic of growing interest in the coming years. In the transportation industry, a wide consortium of auto manufacturers, chipmakers, and other industry members put together guidelines for automated driving back in 2019. These examples of guidelines, although they are not legally binding, can help set expectations and establish best practices for AI governance within their respective industries. The National AI Institute at the Department of Veterans Affairs has also been building on and harmonizing these frameworks for trustworthy AI and operationalizing them in the health care sector.

AI-related concerns, such as data privacy and algorithmic fairness, may be addressed by existing regulations and guidelines that are not AI-specific, such as GDPR, CCPA, and guidelines on algorithmic transparency and fairness from the Federal Trade Commission (FTC). As AI continues to evolve, it is likely that more targeted regulations and standards will be developed to address AI-specific concerns and ensure ethical and responsible AI governance. Comprehensive AI auditing processes will need to be developed in a timely manner and updated periodically. Additionally, a system of incentives may be needed to encourage companies to actively develop tools to address and solve AI fairness concerns.

Standard-setting bodies work well when there are mechanisms for accountability built-in. For instance, external audit committees29 can provide an accountability mechanism as long as the audits are performed periodically (e.g., quarterly or annually) and if the auditors are not influenced by the position of those who are being audited (no revolving door scenario). To ensure accountability, such auditors may be hired by testing organizations such as the Technical Inspection Association (TUV), Underwriters Laboratories (UL), or Intertek, among others. Alternatively, auditors may be part of a volunteer community, similar to code maintainers in the Linux open-source community who control the quality of any changes to the codebase. Therefore, we suggest creating fairness audits by external auditors to the firm and codifying the type of audit and frequency in an industry standard.

We propose an approach where regulations complement industry standards by adding to them tools for enforcement, rather than as a one-size-fits-all tool. Unfortunately, firmsat least right nowdo not have a strong commercial incentive to pursue ML fairness as a goal of product development and incur additional liability in the absence of agreed upon standards. In fact, it is well known that standards may not reach their potential if they are not effectively enforced.30 Since consumers do not currently have widespread visibility into which products abide by fairness criteria and fairness criteria are not yet accessible to the general consumer since they require specialized knowledge (e.g., data science and programming skills in addition to knowing the literature), it is perhaps not feasible that a majority of consumers could themselves test for unfairness in a product or service. Furthermore, it is not yet the case that most consumers have access to training sets or company proprietary algorithms to prove whether they have been harmed by an ML system, which is required for damages under the newest regulations such as the EU AI Act (see the commentary in Heikkil, 2022). The literature on ML fairness is a complex, multidisciplinary one, so computer scientists, lawyers, ethicists, and business scholars are needed to be part of driving regulations.

Under such circumstances, it is not surprising that companies do not perceive a direct financial incentive to maintain specialized staff to audit ML fairness or supervise with a fairness-oriented goal the development of ML based products, especially in a downturn in the markets. Recently, many leading companies have unfortunately laid off a number of specialized ML fairness engineering staff, in some cases closing entire departments, which results in loss of company-specific knowledge and will mean a much slower adoption of fairness principles in industries in the future. Although regulations can provide general principles (for example, meeting at minimum an equality of opportunity fairness criterion and performing an annual fairness audit; see the guidelines by the Consumer Financial Protection Bureau (CFPB) in auditing compliance to the Equal Credit Opportunity Act [ECOA] in the United States as well as expectations regarding transparency in algorithmic decision-making when related to credit applications) and provide some consumers relief in cases of egregious violations of basic fairness criteria, they are insufficient to provide incentives to companies to perform better than such minimums and do not incentivize companies to innovate beyond meeting the regulatory requirements. Although companies may wish to implement fair ML as part of every stage of their software development processes to ensure they meet the highest ethical standards,31and we encourage that in related work,32 we recognize that people, and thus companies at large, respond to incentives and the current rules of the road are still in their infancy when it comes to ML.

Market forces can provide incentives to companies to innovate and produce better products provided the advantages of the innovation are clear to a majority of consumers. A consumer may choose a product and pay more if it satisfies a fairness standard set by a leading, recognizable standard-setting body and if the benefits of that standard are apparent. For example, a consumer may prefer a lender that markets itself as ensuring no discrimination based on subgroup fairness (i.e., combinations of categories of race, age, gender, and/or other protected attributes) if the alternatives only guarantee group-level fairness. If the consumer is aware of the higher standard this product is satisfyingfor example through a standard that the product is displayingthe consumer may choose to pay a higher price for two feature-equivalent products if one satisfies a fairness standard. Thus, we call on the industry and organizations such as the Information Systems Security Association (ISSA), the IEEE, the Association for Computing Machinery (ACM), and the ISO, among others, to invest in developing an ML fairness standard, communicate their rationale, and interact with policymakers over these standards as they deploy them into products over the next five years.

We also suggest that firms create bug bounty programs for fairness errors, in which users themselves can file bug reports with the producer of the ML system, much like what exists in cybersecurity. For example, if the ML system is a voice assistant that often misunderstands the user based on a speech impediment due to a disability, the user should be able to report that experience to the company. In another example, a user utilizing an automated job resume screening tool (as some companies now have implemented in their hiring processes) who gets consistently denied and suspects the reason may be because of a protected attribute, should be able to request a reason from the service provider. In yet another example, a mobile phone application allowing the user to test for melanoma by taking a picture should allow the user to report false positives and false negatives following consultation with a physician should such consultation prove the application misdiagnosed the user, which would allow the developers to diagnose the root cause, which may include information covered by protected attributes. A researcher or independent programmer should also be able to report bugs or potential fairness issues for any ML-based system and receive a reward if that bug report is found to reveal flaws in the algorithm or training set related to fairness.

In this report, we shared some background information on the issues of ML fairness and existing standards and regulations in software. Although ML systems are becoming increasingly ubiquitous, their complexity is often beyond what was considered by prior privacy and cybersecurity standards and laws. Therefore, we expect that what norms should specify will be an ongoing conversation requiring both industry collaboration through standardization and new regulations. We recommend a complementary approach to fairness by creating a fairness standard via a leading industry standard-setting body to include audits and mechanisms for bug reporting by users and a regulation-based approach wherein generalizable tests such as group-fairness criteria are encoded and enforced by national regulations.

Link:
Fairness in machine learning: Regulation or standards? | Brookings - Brookings Institution

In a groundbreaking study led by UC Santa Cruzs Professor of Computer Science and Engineering, C. Sesh Seshadhri, and co-author Nicolas Menand, the effectiveness of the widely used AUC metric in measuring link prediction performance is being questioned. The researchers propose a new metric, VCMPR, which they claim offers a more accurate measure of performance in machine learning (ML) algorithms.

The Area Under the Curve (AUC) metric has been a standard tool for evaluating the performance of machine learning algorithms in link prediction tasks. However, the new research suggests that AUC fails to address the fundamental mathematical limitations of low-dimensional embeddings for link predictions. This inadequacy leads to inaccurate performance measurements, thereby affecting the reliability of decisions made based on these measurements.

The study introduces a novel metric known as VCMPR, which promises to better capture the limitations of machine learning algorithms. Upon testing leading ML algorithms using VCMPR, the researchers found that these methods performed significantly worse than what is generally indicated in popular literature. This revelation has serious implications for the credibility of decision-making in ML, as it suggests that a flawed system used to measure performance could lead to incorrect decisions about which algorithms to use in practical applications.

The findings of this research have considerable consequences for the field of machine learning. The introduction of VCMPR throws a spanner in the works, challenging the status quo and pushing ML researchers to rethink their performance measurement practices. By highlighting the shortcomings of the AUC metric, the study underscores the importance of accurate and comprehensive performance measurement tools for making trustworthy decisions in machine learning.

While the research is undoubtedly groundbreaking, its recommendations are yet to be universally accepted. The machine learning community is currently grappling with the implications of this study, with some experts supporting the switch to VCMPR, while others are apprehensive about abandoning the traditional AUC metric. However, the conversation sparked by this research is crucial, as it pushes the field towards more accurate and reliable performance measurement practices.

This research by UC Santa Cruz signifies a potential paradigm shift in the field of machine learning. By challenging the effectiveness of the AUC metric and proposing a more accurate alternative, it highlights the need for constant innovation and scrutiny in the pursuit of more reliable and trustworthy machine learning practices. Whether or not VCMPR will replace AUC as the standard performance measurement tool is yet to be seen. However, one thing is certain: this research opens up a new chapter in the ongoing endeavor to enhance the accuracy, reliability, and practicality of machine learning applications.

Read the rest here:
Groundbreaking Study Questions AUC Metric in Link Prediction Performance - Medriva

Comcast is one of the early edge network partners for a new AI- and 5G-as-a-service offering operated by Vapor IO that will eventually become available nationwide.

Vapor IO's new micro-cloud offering, called Zero Gap AI, is underpinned by the Nvidia MGX platform with the Nvidia GH200 Grace Hopper Superchip and Supermicro's AI-optimized servers.

In use cases focused on enterprise and smart city applications, Zero Gap AI aims to deliver private 5G and GPU-based "micro-clouds" to locations such as retail storefronts, factory floors and city intersections, the company said.

Zero Gap AI initially will be available as a pilot offering in two markets Atlanta and Chicago and will tie into Comcast's network infrastructure there. That activity builds on integration work that Vapor IO and Comcast announced last year focused on tests of low-latency edge services and applications.

"Our low-latency, high-bandwidth network and connectivity services unlock a world of applications for large and small businesses, residences and mobile customers," Comcast Chief Network Officer Elad Nafshi said in a statement. "We're continuously innovating and collaborating with partners like Vapor IO to identify new ways to leverage our network, and Zero Gap AI is a unique opportunity to expand the limits of what we can do together with edge computing services."

Plans to expand

Zero Gap AI will also expand into dozens of other US markets that have access to Vapor IO's "Kinetic Grid" infrastructure, including Dallas, Las Vegas and Seattle.

With the potential to connect more cable headends to the Vapor IO fabric, Vapor IO believes its cable ambitions will extend well beyond Comcast.

"We think this is going to be a big benefit to the cable operators," Vapor IO CEO and founder Cole Crawford said, noting that the company also has "enjoyed a good relationship with CableLabs for several years."

Vapor IO's launch of Zero Gap AI follows the company's buildout of a footprint of network backbone and individual points of presence across 36 US markets (via Vapor IO's own facilities or those of its colocation partners) to put that capability alongside the radio access network (RAN).

Bringing AI to the edge

The broader idea is to bring the kind of as-a-service capability that companies expect from a cloud company without the complexities of having to figure out how to build a wireless network and build out the elements for AI and machine learning. And the inferencing capabilities of AI can then be deployed at the edge instead of on-premises to support enterprises that are operating in multiple markets or wide-scale smart city deployments. Once the AI model is trained and set up, the AI inferencing component is used to make predictions and solve tasks.

"Generative AI costs you money. Inferencing makes you money," Crawford said. "And the inferencing action, I think, is where the industry will make a lot of money."

Vapor IO didn't announce any early, specific deployments of Zero Gap AI, but the company did spell out several potential use cases. A retailer, for instance, could use it for AI-assisted automated checkouts without having to deploy expensive AI gear at each store.

'Computer vision' a driving force

In another example, a city could use Zero Gap AI for "computer vision" services to support a pedestrian safety system across hundreds of intersections without having to deploy AI equipment at every corner. Additionally, construction sites could use computer vision with AI inferencing to determine if everyone working there is wearing a hardhat.

In a more specific example, the City of Las Vegas is working on computer vision inferencing capabilities that would take advantage of thousands of cameras deployed around the city for use in areas such as public safety, law enforcement and traffic management.

"All applications we are seeing demand for today use computer vision in some form or another. Computer vision certainly is the biggest driver," Matt Trifiro, Vapor IO's chief marketing officer, said.

"We think large vision models as a basis for how to do inferencing are going to generate more revenue for the industry than large language models, I think, over the next five years," added Crawford.

Read the original:
Vapor IO connects with Comcast on AI-as-a-service offering - Light Reading

Contamination level and representativeness of subsampled areas

Analysis of the procedural blank sample indicated that contamination from the experiment environment was low. The detailed results are summarized in Table S1.

The positive control sample was used to examine the representativeness of the nine subsampled regions, determined by the ratio of the estimated mass of particles on the filter to the initial 0.05mg of nylon microspheres. It was found that the method slightly overestimated the mass of nylon microspheres, with an obtained ratio of 1.270.06. (Detailed information is displayed in Fig. S1). In an endeavour to optimize the number of regions for our spectral imaging model, we systematically analyzed the ratio between estimated and actual values across varying numbers of regions.

Generally, for identification of MP using the O-PTIR technique, there are three commonly used methods, i.e., DFIR imaging, point spectra measurements, and HSI. However, each of these methods brings some challenges: for example, DFIR imaging is fast yet provides unreliable results while HSI and point spectra measurements allow for accurate results, but they are time-consuming for data collection. With the QCL system integrated within the O-PTIR microscope, the microscope can generate a single frequency IR image of a 480m640m (spatial resolution: 2m) area of a filter in approximately 3min and 20s. When an appropriate wavenumber and a threshold value are selected, the generated image shows the majority of MP particles while ruling out most non-MP particles. With this method however, careful selection of a suitable wavenumber and a threshold value for MP particles are necessary; multiple threshold values might be needed in case of interference from the complex non-MP particles. In our study, the discrimination between MPs and non-MP particles based on single-wavenumber images proved to be unfeasible, as illustrated in Fig. S2.

The second method commonly used for MP identification is point spectra measurements. After particles are observed in the mIRage microscope, point spectra could be collected for each particle and compared against parent plastic to achieve chemical identification of the particles. This method presented two challenges: (1) When using visible light for particle location under the microscope, non-MP particles were inevitably included as spectral acquisition targets, thus adding to the analysis time. (2) For an individual particle, the O-PTIR spectra could vary significantly across different spots of the particle (see example in Fig.1). This necessitates the collection of spectra from multiple areas of the particle to enhance the reliability of identification results. Consequently, the analysis time will be multiplied. For example, it takes 25s to obtain a spectrum (a total of 5 scans acquired for each single spectrum), so if there are 100 particles from the regions of interest on the filter and three spectra are required for each particle, the total analysis time needed is at least 2h. This estimation only accounts for the raw data acquisition, excluding additional durations associated with manual adjustments such as repositioning the objective or refocusing. In light of this, such an approach becomes exceedingly time-intensive, especially when a vast number of particles are in play.

Two spots of the particle encircled in a red dashed line (A) selected for point spectra collection and (B) the corresponding O-PTIR spectra of the two spots. a.u. is arbitrary units. The scale bar is 20m.

HSI was the third method employed for MP identification. HSI generates an image where each pixel contains a full spectrum. Hence, it is a reliable method for MP identification. However, this reliability comes at the cost of drastically longer data collection time, which makes HSI impractical for routine MP analysis. For example, capturing a hyperspectral image for a 480m640m area (spatial resolution of 2m and spectral resolution of 2cm1, from a spectral range of 7691801cm1) requires almost two weeks.

In response to the challenges mentioned above, we have developed a reliable MP detection framework with an improved speed that is suitable for detecting a large quantity of nylon MPs. It can collect spectral data from nine areas (the size of each area is480m640m) of a filter (at a spatial resolution of 2m) within just approximately 2h. Powered by machine learning, the reliability of this framework is not compromised in response to reduced data collection time.

In order to effectively utilize DFIR imaging for high-throughput analysis of MPs, it is crucial to carefully select specific wavenumbers that provide the greatest discriminatory power between MP and non-MP particles. Making incorrect choices in wavenumber selection can directly impact the accuracy of identification. Acquiring too many wavenumbers increases measurement time, resulting in decreased throughput. For instance, adding just one more wavenumber can lead to an approximate 30-min increase in the time required for our proposed MP detection framework to collect data from a single filter. To identify the important wavenumbers and determine the optimal number of such wavenumbers, a database collected from bulk nylon plastic was assembled, containing 1038 spectra of MP and 1052 spectra of non-MP.

We found several types of non-MP particles in our dataset. Figure2 displays the spectra of two non-MP classes (type I non-MP and type II non-MP), along with the mean spectrum of MP, enabling a comparison. Upon initial inspection, type I non-MP exhibits a prominent sharp peak in the 17001800cm1 spectral range, while type II non-MP displays a broad peak in the 10001200cm1 spectral range. In contrast, the apparent characteristic peaks of MPs are two consecutive sharp peaks in the 15001650cm1 range.

Mean spectra for nylon MP class and two non-MP types from the database constructed, following standard normal variate (SNV) to minimize the multiplicative effects.

Two thirds of the spectra from each class were randomly selected as the training dataset for model development, and the remaining samples formed the test dataset. Based on the obtained results, the model utilizing the full wavenumber spectrum yields a correction accuracy rate of 85.31% (see Table 2). The confusion matrix of the SVM-Full wavenumber model (Fig.3A) implies that there are 8 point spectra of MPs wrongly classified as non-MPs and 97 of non-MPs mistakenly assigned as MP.

Confusion matrix showing classification accuracy for the test set of SVM-Full model using full spectral variables (A) and SVM-Four model (B).

Subsequently, the coefficient based feature importance for the full wavenumber model (Fig.4) was plotted to visualize the contribution of individual spectral variables. According to Fig.4, we could choose the important wavenumbers to our dataset based on the feature importance. The higher feature importance signifies stronger discriminative capability. Based on the analysis of the coefficients of the SVM-Full wavenumber model, wavenumbers 1711cm1, 1635cm1, 1541cm1, and 1077cm1 (indicated in Fig.4) showed the feature importance, hence, were selected as important wavenumbers for distinguishing between MPs and non-MPs. As seen from Table 2, the model optimized with these four wavenumbers demonstrates an enhanced correction rate of 91.33%. Meanwhile, the SVM-Four wavenumbers model (Fig.3B) resulted in 34 point spectra of MPs wrongly classified as non-MPs and 28 of non-MPs mistakenly assigned as MP, which shows it is a balanced model for classification tasks. The SVM-Four wavenumbers model appears to outperform the SVM-Full wavenumber model in terms of specificity, CCR, and MCC, suggesting that it is a better model for this classification task. However, the SVM-Full wavenumber model has a higher sensitivity, making it better at identifying true positive cases.

The coefficients (or weights) of the SVM model, which indicate the importance of each feature (wavelength), are then plotted. Four wavenumbers which has relatively higher feature importance than other are marked above the curves (i.e., 1711cm1, 1635cm1, 1541cm1, and 1077cm1).

After the selection of the four important wavenumbers, DFIR images were obtained at the important wavenumbers from the nine subsampled regions of the filter. Particle identification could be performed through visual inspection of these DFIR images. For instance, Fig.5A shows an optical image of a small region of a filter with a particle in the centre, and Fig.5B shows chemical images of that region based on the intensity of 1711cm1, 1635cm1, 1541cm1, and 1077cm1 bands. The absorbance intensity of each chemical image was normalized to the same range. The particle in this region exhibits high signal intensity at 1635cm1 and 1541cm1, while showing weak signal intensity at 1711cm1 and 1077cm1, indicating that it is a MP particle. On the other hand, non-MP particles would show weak signal intensity at 1635cm1 and 1541cm1, while showing strong signal intensity at 1711cm1 and/or 1077cm1 (See Figs. 6A,B for an example of non-MP particles).

An optical image of an area of a prepared filter, with a MP particle in the center of the image (A), single frequency images of that area using 1711cm1, 1635cm1, 1541cm1 and 1077cm1 band intensity, with the absorbance intensity of each chemical image normalized to the same range (B), support vector machine (SVM) prediction results of the particles in this area (C), and normalized O-PTIR spectra of the particle and the bulk plastic (D). The +1 in (C) indicates where the spectrum of the particle in (D) was collected. The scale bar is 20m.

An optical image of an area of a prepared filter, with a non-MP particle in the center of the image (A), single frequency images of that area using 1711cm1, 1635cm1, 1541cm1 and 1077cm1 band intensity, with the absorbance intensity of each chemical image normalized to the same range (B), support vector machine (SVM) prediction results of the particles in this area (C), and normalized O-PTIR spectra of the particle and the bulk plastic. The +1 in (C) indicates where the spectrum of the particle in (D) was collected. The scale bar is 20m.

However, for accurate particle identification, visual inspection is not advisable due to low accuracy. Meanwhile, application of SVM-Full model requires a huge amount of time in the collection of point spectra from all particles. Therefore, an SVM-Four wavenumbers model was trained from the four important wavenumbers to predict each particle accurately. Spectral data at the four important wavenumbers were extracted from the same database used for the SVM-Full wavenumber model. The trained SVM model on the selected four wavenumbers demonstrated good performance, evidenced by a high CCR, MCC, sensitivity and specificity (Table 2).

After applying the SVM classifier to the particle in Fig.5A, each pixel of the particle was labelled as either MP (red) or non-MP (blue), providing an intuitive and accurate identification result. Figure5C displays the SVM prediction results for one example area. As can be seen, most pixels in the particle have been labelled as MP, with a small portion labelled as non-MP. The result for a particle was determined by the majority vote of the labels of all pixels within the particle. Thereby this particle was identified as a MP particle. This was further confirmed by the full spectrum of this particle (Fig.5D). Also, by applying the SVM classifier to the particle in Fig.6A, the particle was predicted to be a non-MP particle (Fig.6C). Figure6D presents a spectrum of this particle, which validates the predicted outcome.

Our developed SVM model offers several distinct advantages over the traditional correlation-based method for MP identification. Firstly, the SVM model only requires four wavenumbers as input, significantly reducing the complexity of data collection compared to the correlation-based approach, which involves obtaining spectra from each particle and calculating correlation coefficients. This efficiency translates into a substantial time-saving advantage. Therefore, the developed method is particularly useful when dealing with a large number of particles on the filter. Secondly, the correlation-based method often relies on establishing a threshold for identification, introducing a subjective element into the process. In contrast, the SVM model automates the assignment of particles to MP or non-MP categories, contributing to a more consistent and reliable MP identification process. Last but not least, once essential wavenumbers are identified and a simplified model is developed, the SVM approach can be extended to identify a range of polymers. This versatility is a significant advantage, enabling the model to adapt to various MP compositions beyond the scope of the original correlation-based method.

Using the novel identification procedure developed, it was possible to investigate the effectiveness of several sample pre-processing steps in a more representative and less biased and efficient way. To this end, high-temperature filtration and alcohol pretreatment were chosen as methods for reducing non-MP. The performance of these two treatments was evaluated separately, including the analysis of the spectra and DFIR images at four selected wavenumbers. The evaluation included an assessment of their impact on the spectra of MP and their effectiveness in removing non-MP. To assess the effectiveness of particle removal, the MP particle/all particle ratio (MP/All) detected by four wavenumbers SVM model was used. A treatment was considered effective if it significantly increased this ratio.

By boiling the nylon bulk, MP particles were released. The released particles were subsequently enriched on the filters through high-temperature filtration and room-temperature filtration, respectively. The mean spectrum of MP from high-temperature filtration, the mean spectrum of MP from room-temperature filtration, and the mean spectrum of nylon bulk were plotted together for comparison (Fig. S3). Results showed that when the mean spectrum of nylon bulk was compared to the mean spectra of MP (regardless of the filtration temperature), no consistent peak shift was found. When the mean spectrum of MP from high-temperature filtration and the mean spectrum of MP from room-temperature filtration were compared, no consistent peak shift was found either. These findings demonstrate that exposure to high temperatures reaching water boiling point will not impact the spectral profiles of MPs when compared to the original bulk plastic.

After the thermal degradation of nylon bulk, the particles released were captured on filters through high-temperature filtration and room-temperature filtration, respectively. Using our developed SVM classifier, particles in the nine subsampled regions of the filter were counted and subsequently the ratio MP/All was calculated. The MP/All ratio from the room-temperature filtration was 0.0900.012, and from the high-temperature filtration was 0.080.012, respectively. The normal t-test results indicated that the effectiveness of high-temperature filtration in removing non-MP was not evident.

Gerhard et al.18 reported that slip agents (such as fatty acid and fatty acid esters) of plastic products are released concomitantly with the release of MP particles, and these slip agents might be dissolved in hot water and washed away during the filtration process. In light of this, our results suggest that the nylon bulk used in our study might have just a small amount fatty acid or their esters. Indeed, Hansen et al.19 reported that as additives in plastics, the amount of slip agents could be as low as 0.1%, and the removal of a small amount of additives from MP samples might not statistically significant. Furthermore, based on observations of the prepared filters, we did not see a thin residue on the room-temperature filter, which was observed by Gerhard et al.18 who confirmed that most part of the thin residue in their experiment was identified as additives. This supports that the amounts of hot water-rinseable additives in our samples were low, however this would generally be sample specific.

After the degradation of nylon bulk in boiling water, the particles released were retained on filters. An alcohol treatment was subsequently applied to the filters to reduce non-MP particles. The mean spectra of MP before and after an alcohol treatment and the mean spectrum of nylon bulk were plotted together and compared (Fig. S4). Results revealed that when the mean spectrum of nylon bulk was compared to the mean spectra of MP (regardless of the alcohol treatment), no consistent peak shift was observed. When the mean spectra of MP before and after the alcohol treatment were compared, no consistent peak shift was observed either.

To further explore the effects of alcohol treatment on released particles, this paragraph focuses on spectral changes of individual particles. The spectral data of individual particles was baseline corrected, smoothed, and normalized to between 0 and 1 prior to comparison. Figure7 shows spectra as well as optical images of 4 MP particles before and after an alcohol rinse. For all four particles presented, peak shifts for signature bands of MP in the range of 769cm1 to 1801cm1 were not observed. Particle 1 has a peak at 1741cm1 before the alcohol treatment; this is a peak that has been assigned to the formation of carbonyl groups during polyamide 66 photo-20 and thermal-oxidation21, which implicates a pathway of oxidation in hot water for the particles during high temperature treatment (filtration at 70C). However, the reduction in signal intensity of this peak after the alcohol treatment might indicate that the alcohol treatment could remove some of the oxidized substances. The spectrum of particle 2 has two new peaks at 1007cm1 and 1029cm1, respectively, after exposure to alcohol, which was possibly due to alcohol residue, as these two new peaks correspond to C=O stretching bonds of alcohol22. No introduction or disappearance of the peak was observed in the spectra of particle 3 and particle 4. By observing the optical images of these MP particles, it can be concluded that alcohol treatment did not have an effect on their morphology.

Optical images of nylon MP particles 1, 2, 3, 4, with the particles circled and marked with numbers. The scale bar is 10m.

Figure8 shows spectra as well as optical images of 4 non-MP particles before and after an alcohol rinse. Particle 1 and particle 2 appear to be yellowish to brownish. These types of non-MPs are easy to be discriminated against from MPs based on visual observation of optical images, as most of the MP particles in our experiments are whitish, similar to the color of their bulk plastic samples. However, judgement based on color is not always correct. Subsequent spectral analyses confirmed that particle 1 and particle 2 are not MP. After the alcohol treatment, most parts of these two particles were washed away, leaving black remnants on the filter. Though the elimination was not complete, it proved that alcohol could remove non-MP particles. Particle 3 is whitish with a glossy surface, and it is a chlorinated polyethylene particle. After the alcohol treatment, particles with a spectrum similar as chlorinated polyethylene (We do not have any appliances containing polyethylene) remained where it had been, and the spectrum was not changed substantially. The glossiness of the particle was reduced; however, this indicates that alcohol treatment could not remove this type of contaminant. Particle 4 is a white particle, and it is covered by a brown, lumpy object on the upper left. The noise spectrum cannot be identified by the database with high certainty. After the alcohol treatment, it appeared dull grey, and its spectrum looked like that of nylon showing five signature peaks (1633cm1, 1533cm1, 1464cm1, 1416cm1, 1370cm1). This implies that the alcohol might be able to remove some contaminants, such as additives, which cover the surface of the MP particle. Li et al.23 have reported the same finding that alcohol could wash away some additives attached to the surface of MP particles. The above experiments prove that an alcohol treatment could remove some particle contaminants and wash away some impurities covering MP particles.

Optical images of non-MP particles 1, 2, 3, 4, with the particles circled and marked with numbers. The scale bar is 10m.

To further explore the significance of alcohol treatment, the developed SVM classifier was used to count the particles in the nine subsampled regions of the filter, based on which the MP/All was calculated. The MP/All ratio before the alcohol treatment was 0.1290.129; and after the alcohol treatment was 0.2860.207, respectively. The paired t-test of the data indicates that an alcohol treatment of the same areas of the filter significantly increases the MP/All (p<0.05). In summary, alcohol treatment was significantly effective in reducing non-MP contaminants.

The proposed MP detection framework was specifically adapted for application to detect MPs released from nylon teabags. However, it's important to note that not the entire framework was employed in this context. Rather, a selective application was implemented, excluding the components based on DFIR imaging and the SVM-Four wavenumber model. After steeping teabags in hot water, MPs were released and collected on a filter through filtration at room temperature. This filter was rinsed with alcohol and air-dried in the fume hood prior to O-PTIR data collection. The contaminants from the teabag are not the same as those originating from reference nylon bulk plastics. For example, teabags might have some contaminants from tea residuals, as noted by Xu et al.13. Particles released from nylon teabags were identified through point spectra measurements due to the relatively low particle count (i.e., <5 particles) observed in the subsampled regions of the filter (see Conventional MP identification).

Characterization of MP particles released from teabags was carried out using the MATLAB image processing toolbox function regionprops, which calculates properties of each particle including area, length (length of the major axis of the fitted ellipse), width (length of the minor axis of the fitted ellipse), and circularity. In Fig.9, we present four optical images toshow nylon MP particles, which have been released from three nylon teabags; they are circled and marked with numbers. To provide a comprehensive analytical context, the spectra of three key references are plotted alongside: a nylon reference sphere, a sample of nylon in bulk form, and the material of the nylon teabag itself. This juxtaposition allows for a direct comparison between the spectra of the isolated particles and these standard nylon references, this contributes to a more detailed understanding of the appearance as well as the spectral properties of the particles.

Optical images of nylon MP particles 1, 2, 3, 4 released from nylon teabag, with the particles circled and marked with numbers.

The average quantity of MP in the nine subsampled regions of the filter was 8.71.2. Extrapolating to the whole filter, we would estimate 31943.7 MP particles released from steeping three teabags, or approximately 106.314.6 MP particles were released from one teabag. The particle counts/quantities of MPs released from teabags previously reported are listed in Table 1. Our reported count is comparable to that reported by Ouyang et al.9, who found 393 MPs using FTIR-based particle-based analysis, although their brewing time was much longer than ours (1h vs 5min). Regarding Hernandez et al.7, their brewing temperature and time are very similar to ours. Nevertheless, as they did not conduct particle-based analysis, their results were overestimated8. The detection limits of O-PTIR spectroscopy and Raman spectroscopy are similar, with O-PTIR spectroscopy being around 500nm and Raman spectroscopy being around 1m. Based on this, we were surprised to find that the number of MPs we detected was one to two orders of magnitude lower than the 5,800 20,400 per teabag (brewed at 95C for 5min) reported by Busse et al.8 using Raman spectroscopy. Busse et al.8 conducted particle-based analysis, indicating that their results should be considered reliable. However, it is important to notethat their use of Raman spectroscopy may have led tomisidentification of non-MP particles as MPs in an unexpected way. To illustrate, Busse et al.8 identified and counted polyethylene (PE) particles in the teabag leachate. However, these PE particles could also be behenamide (CH3(CH2)20CONH2), which is a typical slip additive widely used in PE plastic. Behenamide exhibits a high level of spectral similarity with PE in Raman spectroscopy, up to 90%, mainly due to the strong Raman signal associated with its saturated alkyl chains (i.e., (CH)) and relatively weak Raman signals from carbonyl and amine groups23. The observed disparities between our results and those of Busse et al.8 could also potentially be attributed to the use of different types of teabags.The counts/quantities reported by other studies listed are expressed in the mass of MPs released per teabag11,12, or the number of MP particles per kg of teabags10. Therefore, direct comparisons with these studies are not possible in our paper.Subsequently, the length, width, area, and circularity of each particle were measured and calculated using the MATLAB function regionprops. Figure10A shows the surface area of the MP particles. Except for the two MP particles with the smallest (100m2) and largest (680m2) surface area, the majority of the remaining particles have surface areas ranging from 150 to 550m2. Figure10B shows the distribution of the length of MP particles. As can be seen, the maximum length is 40m and the minimum length is 16m, while most MP particles have a length ranging from 18 to 28m. Figure10C displays the width of the MP particles. As seen from the graph, the smallest width is 9m, while the largest width is 30m. The majority of the MPs have a width range between 12 and 24m. Figure10D shows the circularity of the MP particles. Among all the MP particles, only 4 have a low circularity (0.10.4), while most of the MP particles have circularity ranging from 0.65 to 0.95. Circularity is a measure of how closely a shape resembles a perfect circle. Circularity values near 1 represent perfect circles, while values close to 0 indicate shapes that deviate significantly from circularity. Based on the literature, particles that are more circular in shape are found to be less toxic, while those that deviate from a circular shape, manifesting more stretched or fiber-like, are associated with a higher level of toxicity24.

Length (A), width (B), area (C) and circularity (D) of MP particles released from steeping a single teabag.

View original post here:
Machine learning driven methodology for enhanced nylon microplastic detection and characterization | Scientific Reports - Nature.com