Loading [a11y]/accessibility-menu.js
3168-2024 - IEEE Standard for Robustness Evaluation Test Methods for a Natural Language Processing Service That Uses Machine Learning | IEEE Standard | IEEE Xplore

3168-2024 - IEEE Standard for Robustness Evaluation Test Methods for a Natural Language Processing Service That Uses Machine Learning

Most Recent
Status: active - Approved
Versions

Abstract:

The natural language processing (NLP) services using machine learning have rich applications in solving various tasks and have been widely deployed and used, usually acce...Show More
Scope:This standard specifies test methods for evaluating the robustness of a natural language processing (NLP) service that uses machine learning. Models of NLP generally feat...Show More
Purpose:The purpose of the standard is to provide test methods for evaluating the robustness of an NLP service. Test methods are used by service developers, service providers, an...Show More

Abstract:

The natural language processing (NLP) services using machine learning have rich applications in solving various tasks and have been widely deployed and used, usually accessible by application programming interface (API) calls. The robustness of the NLP services is challenged by various well-known general corruptions and adversarial attacks. Inadvertent or random deletion, addition, or repetition of characters or words are examples of general corruptions. Adversarial characters, words, or sentence samples are generated by adversarial attacks, causing the models underpinning the NLP services to produce incorrect results. A method for quantitatively evaluating ...
Scope:
This standard specifies test methods for evaluating the robustness of a natural language processing (NLP) service that uses machine learning. Models of NLP generally feature an input space being discrete and an output space being almost infinite in some tasks. The robustness of the NLP service is affected by various perturbations including adversarial attacks. A methodology to categorize the perturbations, and test cases for evaluating the robustness of an NLP service against different perturbation categories is specified. Metrics for robustness evaluation of an NLP service are defined. NLP use cases and corresponding applicable test methods are also describ...
Purpose:
The purpose of the standard is to provide test methods for evaluating the robustness of an NLP service. Test methods are used by service developers, service providers, and service users to determine the robustness of an NLP service.
Date of Publication: 09 August 2024
Electronic ISBN:979-8-8557-0910-0
Persistent Link: https://ieeexplore.ieee.org/servlet/opac?punumber=10631889

Subscription Options - Standards

References

References is not available for this document.