Goals, Applications and Examples of Natural Language Processing(NLP)

GOALS OF NLP

The goal of natural language processing is to specify a language comprehension and production theory to such a level of detail that a person is able to write a computer program which can understand and produce natural language. The basic goal of NLP is to accomplish human like language processing. The choice of word “processing” is very deliberate and should not be replaced with “understanding”. For although the field of NLP was originally referred to as Natural Language Understanding (NLU), that goal has not yet been accomplished. A full NLU system would be able to:

®      Paraphrase an input text.

®   Translate the text into another language.

®   Answer questions about the contents of the text.

®   Draw inferences from the text.

While NLP has made serious inroads into accomplishing goals from first to third, the fact that NLP system can not, of themselves, draw inferences from text, NLU still remains the goal of NLP. Also there are some practical applications of NLP. An NLP-based IR system has the goal of providing more precise, complete information in response to a user’s real information need. The goal of the NLP system is to represent the true meaning and intent of the user’s query, which can be expressed as naturally in everyday language.

APPLICATIONS OF NLP

NLP lie in a number of disciplines like computer and information sciences, linguistics, mathematics, electrical and electronic engineering, artificial intelligence and robotics, psychology etc. Applications of NLP include a number of fields of studies such as machine translation, natural language text processing, summarization, user interfaces multilingual and Gross language information retrieval (CLIR), speech recognition, artificial intelligence and expert system. Research on NLP is regularly published in a number of conferences such as the annual proceedings of ACL (Association of Computational Linguistics) and its European counter part EACL, biennial proceedings of the Message Understanding Conferences (MUCS), Text Retrieval Conferences (TRECS) and ACM-SIGIR (Association of Computing Machinery-Special Interest Group on Information Retrieval) conferences.

As natural language processing technology matures, it is increasingly being used to support other computer applications. Such use naturally falls into two areas, one in which linguistic analysis merely serves as an interface to the primary program and the second one in which natural language considerations are central to the application. Natural language interfaces into a request in a formal database query language, and the program then proceeds as it would without the use of natural language processing techniques. The design of question answering systems is similar to that for interfaces to database management systems. One difference however, is that the knowledge base supporting the question answering system does not have the structure of a database. Similarly in message understanding systems, a fairly complete linguistic analysis may be required but the messages are relatively short and the domain is often limited. Also some more application areas include information and text categorization. In both applications, natural language processing imposes a linguistic representation on each document being considered. In text categorization a collection of documents is inspected and all documents are grouped into several categories based on the characteristics of the linguistic representations of the documents. In information filtering documents satisfying some criterion are singled out from a collection.

Discourse Knowledge

While syntax and semantics work with sentence-length units, the discourse level of NLP works with units of text longer than a sentence i.e. it does not interpret multi-sentence texts as just concatenated sentences, each of which can be interpreted singly. Discourse focuses on the properties of the text as a whole that convey meaning by making connections between component sentences. Several types of discourse processing can occur at this level like anaphora resolution and discourse/text structure recognition. Anaphora resolution is the replacing of words such as pronouns which are semantically vacant with the appropriate entity to which they refer. For example, newspaper articles can be deconstructed into discourse components such as: lead, main story, previous events, evaluation etc. A discourse is a sequence of sentences. Discourse has structure much like sentences do. Understanding discourse structure is extremely important for dialog system.

For example: The dialog may be

When does the bus to Bhubaneswar leave?

There is one at 10 a.m. and one at 1 p.m.

Give me two tickets for the earlier one, please.

The problems with discourse analysis may be non-sentential utterances, cross-sentential anaphora.

Pragmatic Knowledge

This level is concerned with the purposeful use of language in situations and utilizes context over and above the contents of the text for understanding. The goal is to explain how extra meaning is read into texts without actually being encoded in them. This requires much world knowledge including the understanding of intentions, plans and goals. Some NLP applications may utilize knowledge bases and inferencing modules. Pragmatic is the study of how more gets communicated than is said. Speech acts in the pragmatic processing is the illocutionary force, the communicative force of an utterance, resulting from the function associated with it. For example: Suppose the sentence is I will see you later.

Prediction: I predict that I will see you later.

Promise: I promise that I will see you later.

Warning: I warn you that I will see you later.

EXAMPLES OF SOME NLP SYSTEMS

In early 1950s, few NLP systems had been developed. Except the theoretical developments many practical systems were developed to demonstrate the effectiveness of particular principles. Weizenbaum’s ELIZA was built to replicate the conversation between a psychologist and a patient; simply by permuting the user input. Winograd’s SHRDLU simulated a robot that manipulated blocks on a table top. Also LUNAR was developed by Woods as an interface system to a database. In the late 1970’s, McKeown’s discourse planner TEXT and McDonald’s response generator MUMMBLE used theoretical predicates to produce declarative descriptions in the form of short texts, usually paragraphs. Some of the earliest NLP systems are described below.