For any tagset designed for the annotation of texts in a given language, the guidelines do not impose any particular set of choices to be used in distinguishing and representing grammatical categories. But it is important that the tagset should be mappable (if possible automatically) on to a set of attribute-value pairs in conformity with the guidelines. This includes the possibility (indeed the probability) that the annotator will need to define optional values other than the special extensions.
This mapping will have the additional value that it will enable the annotator to transfer the information in a morphosyntactically-tagged corpus to the morphosyntactic component of a lexicon (e.g. in order to record frequencies of word-tag pairs). It will also enable a lexicon of the given language to be used as a major input to automatic tagging.
To aid this mapping, and to test out its efficacy, we suggest that an Intermediate Tagset can be used as a language-neutral representation of a set of attribute-value pairs, based on the word categorisation. This can act as an intermediate stage of mapping between the tags assigned to textwords in corpus annotation and the labels assigned to words in a lexicon. Another important function of this Intermediate Tagset is to act as a basis for interchange between different local tagsets for particular corpora and particular languages.
A convenient linear method of representation is arrived at as follows:
|N = noun||AV = adverb||I = interjection|
|V = verb||AP = adposition||U = unique/unassigned|
|AJ = adjective||C = conjunction||R = residual|
|PD = pronoun/determiner||NU = numeral||PU = punctuation|
|AT = article|
Wherever an attribute is inapplicable to a given word in a given tagset, the value 0 fills that attribute's place in the string of digits. (See further, for the use of 0, the section on underspecification). When the 0s occur in final position, without any non-zero digits following, they could be omitted without loss of information. Thus a comparative general adjective could simply be represented: AJ2. However, for clarity, the 0s should be added.
There may be cases where a category needed for tagging in a specific language (given current limitations of automatic tagging) cuts across two or more values in the optional categories of the guidelines, and may even cut across different attributes as well. It is necessary to define what this value means by using the OR operator ( | ), and brackets to identify the arguments of this operator. Another operator we can use is the negative operator, signalled by the minus (-), so that -4 means ``all values of this attribute except the 4th''.
A good example is the base form of the English verb. The finite base form in English can be specified by using a disjunction ``[finite indicative present tense [plural or [first person or second person] singular] or imperative or subjunctive]''. This is spelled out, using the intermediate tagset, as follows:
Even this leaves out the non-finite use of the base form, as an infinitive. This example, awkward as it is, has an explanatory value: the relation between tagsets and a language-neutral representation can be very indirect. Although such cases as this are unusual, they show that the mapping between a lexicon and a tagged corpus is not always an easy one to automate.
To illustrate the method of converting a tagset into this type of language-neutral labelling, we present in Appendix A a rendering into an Intermediate Tagset of a tagset for English and in Appendix B of a set of dictionary codes for Italian; the former are based on the English implementation of the lexicon guidelines and the latter on the codes of the DMI (Calzolari et al. 1980). (For English, with its simple morphology, we find the most complex interrelation between the morphosyntactic guidelines and the requirements of a particular language. With other languages, the mapping from the language-specific tagset to the Intermediate tagset is likely to be more straightforward.)