Interpretation

 

The following are a few hints and directions for the interpretation of the results produced by the different taggers.

• The percentages of lexical errors and disambiguation errors have been kept separate and should be recorded in relation to each other. Ideally, the best result which could be achieved by a tagger would be a rate of 0% of disambiguation errors. It could then still have a small overall error rate which would then be due to lexical errors. For example, if there is a lexical error rate of 0.5%, the best result which could be achieved is 99.5%.

  When comparing the Tree Tagger and the Xerox HMM Tagger, a number of overlaps in errors can be found. In particular, the distinction between NN and NE, i.e. between common nouns and proper names, is problematic for both types of taggers. Even at the level of individual word forms, there are overlaps: Osthold, das, werden are highly ranked in the error frequency list of both tools.

• Similarly, differences between finite and infinite verb forms, and between participles and finite verb forms are hard to identify, since indeed many such forms are ambiguous (cf. DE er wird verraten). Against this background, a relatively high ranking of disambiguation errors in this area is expected.
• In the ambiguity cases, where proper names are involved, there was a marked difference between the two taggers. The Tree Tagger favours NN over NE, whereas the Xerox HMM Tagger seems to equally distribute the two tags. A further analysis revealed that the TreeTagger in general tends to prefer more frequent tags. This could be traced to a difference between the tagging formula used in the TreeTagger and the standard HMM formula which was previously considered irrelevant. The formula used in the TreeTagger is equivalent to multiplying the HMM formula with the a priori probability of the tag. This explains the preference for more frequent tags. This analysis was confirmed by an experiment in which the TreeTagger was used with the standard HMM formula (everything else unchanged). The modification improved tagging accuracy by 0.5 % .
• The two taggers differ with respect to the amount of errors produced on items such as DM, um, Aber (see ranking in word form error frequency list). Again, it is quite hard to interpret the data in detail. To be able to identify reasons for the differences in the analysis of say um, also the overall amount (absolute number of occurrences) of um, of the two tags in question, KOUI and APPR would be needed, as well as the absolute number of cases of ambiguity (cf. the discussion above about verb forms). A full noise/silence-test would thus be needed.