The tagging vs. tagset evaluation exercise

The proposals made in EAGLES (1996g) are intended to be applicable to different European languages and to be independent from a particular NLP application. The ELM series contains typed specifications again not geared towards a specific single application.

In the tests on the interaction between tagging methods, tagsets and tagging results, it was shown that the EAGLES-based ELM-DE specifications indeed allow tagset to be derived which can be practically used for the tagging of German and which leads to acceptable results.

Moreover, part of the history of the tagset could be followed, and the impact of the modifications introduced could be evaluated.

The following tests have been run, all with tagsets derived from ELM-DE:

Tagger evaluation

-- Tests allowing the impact of different statistical tagging methods on the results to be assessed, by comparing the performance of different taggers on the same training and test data, using the same tagset;

Tagset evaluation

-- Tests allowing the impact of tagset modifications on the results to be assessed, by using different versions of a given tagset on the same texts; differences between the versions of the tagset were documented and classified, and the impact of each modification was tested;

Text type evaluation

-- Tests allowing the impact of perceived linguistic differences between training texts and test (or: application) texts on the results to be assessed, by using texts from different text types in training and testing, tagsets and taggers being unchanged otherwise.