News Story

SEALS: Supporting the Evaluation of your Semantic Tools

Ning Li, Friday 13 Jul 2012

The three-year SEALs (Semantic Evaluation at Large Scale) project has now come to an end with a very successful final review with the independent expert reviewer stating it is “technically remarkable” and the Commission’s Project Officer marking it as excellent. In addition to the above quota, the expert reviewer said the project have done something the semantic technology community really needed.

SEALS addressed two key challenges: the creation of a lasting reference infrastructure for automated and reproducible evaluation of various semantic technologies, such as ontology matching, semantic searching, semantic Web Services, and semantic storage and reasoning, and the continuous benchmarking of semantic technologies at a large scale via public worldwide evaluation campaigns.

KMi’s significant involvement in this project was to lead the Semantic Web Service (SWS) evaluation research work package as well as organize worldwide evaluation campaigns for SWS tools. The major outcomes of KMi from this project include a reference implementation of services required for the evaluation of SWS discovery tools, the generation of the WSMO-Lite SWS test collection and the successful organization of two evaluation campaigns for SWS discovery tools.