Guide

Background

Many people believe that it makes sense to process data on a semantic level, i.e. on the basis of a shared structured vocabulary. This obviously helps stakeholders who are not in direct contact with the data, but it also makes it easier to integrate data from different sources, as the shared vocabulary allows for a common understanding of the data and facilitates data interoperability.

The basis of the shared vocabulary is an ontology which defines classes and relationships between them, providing a structured framework for representing and organizing knowledge. This allows for a more precise and standardized representation of data, which can improve data quality and facilitate data exchange and integration. On the basis of the ontology, data can be integrated in a knowledge graph, which assigns terms of the ontology to the data.

SPARQL is a language for querying and manipulating data in a knowledge graph. It allows users to retrieve and manipulate data stored in the graph, but it is very hard for humans to write and understand, below is a simple example of a SPARQL query for the lead actor of a movie:

PREFIX ex: <http://example.org/>
SELECT ?actor WHERE {
  ?movie ex:title "An Example Movie" .
  ?movie ex:leadActor ?actor .
}

To write this query you need to know that "ex:title" describes titles and "ex:leadActor" is a relation in the ontology. NLP2SPARQL aims to establish a natural language interface, increasingly based on agentic AI, to generate these complicated SPARQL queries, allowing non-experts to interact with semantic data.

Why you are here

To evaluate NLP2SPARQL methods, we need benchmark questions from a real-life context in a variety of domains. Presently very few of such datasets exist.

In this community effort, we ask you to contribute questions that will be later used to benchmark NLP2SPARQL methods. In the Ask tab, we have implemented an agentic AI pipeline that evaluates the proposed questions with respect to an existing ontology and knowledge graph. Ideally we look for questions that are

• gold questions: a conceptually hard question that can be answered on the basis of the current knowledge graph
• competency questions: that an expert would expect to be answered, but the current knowledge graph does not contain the answer
• easy questions: that an expert would expect to be answered, and the current knowledge graph can answer.

The challenge

You can ask as many questions as you like. Every question gets a score from 0 to 100 once its check ends, also a check that could not finish, and you see it at once. A question asked again in the very same words shows "Not assessed": the earlier one holds the score.

The score says how useful, clear and checkable a question is, what new ground it covers in our data, and how demanding it is.

Take part and win: the best question takes first place in the Library.

The helper in the Ask tab tells you the score of your question and its place in the Library. Press the helper to put its words away or bring them back. Press and hold its figure and drag it to move it; it stays inside the window and your browser remembers the place.

Your name stands next to your questions in the Library. If you prefer, switch on "Hide my name in the Library" in the menu of your account. Your name then looks blurred for everybody.

Your account and your email address

Every account needs an email address that you have confirmed once. With a KIT address (also partner.kit.edu and other KIT sub-domains) you confirm it by the link we email you. With another address you enter an invitation code, or send a request (a few words on why are optional); once the research group approves it, you confirm by the emailed link too.