Educational services

 

 

The next layer in the DEPTHS architecture consists of learning support services, namely Semantic Annotation and Indexing Service and Context-aware Learning Services.

Semantic Annotation and Indexing Service is used for the semantic annotation and indexing of documents from publicly accessible repositories of DPs, as well as internal content created by students and teachers (e.g., exchanged messages, and assignments). Since all these resources are used by Context-aware Learning Services to help students in specific learning contexts, DEPTHS needs additional information about them in order to use them in an appropriate way. Specifically, the most important information for DEPTHS regarding these resources is what each specific resource is about and how relevant it is for a specific DP. However, this information is not directly available. Having analyzed many online repositories of DPs (e.g., Portland Pattern Repository, http://c2.com/ppr/; Hilside patterns, http://hillside.net/patterns/; etc.), we found that the majority of them contains mass of documents, but only a few of those documents describe a specific DP. In addition, most of relevant documents describe in detail exactly one DP (we refer to it as dominant pattern throughout the section), but also mention many other DPs (e.g., in the form of links to other documents and related DPs). Moreover, one particular DP is often discussed in many different documents. In order to make this crowd of documents useful for learning purposes, DEPTHS has to analyze how relevant each document is for learning a specific DP, that is, it has to perform semantic annotation and indexing of these documents.
Having tested many of the available tools for semantic annotation, we decided to use the KIM framework (http://www.ontotext.com/kim/index.html), that provides APIs for automatic semantic annotation of documents. Semantic annotation in KIM is based on the PROTON (http://proton.semanticweb.org/) ontology which we have extended with the ontology of software patterns (see Section 3.1) in order to make KIM aware of the concepts from the domain of software DPs. In that way, we were able to use KIM annotation facilities to automatically annotate documents from an online repository. To make documents from an online repository available to KIM for processing, DEPTHS uses a Web crawler that traverses through the structure of the online repository, collects URLs of all its documents and sends them to the KIM annotation facilities.

Semantic annotation of the internally produced content (e.g., chat messages, forum messages and ideas) is performed in the similar way, immediately after the user creates a content unit (i.e., following the event of submitting a new content unit to the system). Additionally, each recognized term (DP label) is changed to the hyperlink, used for launching web-recourse and internal content finding services.
Based on the semantic (meta)data generated using KIM, DEPTHS is aware of the DPs mentioned in each document. However, in order to find out what the most relevant DP for a given document is, and what the most relevant document is for a specific DP in the whole corpus of documents, additional indexing is performed.
The index data for each document contains two values: the dominant DP (i.e., concept from the DP ontology) and its accompanying relevance value. To find the dominant software DP in each document (i.e., what the document is about), DEPTHS uses the term frequency-inverse document frequency (TF-IDF), which is a proven statistical measure used in information retrieval to evaluate how important a word is to a document in a collection. DEPTHS uses a document’s semantic annotations to create a collection of all software DPs mentioned in that document. Subsequently, it checks how relevant the document is for each DP found in it in order to find the dominant DP. If the TF-IDF value for the dominant DP is lower than the predefined threshold, DEPTHS eliminates this document as not enough relevant for any specific DP. As the result of this process, we got a set of documents, each of which describes one DP. Additionally, for each of the indexed documents (i.e., those whose dominant DP pass the threshold), we also capture the relevance value. The relevance value is computed as a cosine similarity between the TF-IDF value of a document’s dominant DP (concept) and the vector of the TF-IDF values of the DPs (concepts) discovered while the document was semantically annotated. Thus, we can use this relevance value in the process of ranking. That is, the data about what each document is about (i.e., concept of the dominant pattern) and how relevant it is for a specific DP (i.e., relevance value), are stored in the Repository of LO metadata, and used later by the Web resource finding service for the recommendation of Web resources.

Context-aware learning services are accessible to all systems and tools integrated in the DEPTHS environment and are exposed to end users (students) as context-aware learning features. They include (but are not limited to):
Web resource finding. Based on the metadata provided by the Semantic Annotation and Indexing Service, this service generates a list of recommended Web resources from publicly accessible repositories of software DPs. The service gets activated when a student selects a hyperlink to a domain topic (i.e., a topic related to software DPs) available within different kinds of learning content (e.g., lessons, chat messages and project descriptions). These hyperlinks result from the semantic annotation of the learning content done by the Semantic Annotation and Indexing Service. Afterward, the service sends a query to the Repository of LO metadata to extracts information about all Web resources that are relevant for the desired domain topic (i.e., DP), as well as their relevance values. Additionally, this service makes use of students estimations of the resource’s relevance for the current DP (each time a student visits a suggested Web resource he is asked to rate its relevance for the given DP). Students’ positive and negative ratings affect the resource’s overall rating according to the influence factor (value between 0 and 1) defined by the teacher or the system administrator.
Discovery of relevant internally produced resources. This service suggests internally created resources (e.g., discussion threads, brainstorming notes, and project description) that could be useful for a student to solve a problem at hand in the given learning context. The computation of relevance is done in a similar manner to the one above described for external, Web resources.

peers competence
Factors affecting estimation of potential collaborators’ competences in DEPTHS



Experts, teachers and peers discovery. Based on the current learning context, this service suggests other students or experts as possible collaborators. Collaborators are selected and sorted using an algorithm which considers their competences on three different levels: same content (i.e., current software problem), similar or related learning content (i.e., similar software problem) and broader content (i.e., software problem in the same course). Estimation of a peer’s competence (C) on each level is performed through assessing three types of competence indicators:

  • participation (Cp) in learning activities (e.g., brainstorming, submitting or assessing peers’ works). Each activity has different impact factor on the system’s estimation of the student’s competences. This factor is defined in the system itself, but could be changed by the teacher.
  • knowledge level (Ck) estimated by the teacher and obtained through the peers’ evaluations, including projects evaluations and idea ratings. However, not all ratings have the same influence on the knowledge level estimation. For example, a high mark given by a student with high competences on the given topic has more impact on the final knowledge level appraisal than a high mark given by a student with average or low competences.
  • social connections (Csc) with the peer asking for help - the stronger the social connection with a specific person, the more suitable that person is for the help provision. We believe that an already appointed social connection could be much more successful and effective than new connections with people one does not know. Social connections among students are mined from their interaction data (represented in accordance with the Learning Context ontology and stored in the DEPTHS’s Repository of interaction data). For newcomers we are using their FOAF profiles, if available. Otherwise, this indicator is not considered.

All this data is collected through the queries performed on both DEPTHS’ semantic repositories. Each type of competence indicator has different influence on the overall competence which depends on the predefined weight factor (WF) assigned to it (default systems’ values could be changed by the teacher). To compute the overall competence of a peer, we use the following formula:
competence
where WFp + WFk + WFsc = 1, and 0 £ WFp, WFk, WFsc £ 1.

Context-based semantic relatedness.This service is used by all other services, as it allows for: i) computing context-based semantic relatedness between tags that students define and/or use in the given learning contexts ; ii) connecting students’ tags with appropriate concepts of the domain ontology (i.e. disambiguation of the tags with the domain concepts); iii) resolution of students’ queries containing both, tags and domain concepts relevant for the given learning context. This service connects tags with the concepts of the domain ontology as well as resources annotated with these concepts and stored in the DEPTHS semantic repositories (Repository of LO metadata and Repository of interaction data). That way the system selects only appropriate tags to show in the given moment based on the current learning context, and connects these tags with appropriate domain concepts related to it. This service is currently in its development stage and uses context-based semantic relatedness measure described in our previous work .