Ontology in association rules

New information is created every day and brought to public notice through the Internet or company databases. Data can come from different sources, in different formats and even with different structuring. To gain insights from such large amounts of data, researchers and businesses have developed techniques to manipulate this humanly unmanageable amount of information. In this scenario, data mining has come to be regarded as a useful technique with which to synthesize a large amount of data into more condensed sets of knowledge. Results for various data mining techniques have been promising, when observed at a small scale. There is concrete evidence of the usefulness of data mining of large datasets. Nonetheless, data mining experts are required to navigate and prune the results, looking for useful, unexpected and promising patterns and associations raised in the process (Fayyad et al. 1996; Berry and Linoff 1997; Kantardzic 2003). However, it is not uncommon for data mining to lead to an unmanageable number of outcomes with a great deal of co-related results that only challenge human understanding. Many pre- and post-processing techniques (Jaroszewicz and Simovici 2002; Goethals et al. 2005; Jager 2008) have been proposed to increase the precision and recall of the results. Mostly, these techniques are based on statistical characteristics of the outcome set.

Syntactic proposals of association rule pruning are insufficient because they prune what should not be pruned and refrain from pruning what should. The semantic approach to pruning improves the precision of the results, as filtering does not prune what it should not. Rajan and Dhas (2012) and Nofal and Bani-Ahmad (2010) evaluated and compared favorable results obtained using semantic methods for association rules. Moreover, it has been shown that post-processing can be efficiently integrated with existing rule reduction techniques to construct a concise, high-quality, and user-specific association rule set (Rajan and Dhas 2012; Chen et al. 2008). By integrating objective methods with a semantic approach, it is possible to successfully identify non-trivial, distinctive, semantically correct, and user-specific rules (Chen et al. 2008). There are also semantic-based data mining techniques that consider the taxonomic relations between the elements of the mined results to constrain the outcomes (Silberschatz and Tuzhilin 1996).

Semantic methods improve data mining by bringing known information about the data set to filter and tune the new knowledge extraction. The semantic knowledge can be provided by means of a domain dictionary (taxonomy), a domain ontology in which relationships between concepts are also included, or even a list of expressions that reflect social constructions intended for communication and crystallization of domain-specific knowledge – Sicilia (2006).

Traditional data mining is based on the frequency of occurrence of instances and the co-occurrence of items in transactions. The meaning of each item or instance is not taken into consideration. The semantic content extracted from the ontologies allows inserting more intelligence and knowledge in data mining, improving their quality.

The use of ontologies for best results in data mining has been extensively researched in many respects. Charest and Delisle (2006) developed an ontology-guided method for data mining using case-based reasoning. The method is based on having an expert system assistant to help non-expert data miners (d'Aquin et al. 2012).

Žáková et al. (2008) use a domain ontology and a task description to create a workflow for guiding data mining process. The knowledge discovery task is converted in a planning task.

Manda et al. (2013) presented a data mining approach, entitled multi-ontology data mining at All Levels (MOAL), that uses the structure and relationships of a Genetic Ontology to mine multi-multi-level association rules.

Zhou and Geller (2008) use ontologies to enrich the web mining for marketing domain. Their work uses two taxonomies (customers and interests) in the data mining pre-processing phase to improve the precision and recall of the rules. Their proposal is the very similar to the SemPrune model presented in this paper.

The purpose of the SemPrune is to generate better data mining results on database for any task or domain through ontology-enrichment of data. Generalization and specialization information guide merging, deleting or adding knowledge rules.

The idea behind SemPrune comes from observations of ecommerce transactions. Consider a dataset of online sales transactions. The simplest way of using subjacent knowledge is to group attributes into value ranges such as by grouping purchasable items by price range (cheap, normal price, expensive), individuals by height (short, medium, tall), or workers by productivity (low, medium, high, excellent). Comprehensive variables, such as the price range of items, generalizers, or grouping representatives, are those which, according to the information-storage perspective, are called dependent, since whenever a specific (or determinant) variable is known, the comprehensive (or dependent) variable is also known. Each type of relationship has its representation and resolution dependency mechanisms. Usually the determinant attributes (which give rise to aggregating attributes) occur in small numbers. This type of analysis is often used in the case of value ranges of attributes and derived aggregator attributes. A pre-processing activity would normally create new rules with the dependent variables established in a file of dependencies. The present research applies the semantic enhancement of rule mining along with the pruning treatment.

Aggregating variables can provide more general or comprehensive rules. These rules can be compared to the mined rules through metrical structures based on objective measurements (coverage in post-processing or distribution uniformity in pre-processing) that indicate if the new rule may substitute a set of original rules (generalization) or if it does not add information, in which case it must be unconsidered (rule specialization). The comprehensive rules derive from domain ontology. The treatment of the set of mined rules employing filtering techniques for the reduction of cardinality is a classic treatment, and the innovation presented in this paper is semantic filtering in lieu of syntactic filtering employing objective rules of interest.

We proposed an ontology-based model, called SemPrune (Ferraz 2008), which combines a semantic pre-processing of the data set with a semantic post-processing of the data mining outcomes. We claim that making use of known domain concepts and relations can positively affect data mining recall and precision, while reducing the volume of the outcomes. We have developed an experiment using public data sets (Lucas et al. 2002; Merz and Murphy 1998; Collective Bargaining Review, monthly publication 1998), the association rules’ data mining technique and domain ontology related to the datasets. Experiments, using SemPrune, have given results that are more concise and precise than the results obtained using semantic filtering techniques, such as Conviction, Specificity, Lift and Novelty (Lavrac et al. 1999). The comparison was performed quantitatively and qualitatively. The quantitative comparison was performed according to the number of pruned rules (recall). The qualitative comparison was performed by human evaluation.

In this paper, we describe our research problem, and then SemPrune. Experiments and a comparison of our approach with other research works are presented.

The SemPrune model is presented in two versions that are composed of a number of modules. The descriptions of the algorithms in these modules are condensed as much as possible to meet the specifications of the Conference, but the text in full is available upon request.

When SemPrune makes the semantic enrichment,we find a more general rule and several specific rules. In the preprocessing enrichment, the computed value of CRg is compared with the specified CRgmin value. When CRg ≥ CRgmin the specific rules can be discarded because they are represented by the more general rule. Otherwise, the discarded rule is the more general rule. In tests with CRgmin it was found that the best results for this threshold varied between different databases (from 0.5 to 0.8). For enrichment in preprocessing the similar comparison was made between TRg and TRgmin.

SemPrune for post-processing

The SemPrune version for enriching the post-processing is illustrated in Figure 1. The Transactions’ database is the source for the data mining technique to extract a set of association rules. The rules obtained from this and the domain ontology are then sent to an enhancer module that enhances the set of association rules with dependent attributes that provide a higher power of intuition for the analyst. From this point forward, there is a confluence between the two versions.

The redundancies between rules were analyzed according to the premise that there are always dependencies between attributes and rules, because some rules may have, in the antecedent or consequent, determinant attributes and determined attributes. The redundancy between them can be of four types:

• the antecedent of the rule has determinant attributes and the consequent has a dependent attribute (redundant with the premise) of the form "If there is a rule of the form X ⇒ Y ^ C, this is redundant, and C is a set of conditions defined on attributes of D";

• a rule has, in the antecedent, determinant attributes, and another with the same consequent, has, in the antecedent, determinant attributes plus the dependent attribute (second rule is redundant with the first) of the form "If there are two forms of rules R1: X ^ A ⇒ B and R2: X ^ Y ^ A ⇒ B, then rule R2 is redundant, A and B being defined sets of conditions on attributes of D;

• a rule has, in the antecedent, determinant attributes, and another, in the consequent, the dependent attribute plus the consequent of the first rule (the second rule is redundant with the first) of the form "If there are two forms of rules R1: X ^ A ⇒ B and R2: X ^ A ⇒ Y ^ B, then rule R2 is redundant”;

• a rule has, as part of the antecedent, a dependent attribute, and another rule has as a sole difference the replacement of the dependent attribute by their determinant attributes (second rule is redundant with the first) of the form "If there are two forms of rules R1: Y ^ A ⇒ B and R2: X ^ A ⇒ B, then rule R2 is redundant".

SemPrune for pre-processing

The pre-processing semantic enrichment SemPrune version is illustrated in Figure 2. Based on a transactions database and a domain ontology of the items of the transactions, a database enhancer module generates an enriched set of transactions. This transaction set is mined to generate an enhanced set of mined association rules. This set of rules and the domain ontology feed a generator module that, for every more general rule, it locates a set of corresponding specific rules. These sets of rules (for each more general rule there is a subset of specific rules) are analyzed by a rules relevance analyzer module, whose output is made up of the entrance sets with the corresponding relevance indexes. Finally, a less relevant rules pruner module executes a filter whereby only the semantically more relevant rules of the sets of rules (more general + specific) will succeed.

The proposed method for the problem solution then aims to eliminate redundant association rules, or less impact rules based on the dependency relationships between attributes, assuming that the information enrichment has already been done in the preprocessing of association rules mining.

The method adopted to solve the problem is similar to that used for filtering with the inclusion of world knowledge in the post-processing. The biggest difference is the definition of dependence between attributes. In the case of a relationship 1:n, it is considered that the descendent in the taxonomy is determinant of its ancestor, which implies greater simplicity in the calculation of values of dependent attributes for grouping, separation, comparison, and so on. In contrast, the number of dependencies increases, because within the hierarchy, every attribute has a father (just has dependency).

With the values obtained, we calculate the standard deviation of the support distribution of specific rules. If the ratio of the standard deviation and arithmetic mean of the distribution is below a specified level, the behavior of specific rules is regular, and they can be replaced by the general rule and pruned. Otherwise, there are singular specific rules, which do not fit the general rule and cannot be pruned. We must then determine if the general rule is representative of non-singular specific rules. If the general rule covers sufficiently the non-singular specific rules, the latter will be pruned. Otherwise, it is the general rule that brings no value, and can be pruned.

Experimental evaluation

To evaluate the results obtained from the semantic enhancements made in the post-processing phase, we used databases from usual Internet repositories and introduced dependent attributes, like the range and aggregator, which is common practice in data mining. In each case, the rule enhancements were compared using knowledge of the world during pre-processing and post-processing. The association rules were extracted using the apriori algorithm (Lucas et al. 2002). Domain ontology emphasized “is-a” and “part-of” relations among domain concepts. We selected the adult database (32,561 records) and labor database (57 records) publicly available from the UCI Machine Learning Repository (Merz and Murphy 1998). The STULONG database (1317 records) was obtained from ECML/PKDD 2004 Discovery Challenge (ECML/PKDD, 2004).

World knowledge aggregated to mined data will generate redundancy, since wide-ranging attributes and specific attributes will coexist. Wide-ranging attributes, such as price range of items, pattern generalizers, or grouping representatives, are called dependents under the information storage point of view, since once a specific (or determinant) attribute is known, the wide-ranging (or dependent) attribute is also known.

For the adult database we used 14 attributes of the data base and from the World Knowledge we included three dependent attributes (Hollingshead Index of Social Position - ISP, social-class and Age-group). The target attribute was the income. For the stulong database we used 18 attributes of the data base and from the World Knowledge we included three dependent attributes (Body Mass Index - BMI, Status and Age-group). The target attribute was the blood-pressure. For the labor database we used 16 attributes of the data base and from the World Knowledge we included two dependent attributes (Wage-inc and Sweat-hours). The target attribute was class.

The support values adopted were 4% for the labor database and 2% for the other two databases. The confidence value adopted in all mining cases was 70%. The maximum size specified for the frequent itemsets was three for the STULONG database and five for the other two databases.

Northwind Traders is a database of a fictitious trading company, provided as an example, with Microsoft products Access and SQL Server, the purpose of which is to support applications and MBP ("Modeling Business Processes").

To turn it into a database transaction, which allows data mining of user behavior, it was necessary to create an adequate view. Data on the purchasing habits of users are recorded in Order Details, Customers and Orders tables.

The attributes that are part of the vision of database transactions were CustomerID, and ProductID Order Date, and that will be part of world knowledge are ProductID and CategoryID.

The concepts and relationships of the ontologies used in this work can be seen in Figure 3.

The support values adopted were 4% for the labor database and 2% for the other two databases. The confidence value adopted in all mining cases was 70%. The maximum size specified for the frequent itemsets was three for the STULONG database and five for the other two databases.

It is observed that the semantic filter was effective for all sample databases. The reduction rates were reasonable, reaching more than 23% for the STULONG database. The proposed semantic method decreases the number of rules in addition to providing more consistency to the results in each case.

To test the effectiveness of SemPrune in the data pre-processing phase, we used the Northwind Traders database. This is a database of a fictitious trading company, shipped as a sample with Access and SQL Server Microsoft products, whose purpose is to support database applications and modeling business processes. The database consists of eight main tables and seven auxiliary tables, and comes with 16 views and seven stored procedures.

The transactions database was generated by the junction of the Orders and OrderDetails tables, creating itemsets of products purchased by the same CostumerID. The world knowledge, the base for the taxonomic relationship, was obtained by joining the Categories and Products tables.

For relationships with the inclusion of world knowledge in the pre-processing, the generated rules had a maximum of five items, minimal support of 0.04, and 9941 mined rules for the Northwind Traders database.

The superiority of semantic filtering can be verified by comparing results thus generated with post-processing filtering guided by objective interest measures, specifically Conviction, Specificity, Lift and Novelty (Lavrac et al. 1999).

For the qualitative comparison of data mining post-processing between syntactical methods (pruning using Conviction, Specificity, Lift and Novelty) and semantic methods (pruning using generalization/specialization), performance tables can be employed. The qualitative comparison must be conducted in light of the domain ontology through the observation of an analyst who can attest to the conformity of the results obtained by filtering with the domain ontology.

In the performance table, shown in Table 5, similar to the case for confusion matrixes, it can be said that the pruning of any rule can be viewed in light of two aspects:

The main diagonal of Table 5 presents the number of rules that should and were pruned and the number of rules that should not and were not pruned. These values indicate the degree to which the syntactical methods were correct. The secondary diagonal of the tables presents the number of cases in which the measure of objective interest did not function correctly, eliminating rules that should not have been kept and keeping rules that should have been eliminated. For example with conviction of 1.5, we see that the main diagonal sum (1079 + 392) is greater than the secondary diagonal sum (785 + 694).

For relationships with the inclusion of world knowledge in the pre-processing, the generated rules had a maximum of five items, minimal support of 0.04, and 9941 mined rules for the Northwind Traders database. The superiority of the results obtained by semantic filtering in the pre-processing can thus be seen by comparing results generated by filtering according to objective interest measures, specifically Conviction, Specificity, Lift and Novelty (Lavrac et al. 1999).

However, this gain cannot be generalized, as it depends on the ontology. A high number of attributes in the database that have ancestors increases the probability of pruning. (The number was too high in the case of the Northwind Traders database.) The alternative hypothesis specified only the recall gain and this has been achieved.

The study described in this article demonstrated the positive effect of using domain ontology to prune association-rule outcomes. We addressed the issue of excessive, though redundant, association rules being generated by traditional data mining techniques. The result, which we were searching for, was a reduction of the set of mined rules while maintaining precision and reducing recall. Related work was carried out by Srikant and Agrawal (1995) and Bürkle (2006). In the first work, the filtering was conducted by a frequency model (interestingness), and in the latter, associations rules were enhanced in the pre-processing step and only for “is-a” relationships.

Recall is already efficiently reduced by syntactic filtering measures that use objective interest measures of the rules. Nevertheless, the fact that there are several dozen competing objective measures is already an indication that none of them is absolute. When analyzing the pruned rules using these various measures, it becomes clear that the degree of superposition is low, and therefore, a loss in precision is inevitable.

SemPrune is limited to datasets for which domain ontology is available and for which data items are considered balanced. Since the technique uses intrinsic semantic relations between values of the same attribute, imprecise domain description may drastically affect the results. Additionally, whenever the dataset is unbalanced, there is a risk of false generalization. For example, if the data set only contains data including Coke and Pepsi, SemPrune might lead to generalized rules of non-alcoholic beverages because there is no evidence in the data set of any other drink.

Our suggestion, which uses knowledge of the world obtained from domain ontology, may enhance the semantics of the rule set obtained and substantially reduce its cardinality. Furthermore, semantic rule pruning does not eliminate relevant rules nor does it fail to eliminate redundant rules.

The results of the experiments obtained with public databases show that the suggested model fully met the desired goals. As a side effect, the integration passage of the pre-processing ontology to the post-processing significantly reduces computational costs, as shown by the number of rules to treat.

SemPrune is based on the classic way humans learn by anchoring new knowledge on previous knowledge, finding shortcuts to generalize, and identifying scenarios for which specifics matter. Our results are cautious steps towards merging these two powerful areas of data mining and domain ontology representation to face the challenge of dealing with big data.

Inhauma Neves Ferraz was born in Brazil, in 1940. He received a B.E degree in civil engineering from Instituto Militar de Engenharia, and a B.S. degree in mathematics from Universidade Federal do Rio de Janeiro. He obtained M. Sc. degrees in mechanical engineering from Universidade Federal de Itajubá and in system engineering from Instituto Militar de Engenharia. A D. Sc. Degree in computer science was obtained from Universidade Federal Fluminense.

He has held lecturing positions at the Universidade Estadual Paulista Júlio de Mesquita Filho, Universidade Federal de Itajubá, Instituo de Tecnologia de Aeronáutica, Instituto Militar de Engenharia e Universidade Federal Fluminense.

His main areas of research interests are information security, artificial intelligence and computer networks.

He is currently Vice Director of ADDLabs, the Artificial Intelligence laboratory of Computer Science Institute at Universidade Federal Fluminense.

He has published more than 40 scientific papers, a book and more than 30 technical reports on computer science.

Ana Cristina Bicharra Garcia is a professor at the Computer Science Department of the Universidade Federal Fluminense, Brazil. She created ADDlabs in 1996 and has since directed the laboratory; ADDlabs is an artificial intelligence research laboratory focused on the petroleum exploration domain (http://www.addlabs.uff.br). She received her M.Sc. (1988) and Ph.D. (1992) degrees from Stanford University. She has advised seven PhD students and approximately 30 MSc students. She has written numerous papers on artificial intelligence applied to design, data mining techniques, cooperative work, e-government, e-commerce and knowledge acquisition. Her current interests involve knowledge acquisition and collective intelligence. She has published 20 journal papers, 12 book chapters and 124 conference papers.