Concepts Selection in Fuzzy Cognitive Map using Evolutionary Learning Algorithm based on Graph Theory Metrics

—Fuzzy cognitive map (FCM) allows to discover knowledge in the form of concepts signiﬁcant for the analyzed problem and causal connections between them. The FCM model can be developed by experts or using learning algorithms and available data. The main aspect of building of the FCM model is concepts selection. It is usually based on the expert knowledge. The aim of this paper is to develop and analyze a new evolutionary algorithm for selection of key concepts and determining the weights of the connections between them on the basis of available data. The proposed approach allows to reduce concepts during learning process based on metrics from the area of graph theory: signiﬁcance of each node and total inﬂuence of the concept. A simulation analysis of the developed algorithm was done with the use of real-life data.


I. INTRODUCTION
F UZZY cognitive map (FCM) is a directed weighted graph for representing knowledge [9].It is an effective tool for modeling dynamic decision support systems [13], [25].Fuzzy cognitive maps allow to visualize complex systems as a set of key concepts (nodes) and connections (links) between them.The FCM model can be built based on expert knowledge [3], [4].Experts choose the most significant concepts and determine type and strength of the relationships between them (weights of the connections).Fuzzy cognitive map can be also initialized with the use of learning algorithms [16] and historical data.Standard supervised [8] and evolutionary algorithms [13], [23], [24] allow to determine the structure of the FCM model based on all available data.For each data attribute new concept is created.Next the weights of the connections are specified during learning process.
Fuzzy cognitive maps with the large number of concepts are difficult to analyze and interpret.Moreover, with the growth of the number of concepts, the number of connections between them that should be determined increases quadratically.Several researchers have attempted to develop methods of reduction of fuzzy cognitive map size.In [5] a new approach for reduction of the FCM model complexity by merging related or similar initial concepts into the same cluster of concepts is presented.These clusters can be used then as the real concepts in the reduced FCM model.Concepts clustering technique based on fuzzy tolerance relations was used in [17] for modeling of waste management system.The analysis of the decision making capabilities of the less complex FCM shows that proper concepts reductions make models easier to be used keeping their original dynamic behavior.Also cluster validity indexes were introduced to evaluate Fuzzy Cognitive Map design before training phase in [6].The resulting FCM models are easy to interpret and properly perform the task of prediction.Homenda et al. [7] introduced a time series modeling framework based on simplified Fuzzy Cognitive Maps using a priori nodes rejection criteria.The obtained results confirmed that this approach for simplifying complex FCM models allows to achieve a reasonable balance between complexity and modeling accuracy.Selvin and Srinivasaraghavan proposed an application of the feature selection techniques to reduce the number of the input concepts of fuzzy cognitive map [21].The feature selection methods were performed based on the significance of each concept to the output concept.However the influences of the connections between the concepts were not taken into consideration.In [18], [19] the structure optimization genetic algorithm for fuzzy cognitive maps learning was presented.It allows to select the most significant concepts and connections between them based on random generation of possible solutions and the error function that takes into account an additional penalty for highly complexity of FCM during learning process.The usefulness of the developed approach was shown on the example of the one-step ahead time series prediction.
The advantage of the FCM model is its graph-based representation that allows to use various methods and metrics from the area of graph theory to analyze the structure and behavior of the modeled system [26].In this paper we propose to use two various metrics to reduce concepts of the FCM model during learning process.The first metric is the degree of a node.It denotes its significance based on the number of concepts it interacts with (is affected by and it affects) [4].The second metric is one of the system performance indicators: the total (direct and indirect) influence of the concept [2], [22].
The aim of this paper is to develop the evolutionary learning algorithm that allows: • to reduce the size of the FCM model by selecting the most significant concepts, • to determine the weights of the connections between concepts, • to approximate the real-life data [10], [15], [20].The comparison of the developed approach with the standard one based on the all possible concepts and data error and the previously developed approach based on density and system performance indicators [12] was done.The learning process was performed using two effective techniques for FCMs learning: Elite Genetic Algorithm (EGA) [14] and Individually Directional Evolutionary Algorithm (IDEA) [11].
The outline of this paper is as follows.Section II briefly describes fuzzy cognitive maps.Section III presents the proposed approach for fuzzy cognitive map learning and concepts selection.In Section IV, the results of the simulation analysis based on real-life data are presented.Section V contains the conclusions and further work.

II. FUZZY COGNITIVE MAPS
Fuzzy cognitive map is a directed weighted graph for representing causal reasoning [9]: where X = [X 1 , ..., X n ] T is the set of the concepts, n is the number of concepts determining the size of the FCM model, W is the connection matrix, w j,i is the weight of the influence between the j-th concept and the i-th concept, taking on the values from the range [−1, 1].w j,i > 0 means X j causally increases X i , w j,i < 0 means X j causally decreases X i .Fuzzy cognitive map allows to model behavior of dynamic decision support systems and can be used in a what-if analysis [1].The values of the concepts determine the state of the FCM model and can be calculated according to the selected dynamic model.In the paper one of the most popular dynamic models was used [23]: where X i (t) is the value of the i-th concept at the t-th iteration, i = 1, 2, ..., n, t is discreet time, t = 0, 1, 2, ..., T .Transformation function F (x) normalizes values of the concepts to a proper range.The most often used function is a logistic one, described as follows [23], [24]: where c is a parameter, c > 0.

III. PROPOSED APPROACH
The aim of the proposed approach is automatic concepts selection in fuzzy cognitive map during learning process using metrics from the area of graph theory.This approach requires determination of the decision (output) concepts.Other concepts are input concepts.The obtained model consists only key input concepts that affect to the decision/output concept (or concepts).During learning process we evaluate the candidate FCMs based on data error calculated for decision concepts.The significance of the concept (the degree of the concept) and the total influence of the concept were taken into account in the process of the key concepts selection.
The proposed approach contains the following steps: STEP 1. Initialize random population.An initial population is generated before starting evolution loop.Each candidate FCM is described by the two vectors.The first vector (4) describes values of weights between concepts [23]: where w j,i ∈ [−1, 1] is the weight of the connection between the j-th and the i-th concept, , j = 1, 2, ..., n and n is the number of concepts.The second vector (5) describes the state of each concept: where c i is the state of i-th concept and n is the number of concepts.
Each concept can be in the one of the three states: active (AS), inactive (IAS) and always active (AAS).The decision concept is always active.This mean, that obtained model always contains decision concept (concepts).The concepts with AS state and the decision concepts creates the collection of key concepts.
During the first step, the elements of the W ′ vector are initialized with the random values form the interval [−1, 1].The state for every node is active for all individual in the initial population.For this reason, the elements of the C vector are equal to AAS for the decision concept (concepts) and AS for the other concepts.STEP 2. Evaluate population.Each individual is evaluated based on the following fitness function: where Error is the objective function calculated on the basis of data error for the decision concepts: where X i (t) is the value of the ith decision concept at iteration t of the candidate FCM, Z i (t) is the value of the i-th decision concept at iteration t in the input data, t = 0, 1, 2, ..., T , T is the input data length, i = 1, ..., n d and n d is the number of decision concepts.STEP 3. Check stop condition.If the number of iterations is greater than iteration max then the learning process is stopped.STEP 4. Select new population.The temporary population is created from a current base population using roulette-wheel selection with dynamic linear scaling of the fitness function [14].

90
COMMUNICATION PAPERS OF THE FEDCSIS.PRAGUE, 2017 STEP 5. Select key concepts.Process of selection of key concepts is carried out in 3 ways: 1) Key concepts are selected at random (SC_RND).The state of each input concept for each individual may be changed with a certain probability.The value of state change probability is in the range (0, 1).The concept, whose state is AS may be removed from the key concepts collection by changing the state to IAS.The concept, whose state is IAS may be added to the key concepts collection by changing the state to AS.The values of W ′ vector are not modified.2) Key concepts are selected based on the degree of the node (CS_DEG).
The degree of the node (8) denotes its significance based on the number of concepts it interacts with (is affected by and it affects) [4]: where n is the number of the concepts; w j,i is the weight of the connection between the j-th and the i-th concept; i, j = 1, 2, ..., n.
The state of some concept without the decision concept (concepts) for each individual may be changed with a certain probability.The value of state change probability is in the range (0, 1).The i-th concept with minimum value of deg i (8) from the set of key concepts (concepts whose state is AS) will be removed from the key concepts collection.The value of state attribute of this concept will be changed to IAS.The i-th concept with maximum value of deg i (8) from concepts whose does not belong to the key concepts collection (concepts whose state is IAS) will be added to the key concepts collection.The value of the state attribute of this concept will be changed to AS.The value of the state attribute change probability is equal to 0.5.3) Key concepts are selected based on total influence of each concept (CS_INF).The total (direct and indirect) influence between concepts is described as follows [2], [22]: where n is the number of the concepts, p j,i is the total (direct and indirect) influence between the j-th concept and the i-th concept calculated on the basis of the total causal effect path between nodes [12], i, j = 1, 2, ..., n.
The state of input concept for each individual may be changed with a certain probability.The value of state change probability is in the range (0, 1).The i-th concept with minimum value of the total influence inf i (9) will be removed from the key concepts collection.The value of the state attribute of this concept will be changed to IAS.The i-th concept with maximum value of the total influence inf i (9) from concepts whose does not belong to the key concepts collection (concepts whose state is IAS) will be added to the key concepts collection.The value of the state attribute of this concept will be changed to AS.Also, in this case the value of the state attribute change probability is equal to 0.5.STEP 6. Apply genetic operators with the use of selected evolutionary algorithm.In this paper Elite Genetic Algorithm [14] and Individually Directed Evolutionary Algorithm were used [11].The genetic operators were applied only to the W ′ vector.The C vector was processed by independent procedure described in STEP 5.
STEP 7. Analyze population.Evolution loop is extended by the process of the analysis of potential solution according to the previously developed approach [12].The values of weights from [−0.05, 0.05] are rounded down to 0 as suggested in [23].Next, the matrices with the total influence between concepts p j,i are calculated.If the value of p j,i is in the interval [−0.1, 0.1], the corresponding weight value w j,i is rounded down to 0. Moreover, genetic operators implement density control method of potential solution for consistency of the algorithm.Go to STEP 2. STEP 8. Choose the best individual and calculate evaluation criteria.To evaluate performance of the proposed approach, we used two criteria that are commonly used in fuzzy cognitive map learning: 1) Initial error allowing calculation of similarity between the input learning data and the data generated by the FCM model for the same initial state vector: where X i (t) is the value of the i-th decision concept at iteration t of the candidate FCM, Z i (t) is the value of the i-th decision concept at iteration t of the input model, t = 0, 1, 2, ..., T , T is the input data length, i = 1, ..., n d , n d is the number of decision concepts.
2) Behavior error allowing calculation of similarity between the input testing data and the data generated by the FCM model for the same initial state vectors: ) where X p i (t) is the value of the i-th decision concept at iteration t of the candidate FCM started from the p-th initial state vector, Z p i (t) is the value of the i-th decision concept at iteration t of the input model started form the p-th initial state vector, i = 1, ..., n d , n d is the number of decision concepts, p = 1, 2, ..., P , P is the number of the initial testing state vectors.

IV. EXPERIMENTS
To analyze the performance of the developed evolutionary algorithm for concepts selection real-life data were used.The aim of the analysis is to select the most significant concepts, determine the influence between them and approximate the real-life data for the output concepts.
Standard approach for fuzzy cognitive maps learning (STD), the approaches: for random concepts selection (CS_RND), for selection based on the degree of the concept (CS_DEG), for selection based on the total influence of the concept (CS_INF) and two previously analyzed algorithms based on density (DEN) [12] and based on system performance indicators (SPI) [12] were compared.

A. Dataset
Real-life data were obtained based on the three FCMs reported in literature [10], [15], [20].The first real-life model is a decision support system in radiotherapy [15].It contains 16 concepts: the factor-concepts (X 1 -X 5 ), that represent the depth of tumor, the size of tumor, the shape of tumor, the type of the irradiation and the amount of patient thickness irradiated, the selector-concepts (X 6 -X 13 ), representing size of radiation field, multiple field arrangements, beam directions, dose distribution from each field, stationery vs. rotationisocentric beam therapy, field modification, patient immobilizing and use of 2D or 3D conformal technique, respectively and the three output-concepts (X 14 -X 16 ): dose given to treatment volume, amount of irradiated volume of healthy tissues and amount of irradiated volume of sensitive organs.The second model is a notional FCM model for the evaluation of mining jurisdiction investment favorability [20].It contains 11 input concepts: national gov.stability (X 1 ), regional gob.stability (X 2 ), support for mining industry (X 3 ), workforce education (X 4 ), workforce skills/experience (X 5 ), infrastructure availability(X 6 ), permitting delays (X 7 ), gov.royalty rates (X 8 ), tax rates (X 9 ), environmental activism (X 10 ), union activism (X 11 ) and one output node: investment favorability (X 12 ).The last fuzzy cognitive map for modeling the behavior of soldiers consists of 10 concepts: cluster (X 1 ), proximity of enemy (X 2 ), receive fire (X 3 ), presence of authority (X 4 ), fire weapons (X 5 ), peer visibility (X 6 ), spread out (X 7 ), take cover (X 8 ), advance (X 9 ) and fatigue (X 10 ) [10].Concepts: X 5 , X 8 and X 9 were selected as an output of the system.
The input data for the learning process were generated starting from the one random initial vector for every map.The resulting FCM models were tested on the basis of 10 testing state vectors (P = 10) and evaluated with the use of criteria ( 10)- (11) and the number of concepts n.

B. Learning parameters
The following parameters were used for the EGA algorithm: • selection method: roulette wheel selection with linear scaling • recombination method: uniform crossover, • crossover probability: 0.75, • mutation method: non-uniform mutation, • mutation probability: 0.02, • population size: 100, • number of elite individuals: 2, • maximum number of iterations: 100, The following parameters were used for the IDEA algorithm: • selection method: roulette wheel selection with linear scaling • mutation method: directed non-uniform mutation, • mutation probability: 1 n 2 −n • population size: 100, • maximum number of iterations: 100, 10 experiments were performed for every set of the learning parameters and the average values (Avg) and standard deviations (Std) were calculated.

C. Results
Table I summarizes the average results of the experiments with the real-life data: the number of the concepts for the resulted FCM models n, initial and behavior error.Fig. 2. Sample results of testing initial and behavior error.Figure 1 presents the structures of the best models received using the standard method and the developed approaches for the second real-life model and EGA algorithm.Figure 2 shows the sample results of testing of the presented FCM models.
The obtained results show that the developed approach for fuzzy cognitive maps learning allows to approximate the real-life data with satisfactory accuracy comparable to the other approaches.It is observed that the proposed algorithm, in most of the cases, gives the lowest or very close to the lowest values of initial and behavior error.The advantage of the developed algorithm is the ability to reduce the size of the FCM models (the number of concepts n) by selecting the most significant concepts using graph theory metrics.

V. CONCLUSION
This paper introduces the evolutionary algorithm for selection of key concepts and determining the weights of the connections between them on the basis of real-life data.Graph theory metrics were used to reduce the number of concepts of fuzzy cognitive map during learning process.Effectiveness of the proposed approach was analyzed with the use of Elite Genetic Algorithm and Individually Directional Evolutionary Algorithm.The experiments confirmed that the developed approach allows to reduce the size of the FCM model by selecting key concepts and determine the weights of the connections between them keeping satisfactory level of error data.We are going to continue analysis of the developed technique for fuzzy cognitive maps learning with the use of

Fig. 1 .
Fig. 1.Structures of the best FCM models for the analyzed approaches: a) STD, b) CS_RND, c) CS_DEG, d) CS_INF Table II shows the best results of the experiments.The highlighted values in bold show the best values achieved for