Publications

In this paper, we present an information-theoretic method for clustering mixed-type data, that is, data consisting of both continuous and categorical variables. The proposed approach is built on the deterministic variant of the Information Bottleneck algorithm, designed to optimally compress data while preserving its relevant structural information. We evaluate the performance of our method against four well-established clustering techniques for mixed-type data – KAMILA, K-Prototypes, Factor Analysis for Mixed Data with K-Means, and Partitioning Around Medoids using Gower’s dissimilarity – using both simulated and real-world datasets. The results highlight that the proposed approach offers a competitive alternative to traditional clustering techniques, particularly under specific conditions where heterogeneity in data poses significant challenges.

Outlier detection is an important data mining tool that becomes particularly challenging when dealing with nominal data. First and foremost, flagging observations as outlying requires a well-defined notion of nominal outlyingness. This paper presents a definition of nominal outlyingness and introduces a general framework for quantifying outlyingness of nominal data. The proposed framework makes use of ideas from the association rule mining literature and can be used for calculating scores that indicate how outlying a nominal observation is. Methods for determining the involved hyperparameter values are presented and the concepts of variable contributions and outlyingness depth are introduced, in an attempt to enhance interpretability of the results. The proposed framework is evaluated on both synthetic and real-world data sets, demonstrating comparable performance to state-of-the-art frequent pattern mining algorithms and even outperforming them in certain cases. The ideas presented can serve as a tool for assessing the degree to which an observation differs from the rest of the data, under the assumption of sequences of nominal levels having been generated from a Multinomial distribution with varying event probabilities.

Clustering mixed-type data, that is, observation by variable data that consist of both continuous and categorical variables poses novel challenges. Foremost among these challenges is the choice of the most appropriate clustering method for the data. This paper presents a benchmarking study comparing eight distance-based partitioning methods for mixed-type data in terms of cluster recovery performance. A series of simulations carried out by a full factorial design are presented that examined the effect of a variety of factors on cluster recovery. The amount of cluster overlap, the percentage of categorical variables in the data set, the number of clusters and the number of observations had the largest effects on cluster recovery and in most of the tested scenarios. KAMILA, K-Prototypes and sequential Factor Analysis and K-Means clustering typically performed better than other methods. The study can be a useful reference for practitioners in the choice of the most appropriate method.

Fuzzy clustering is used to identify overlapping geometric cluster structures through partial memberships. However, classical methods are limited by the assumption of equal variable importance and by sensitivity to the fuzzifier parameter. These limitations may yield equal cluster membership probabilities, which we refer to as the uniform solution. To address these issues, we propose Kernel Fuzzy Relational Clustering (KFRC) equipped with a bandwidth selection algorithm tuned via the spectral properties of the induced kernel Gram matrix. The KFRC framework implicitly performs unsupervised kernel metric learning by controlling the geometric embedding of the data through adjustable bandwidth parameters. We conduct a formal stability analysis to identify the exact theoretical conditions under which relational clustering collapses, thereby ensuring the stable performance of KFRC. We find that our two-stage bandwidth selection procedure adapts to the data structure while actively avoiding the uniform solution. Furthermore, this theoretical analysis leads to the proposal of a novel fuzzifier function that presents distinct advantages over the power fuzzifier function. We conduct experiments on several synthetic and publicly available data sets to demonstrate that the proposed framework consistently recovers complex structures that traditional methods fail to resolve, while ensuring a purely fuzzy solution.

We present an outlier identification method for mixed type data sets comprising continuous and ordinal variables. We define outliers based on using a multivariate Gaussian distribution as reference distribution for non-outliers, with a latent Gaussian assumed for ordinal variables. The proposed algorithm is based on the robust Minimum Covariance Determinant estimator for estimating the parameters of the multivariate Gaussian for the non-outliers. This is extended to account for the fact that the full Gaussian information underlying the ordinal variables is not observed. A breakdown theorem shows that replacing observations will noty stop extreme enough outliers from being identified. The effectiveness of our approach is demonstrated via simulations on synthetic data with various types of contamination, achieving high detection and low false positive rates. Practical relevance is illustrated through an application to Airbnb listing data containing both continuous and ordinal attributes.

Cluster analysis relates to the task of assigning objects into groups which ideally present some desirable characteristics. When a cluster structure is confined to a subset of the feature space, traditional clustering techniques face unprecedented challenges. We present an information theoretic framework that overcomes the problems associated with sparse data, allowing for joint feature weighting and clustering. Our proposal constitutes a competitive alternative to existing clustering algorithms for sparse data, as demonstrated through simulations on synthetic data. The effectiveness of our method is established by an application on a real-world genomics data set.

In this paper, we present an information-theoretic method for clustering mixed-type data, that is, data consisting of both continuous and categorical variables. The method is a variant of the Deterministic Information Bottleneck algorithm which optimally compresses the data while retaining relevant information about the underlying structure. We compare the performance of the proposed method to that of three well-established clustering methods (KAMILA, K-Prototypes, and Partitioning Around Medoids with Gower’s dissimilarity) on simulated and real-world datasets. The results demonstrate that the proposed approach represents a competitive alternative to conventional clustering techniques under specific conditions.

Mixed-type data sets, that is, data sets consisting of continuous and categorical variables, are prevalent across multiple sectors, such as healthcare, finance, and social sciences. While many supervised methods account for this heterogeneity in the data, unsupervised techniques that rely on a Euclidean intuition are not directly applicable to non-continuous features. This underscores the need for unsupervised learning techniques which can handle categorical, as well as mixed-type data in a flexible manner. This thesis presents novel methodological developments in two fundamental tasks in unsupervised learning; cluster analysis and outlier detection.

The first part of the thesis explores existing non-model-based algorithms for clustering mixed-type data. It identifies the main strengths and weaknesses of specific methods and provides a taxonomy of eight state-of-the-art algorithms. Moreover, the most influential data set characteristics to the clustering performance are investigated, thus motivating the development of a novel clustering framework. The proposed framework leverages ideas from information theory, highlighting the versatility of this approach in integrating different variable types to the clustering problem.

In the second part of the thesis, the focus is on defining and detecting outliers in discrete and mixed-attribute domains. A definition of outlyingness for unordered categorical data is provided and a framework for outlier quantification is devised. The presented framework reflects on the considerations implied by the given definition and employs concepts from association rule mining to assign scores of anomalous behaviour to observations. The case of mixed-type data is also explored by extending ideas from the robust statistics literature for the case of mixed continuous-ordinal data. This provides a statistically principled approach to the problem of outlier identification. The extension to nominal data and the challenges associated with the latter are thoroughly discussed. Overall, this thesis introduces novel techniques for unsupervised learning problems in the presence of categorical and mixed-type data.