Combining Images, Geometry, and Topology for Medical Data Processing and Analysis

To process and extract knowledge from biomedical data, we aim to investigate three levels of abstractions: images, geometry, and topology, which provide information from fine to coarse granularity. Images provide fine-granularity information in the form of pixel grids, which are frequently studied by convolutional neural networks for extracting visual features from medical images. Geometrical models, such as graphs, point clouds, and manifolds, summarize important relations and structures derived from images. For instance, graph models capture pairwise relations of cells/nuclei, while point cloud models can describe tubular structures of capillaries at a sub-pixel level. Topology offers a global perspective on probability manifolds, data distribution, arrangement, and geometric patterns. By combining multiple levels of abstraction, we can gain a deeper intuition of the structure of medical objects, achieve a more comprehensive understanding of biomedical knowledge, and develop more powerful deep learning algorithms. In our study, we explored all these abstractions.

First, I examined pathology imaging condition distribution in a latent space, to guide a model in generating hard and underrepresented samples for data augmentation. This method allows limited labeled data to better cover diverse image characteristics resulting from various imaging conditions. Second, I utilized the acyclic and hierarchical topology of the gene ontology graph to design a more accurate algorithm for embedding knowledge graphs. This algorithm enables more precise predictions of protein similarity. Third, I propose CCF-GNN, a unified graph-based framework that integrates image, geometry, and topology to jointly examine the appearance, relations, and distribution topology of cell instances and cell communities. This algorithm significantly improves the accuracy of pathology classification across six different tissues. Fourth, in the retina vessel classification task, I extracted the vessel structure by observing the ridge manifold on the probability map. I then developed a point-cloud-based model for its flexibility in describing the tubular structure of arteries/veins. This approach greatly enhances the correctness of the vessel tree topology, which is crucial for downstream vessel analysis. All the developed methods have been evaluated on multiple datasets and consistently improve performance.