An Artificial Intelligence Framework for Universal Landmark Matching and Morphometry in Musculoskeletal Radiography

Dennis Eschweiler1*, Eneko Cornejo Merodio1, Felix Barajas Ordonez1, Aleksandar Lichev1, Nikol Ignatova1, Marc Sebastian von der Stück1, Christiane Kuhl1, Daniel Truhn1, Sven Nebelung1*

1Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Aachen, Germany
*{deschweiler,snebelung}@ukaachen.de

Paper Code Team Department

Abstract

This study presents a zero-shot artificial intelligence (AI)-based approach for automated landmark matching in radiographs, designed to improve scalability and reliability in deriving musculoskeletal (MSK) measurements across various anatomic regions and conditions. Key anatomic landmarks were manually identified under expert supervision, building the basis to automatically derive pertinent angle and distance measurements. Using one patient's radiograph as a reference, a robust AI-based feature matching approach establishes dense matches between the reference and unseen target images. Reference landmarks are transferred along the dense matching, resulting in precise landmark placement and automated calculation of MSK measurements within an entire patient cohort. The results demonstrate versatile and robust application without requiring anatomy-specific training, while remaining stable with in the presence of orthopedic implants.

From Head to Toe: A Single AI Framework for Universal Application

Artificial intelligence (AI) has shown great potential in assisting radiologists with musculoskeletal (MSK) assessments across various anatomical regions1,2,3,4. However, most existing tools are narrowly tailored for specific anatomies or rely on segmentation-based methods as a basis. To address these limitations, we propose a versatile and generalist AI-based approach for landmarks matching. In contrast to typical generalist landmark matching approaches that depend on automatically identified, optimally descriptive landmarks5,6,7, our use-case has to be based on manually selected landmarks tailored for MSK measurements. As these landmarks are potentially subotimal for precise matching, we employ an AI-based method8 that first performs multi-scale dense image matching between the reference and target images independently of the landmarks. The AI uses a combination of transformer and convolutional neural networks to establish a multi-resolution descriptive feature representation of each position within the images, allowing to define correspondences between the reference and target images. In a subsequent step, landmark positions are transferred to the target image based on these correspondences, with the determination of final locations benefiting from the redundancy of the dense matching. This enables robust automation of landmark matching from a reference patient across an entire cohort of target patients. The approach is anatomy-agnostic, enabling the use of the same backbone model for a wide range of anatomical regions and MSK measurements. For more detailed information we refer to our paper.

Interactivity: Real-Time AI-Assisted Landmark Matching

Once a correspondence between the reference and target radiographs is established, landmark matching can be performed in real-time. This process enables the alignment of individual points and allows for the definition of a predefined set of landmarks that can be mapped onto the target patient. These predefined landmarks can be customized for the automated calculation of specific measurements, which, once defined on a reference patient, can be consistently and automatically transferred to any radiograph within the target cohort.

Hover your mouse over the video to stop automatic transitions.

Behind the Curtain: Visualizing the Inner Workings

The process of establishing a dense match between two images relies on abstract feature representations for each position in both the reference and target images. To better understand which information is encoded by the AI, we examined the feature similarities between a specific position within the reference image and the entire target image. Our findings show that the AI captures both semantic and texture features. The matching process utilizes this encoded information, along with precise positional data, to create a reliable mapping between the images.

Hover your mouse over the image to stop automatic transitions.

From Novice to Expert: Annotation Through Imitation

Our findings indicate that the selection of the reference radiograph significantly influences the outcome, as each radiograph is unique and often presents highly superimposed structures. This can introduce ambiguity regarding the precise position and the intended anatomical structure to be marked by the landmark. To enhance the reliability of the matching process, multiple reference radiographs can be used to establish a more robust alignment based on overall agreement. Increasing the number of reference radiographs systematically improves the framework’s ability to account for variations between radiographs, leading to a steady improvement in matching accuracy as demonstrated by the results below.


BibTeX


@article{eschweiler2025RadiographMatching,
  title={An Artificial Intelligence Framework for Universal Landmark Matching and Morphometry in Musculoskeletal Radiography},
  author={Dennis Eschweiler and Eneko Cornejo Merodio and Felix Mauricio Barajas Ordonez and Aleksandar Lichev and Nikol Ignatova and Marc Sebastian von der St\"{u}ck and Christiane Kuhl and Daniel Truhn and Sven Nebelung},
  year={2025}
}