Electroanatomic mapping reconstruction with photogrammetry across different mapping systems

Automatic digital photogrammetry produces digital reproductions of objects using photographs. The aim of this study is to analyze feasibility of photogrammetry for electroanatomic map (EAM) reconstruction from different mapping systems. Furthermore, the possibility to import the reconstructed EAMs in a common working space is evaluated. All consecutive patients undergoing EAM with one of the following EAM systems were screened for the study: (1) CARTO™; (2) Ensite™ X; (3) Rhythmia™; (4) Affera™ PRISM-3. All patient geometries were reconstructed from a video acquisition within the source EAM software. The video obtained was processed with Zephyr software and a dense point cloud was obtained. An image or sequence of images was selected to build a 3D mesh. At the end, the mesh was imported in the 3D graphics software Blender. A total of 24 EAMs from 24 patients were included in the study. All EAMs were reconstructed with success using photogrammetry from all 4 mapping systems assessed. The process time was ≈ 25 min. In particular, EAMs were as follows: left atrium (2 Carto; 2 Ensite; 5 Rhythmia; 2 Affera), right atrium (1 Carto; 6 Ensite; 3 Affera) and left ventricles (1 Carto; 2 Ensite). All the reconstructed EAMs were imported in Blender with success. They could be visualized in Blender and all the operations were allowed including moving EAMs in a common working space and EAMs overlap. This study demonstrated for the first time the possibility of realizing 3-D objects from digital video formats of different EAMs.

In the last decades, advances in medical technology have revolutionized the field of cardiac electrophysiology, offering new diagnostic and therapeutic tools for the management of cardiac rhythm disorders (Sy et al., 2012). Among these, electroanatomic mapping (EAM) systems are vital for accurately assessing heart electrical activity and are indispensable in diagnosing and guiding ablation procedures (de Groot et al., 2003). The CARTO system (Biosense Webster Inc., Diamond Bar, California, USA), developed by Biosense Webster, has continued to represent a milestone in three-dimensional cardiac mapping. Through the use of a catheter equipped with magnetic sensors, the Carto system offers detailed visualization of cardiac anatomy, with an average distance error < 1 mm, and electrical activity in real time (Gepstein et al., 1997). Boston Scientific's Rhythmia system (Rhythmia™, Boston Scientific Corporation, Cambridge, MA) introduced an innovative approach to high-density mapping. Leveraging advanced technology, this system stands out for its ability to acquire a large number of electrical data points, enabling detailed mapping and a better understanding of the heart's electrical dynamics (Viswanathan et al., 2017). Ensite system (St. Jude Medical Inc., St. Paul, Minnesota, USA), developed by Abbott, has established its position as a reliable and versatile EAM system. Its ability to integrate electrophysiological data without using fluoroscopy with anatomical images offers clinicians precise guidance during ablation procedures (Eitel et al., 2010).

EAM allows to project cardiac signals onto an anatomical shell. The result is an image integrating anatomical and electrical data. Three main mapping methods have developed over time (Koutalas et al., 2015). Contact- based electroanatomical mapping, where the 3D mesh is representative of the electrical activity acquired by the catheters through direct contact with the anatomical structure (Gepstein et al., 1997; Jiang et al., 2009; Wittkampf et al., 1999; Marchlinski et al., 2000; Sanders et al., 2005). Non-contact and Multi-electrode Mapping, which involves using an inverse mathematical solution of Laplace's law to project onto an already constructed 3D mesh, virtual unipolar far-field EGMs (Thiagalingam et al., 2004; Kumagai and Nakashima, 2009; Hindricks and Kottkamp, 2001). Non-invasive cardiac mapping where epicardial surface potentials of the heart derived from torso potentials and projected onto the cardiac mesh processed from images derived from the patient's CT scan (Rudy, 2010). However, different EAM systems have been developed by different companies. This implies a limited possibility of manipulation for the relevant clinical images and of cross correlation on a common platform.

Photogrammetry enables the creation of precise three-dimensional models of photographed objects. It utilizes several mathematical formulations as well as acquisition tools (Fritsch et al., 2021). Meshes are generated through SfM (structure of motion) methods through algorithms such as SIFT (scale-invariant features trans-form) that allow the identification of shared characteristics between images (Westoby et al., 2012). Nowadays, due to its evolution, Automatic Digital Photogrammetry is a science used in various fields (Donato et al., 2020; Reinoso et al., 2018; Bemis et al., 2014; Hampel and Maas, 2003; Talevi et al., 2023; Kucukkaya, 2004). In the medical field, our group was the first to describe the use of Photogrammetry for mapping reconstruction. We employed Photogrammetry to reconstruct and manipulate electroanatomic maps generated from non-invasive ECGs in Brugada syndrome, highlighting its diagnostic implications (Talevi et al., 2023; Kucukkaya, 2004; Talevi et al., 2022). This integration of advanced techniques offers unprecedented opportunities for comparative diagnostic and educational applications, bridging the gap between electrophysiological maps and detailed three-dimensional EAM models.

The aim of this study is to analyze feasibility of photogrammetry for EAM reconstruction from different mapping systems using video media material acquired consistently between the different systems. Furthermore, the possibility to import the reconstructed EAM in a common working space for the purpose of creating educational tools and for research purpose is evaluated.

Study population

All consecutive patients undergoing EAM with one of the following EAM systems were screened for the study: (1) CARTO™ system (Biosense Webster Inc., Diamond Bar, California, USA); (2) Ensite™ X system (St. Jude Medical Inc., St. Paul, Minnesota, USA); (3) Rhythmia™ system (Boston Scientific Corporation, Cambridge, MA); (4) Affera™ PRISM-3 system (Medtronic Inc, Minneapolis, MN). They were included if the following inclusion criteria were fulfilled: (1) Complete EAM of the cardiac chamber of interest; (2) High-density catheter mapping (Pentaray or Octaray for CARTO, HD-Grid for Ensite X, IntellaMap Orion for Rhythmia or Sphere-9 for Affera).

In this study, all different patient geometries were reconstructed from a video acquisition of the EAM within the source EAM software. Maps from both atria and ventricles were considered. The study complied with the Declaration of Helsinki as revised in 2013; the ethic committee approved the study.

Photographic report

The methodology used in this study is the extraction of multimedia information through the photogrammetric reconstruction of digital objects. The video format “.mp4” was chosen as the input data acquisition process to simplify and speed up the acquisition process. Videos were taken by making a 360-degree horizontal turn, around the EAM, starting from the Postero-anterior (or PA) view. If a geometric abnormality was seen, a map was also acquired vertically at 360 degrees after returning to the initial position at the end of the horizontal turn. The videos duration was according to the possibilities offered by the source software. The information obtained was processed with 3DF Zephyr 7.507 (3DFlow©) software. The data were loaded into the software in video format, from which the frames for 3D reconstruction were extracted in.jpg format.

Image processing by photogrammetry

Images were processed with Zephyr 7.507(3DFlow©). The final 3D volume can be created with this software that extracts a dense point cloud and then selects an image or sequence of images to build a 3D mesh (Figure 1). In addition, the software allows new calibrations to be added and previously performed automatic calibrations to be modified. Interactions are then created through an interface that offers selection based on camera orientation, category and preset. The program works semi-automatically. After loading the image, the Masquerade tool on Zephyr is used if necessary and the reconstruction parameters are set. Masquerade software is used if an error, like including the background in the geometry, occurs. The predefined categories are divided into Generic Objects, Aerial Photos, Urban Environment, Human Body, Superficial Scans and Vertical Structures. The most appropriate category can be selected according to the type of object to be processed. Presets are divided into Fast Scan, Basic Scan and Deep Scan. For the purposes of our study, reconstructions were carried out using the Deep Superficial scan option. When this did not produce satisfactory results or the software presented an error in the reconstruction preventing the procedure from being completed, the images were processed manually. The advanced parameters were used consistently for all map reconstruction as follows: Keypoint sensibility: low; matching type: default; Matching stage depth: high; Tie-points quantity: very high; Reconstruction engine: Incremental; Bounding box method: Discard outliers. At the end of the process, the software allows users to export the mesh in formats compatible with the 3D graphics software Blender 3.5 (Blender Foundation). Within Blender, different maps can be imported and managed simultaneously in a shared workspace, serving primarily as a visualization tool to enhance the presentation and interpretation of the data.

Statistical analysis

Statistical analysis was performed using JASP 0.14.1. 0 software (GNU Affero GPL). The statistical analysis conducted on the data focused on understanding the differences between the parameters of the various cardiac mapping systems under study. Shapiro-Wilk test was carried out for normality. Correlation analysis was applied to investigate whether linear parameter relationships existed between video length in seconds and the number of points in the scattered cloud under the condition of reconstruction type (Preset or manual use). The correlation analysis was performed using Spearman's coefficient. Finally, the use of the analysis of variance (ANOVA) was crucial to examine how the control variables could influence the relationships between the type of mapping system and the final number of points. Before performing the ANOVA, normality was assessed using Levene's test and to confirm normality, the Q-Q plot was also performed. Subsequently, post hoc tests were performed with Tukey's test.

Electroanatomic mapping characteristics

A total of 24 EAMs from 24 patients were included in the study. All EAMs were reconstructed with success using photogrammetry from all 4 mapping systems assessed. The whole process time was 25 min ± 15 min.

In particular, EAMs were as follows: left atrium, 11 EAMs (2 Carto; 2 Ensite; 5 Rhythmia; 2 Affera), right atrium, 10 EAMs (1 Carto; 6 Ensite; 3 Affera) and left ventricle, 3 EAMs (1 Carto; 2 Ensite), Figures 2–5.

The data of video and photogrammetric reconstruction are summarized in Table 1. The overall mean length of the video used was 16.70 s. The video used was shorter for Ensite system (10 s ±3.4) compared with Carto, Rhythmia and Affera (22 s ± 6.6, 22 s ± 1.7, 20.6 s ± 4.8, respectively, p < 0.001). However, after reconstruction, there was no difference between different systems in the number of scattered cloud points (p = 0.16), Table 1. The frame extraction operation was carried out within the Zephyr software, which allowed the extraction of a mean of 132.2 number of frames. In particular, the number of frames used was higher for Rhythmia system (203.6 ± 59.7) compared with Carto, Ensite and Affera (99.75 ± 40.2, 124.0 ± 40.3, 103.4 ± 5.2, respectively, p = 0.002), Table 1. Correlation analysis showed no significant correlation between video length and scattered cloud points (Spearman's rho = -0.022, p = 0.38).

Electroanatomic mapping reconstruction

All 24 EAMs were reconstructed with success. Out of 24 EAMs, 12 were reconstructed using the semi-automatic procedure with the presets offered by the Zephyr software. In particular, all 5 EAMs (100%) derived from the Rhythmia system, 2 out of 10 (20%) EAMs extracted from the Ensite system (2 right atria) and all the 5 EAMs (100%) derived from Affera system were reconstructed using the “surface scan” and “high detail” selection. For the remaining 12 electrophysiological maps, the reconstruction approach using presets proved to be ineffective as the results were not satisfactory in three-dimensional reconstruction as the software was unable to position the cameras used for the images correctly, causing a geometric distortion of the subjects. Therefore, the scattered clouds of the remaining eight electrophysiological maps derived from the Ensite system and the four derived from the Carto were reconstructed by manual selection of the reconstruction parameters. Three reconstructions of the electrophysiological maps derived from the Ensite software presented deformations due to background integrations within the mesh (Figure 6). While one mesh derived from Carto showed geometric deformations (Figure 7). In order to solve the problem, two approaches were applied: one using Masquerade and the other using a video with two acquisition directions, (horizontal and then vertical). In both cases, the error present in the reconstructions was corrected (Figures 6, 7).

All the reconstructed EAMs were imported in the Blender software with success, Figure 8. They could be visualized correctly on the Blender software and all the operations were allowed including moving the EAMs in the common working space and EAMs overlap, Figure 9.

The results of the current study can be summarized as follows: (1) Photogrammetry is feasible for EAM reconstruction from different EAM systems and different cardiac chambers using video media material and (2) The workflow described allows to import the reconstructed EAMs in a common working space.

The role of different mapping systems in cardiac photogrammetry

Cardiac EAM systems are crucial tools in the diagnosis and treatment of cardiac arrhythmias (Earley et al., 2006). Three-dimensional mapping systems combine electrophysiological data with anatomical information, allowing detailed visualization of cardiac electrical activity in a three-dimensional space. These systems provide physicians with a detailed map of the heart's electrical activity, aiding in diagnosis and guidance during therapeutic procedures such as ablation (de Groot et al., 2003) thus improving treatment strategies for cardiac arrhythmias. This study demonstrated the feasibility of EAM reconstruction with photogrammetry. All 24 maps analyzed, from different cardiac chambers, were correctly reconstructed. The results obtained with the methodology used in this study showed that it is possible to achieve significant results in 3D reconstruction using simple video media material. The input video was obtained from the source proprietary EAM software. In particular, EAMs from different systems could be reconstructed with success and also imported in a common working space. In this study, the four main cardiac mapping systems currently in use were assessed: Carto, Ensite, Rhythmia and Affera.

All these mapping systems can be used singularly in a patient. However, there is no software for cross evaluation between different maps from different companies. The current study is the first to describe an optimized workflow that allows to build a reproducible and easy platform for EAMs analysis, merging and comparison. To visualize and compare different EAMs, there are several 3D modeling software options that can effectively meet these needs. Autodesk 3ds Max is a prominent choice, renowned for its advanced toolset that is ideally suited for professional productions. However, its high cost can pose a significant barrier. Blender proved to be the optimal solution, due to its open-source nature and cost-free availability. Additionally, Blender's Cycles rendering engine efficiently leverages GPU acceleration to enhance the rendering process, which is crucial for visualizing complex meshes an essential aspect of EAM analysis.

The role of different software and the optimized workflow in cardiac photogrammetry

The software, Zephyr 7.507 (3DFlow©), was used to create 3D volumes. The use of Zephyr, allowed to create a three-dimensional mesh from videos and to extract frames. Due to technological advances, it is now possible to model objects of high precision and geometric complexity in 3D (Hess et al., 2014). The image-based photogrammetric approach (Remondino and El-Hakim, 2006) has advanced considerably with the development of computer vision techniques (Furukawa et al., 2010; Pollefeys et al., 2008), which are now recognized as automated methods for image orientation (Remondino et al., 2012) and three-dimensional reconstruction at various scales (Haala, 2013; Lafarge and Mallet, 2012). Although the feasibility of reproducing digital objects for medical applications by means of photogrammetry has been described (Hampel and Maas, 2003), this study is the first to achieve the results using video as input data for frame extraction and subsequent reconstruction of electrophysiological maps. In the previous study from our group, photogrammetry accuracy was evaluated using CT scan reconstruction as gold standard. The measurements of the photogrammetric reconstruction closely matched those of the CT-based reconstructions. The measurements obtained for the photogrammetric reconstruction and the CT scans showed a strong correlation, according to the data analysis with correlation coefficients of 1 for the dimensional evaluation of the aorta and 0.988 for the dimensional evaluation of the vena cava. The results showed for the Wilcoxon signed-rank test a p > 0.05, which indicates that the differences between the photogrammetric and CT-derived images were not statistically significant. In the current study, although the acquired data showed a difference between the materials collected by the different mapping systems, the analysis did not show any significant differences and it was possible to achieve reconstruction of all maps. It was found that the position identified by the camera software used to capture the multimedia information is a determining factor in the success of the three-dimensional reconstruction. Indeed, almost half of the maps were reconstructed by means of a consistent, but manual approach through the use of the advanced parameters offered by the software, as the use of presets did not allow the correct identification of the camera position.

Using specialized software such as Zephyr simplifies and accelerates the process of obtaining and processing multimedia information. Zephyr's 3DF photogrammetry software leverages advanced 3D reconstruction technology through a fully automated process. This feature is driven by a proprietary algorithm that effectively combines cutting-edge computer vision and photogrammetry techniques. The software's technical core consists of two primary components: SfM and “Multi-View Stereo” (MVS). During the SfM phase, Zephyr automatically determines the position and orientation of the cameras based on the uploaded images. Subsequently, in the MVS phase, it constructs a dense point cloud from the data gathered in SfM. Additionally, one of Zephyr's standout features is its use of photoconsistency-based optimization, a technique that enhances mesh details by extracting additional information from images. The study found that average image coverage for object reconstruction ranges from 65% to 87%, with optimal results between 80% and 90%. The current study achieved an average coverage of 76%, slightly below the optimal range, while a previous study had about 10% lower coverage but similar results. Masquerade software and topology optimization were used in both studies, proving an indirect impact on improving data utilization and point cloud accuracy. However, it is crucial to choose the correct process for processing multimedia material so that the software can correctly process the information received and process the data efficiently. Furthermore, the configuration of the software settings for the sparse cloud used to process the data is a critical factor for the success of the three-dimensional reconstruction. The analysis conducted showed a denser creation of a sparse cloud for the Rhythmia software, whereas for the Carto software the average of the points of the sparse cloud showed the lowest values among the systems analyzed. This occurrence may be due to the type of reconstruction performed. In fact, the reconstruction by preset identified on average a greater number of points for the four systems compared to the points identified by manual reconstruction.

The direct use of frames extracted from videos may not be sufficient for successful reconstruction. The absence of a real background that can provide reference points and taking video footage of the object too closely may lead to photocoherence errors that can lead to incorrect reconstruction of the geometry or inclusion of the background within the mesh. Tools such as Masquerade or the use of videos with different starting directions proved to be crucial for a successful reconstruction with the elimination of all geometric deformations. Special attention must therefore be paid to the reconstruction workflow to avoid geometric reconstruction errors or loss of electrophysiological information with the inclusion of the background in the reconstructed object.

This study demonstrated for the first time the possibility of realizing three-dimensional objects from digital object video formats of different EAMs. It also highlighted the importance of adequate subject coverage, the use of a sufficient number of frames and the choice of input data format for successful 3D reconstruction. Therefore, the successful integration of these technologies can significantly improve the accessibility and usability of electroanatomical maps in clinical settings. Future studies should be conducted with more EAMs and in particular with different mapped cardiac arrhythmias.