An article published this summer seeks to investigate the diagnostical reliability of heat map explanations based on machine-learning models. The article is a product of the Data Science Spring School, which saw researchers from both clinical and computer scientist backgrounds collaborating.

In the article titled Usefulness of Heat Map Explanations for Deep-Learning-Based Electrocardiogram Analysis published in Diagnostics, researchers from different disciplines sought to investigate the usefulness of machine-learning models by developing a deep neural network to predict sex from electrocardiograms.

The article is the result of group work from last year’s Data Science Spring School – a data science course for PhD students, organised by both the Danish Diabetes Academy (DDA, now DDEA) and the Danish Data Science Academy (DDSA) to facilitate and strengthen interdisciplinary collaborations.

Ole Emil Andersen, a PhD student at Aarhus University and Steno Diabetes Center Aarhus with a grant from DDA, is one of the authors of the article.

What was your experience like with the Data Science Spring School?

Ole Emil Andersen: “For me, the Data Science Spring School was a great learning and networking experience. The mix of both participants and speakers from diverse backgrounds was exciting. During the course, we were placed into groups of six students. All groups were given the same challenge: Make a prediction model based on ECG data that utilise AI. In our group, we were six people out of which two had a data science background: Andrea Storås from Norway, Roman Thielemann from Denmark, and then three people with a clinical background: Sam Lockhart, UK, Filip Gnesin, Denmark, and me. The diverse backgrounds of the group members created an interesting group dynamic.”

Ole adds: “It must also be mentioned that a movie night watching the movie AlphaGo with popcorn and candy was a great touch to the course.”

The groups worked on their projects both at the boot camp and via remote work. What was this process like for you and your group?

Ole Emil Andersen: “At the boot camp, we spent quite some time brainstorming ideas on how to tackle the task we were given. We had a LOT of ideas, but we aimed at something that was both feasible for us to do and still had some interesting scientific perspectives.”

“Our plan was to make a model, that could predict sex based on electrocardiograms (ECGs). We did not have too much time to write code during the first boot camp, but we agreed to meet regularly after. This ended with 4-5 online meetings where we discussed the progress of the project, discussed how our models performed, and discussed the results we got. At the final physical meeting, we presented our results to the other groups. At this time, we had a model that was able to predict sex. After the final meet-up, Andrea Storås from our group came up with the idea that we could use explainable AI and have the clinical members of the team try to make sense of the explanations provided by the model.”

What were some of the most significant results of your project?

Ole Emil Andersen: “Our project succeeded in building a deep neural network that could predict sex based on ECGs with high accuracy. However, the most intriguing part came when we used explainable AI techniques, specifically Grad-CAM, to produce heat maps for understanding the model’s predictions. Despite the technical achievements, the feedback from those of us with a clinical background revealed a critical finding: the heat maps did not provide meaningful or clinically useful information. This outcome was a strong indicator that while deep neural networks show promise, there’s a need to develop more tailored explanation techniques before they can be reliably used for diagnostic purposes in a clinical setting.”

What was it like to meet up again at the closing event and hear about the other groups’ projects?

Ole Emil Andersen: “I think that reconvening at the closing event was enlightening. It was eye-opening to see how other interdisciplinary groups approached their challenges and applied data science methods. Hearing about their achievements and even the hurdles they faced only deepened my understanding of the versatility and applicability of data science.”

What do you think are some of the benefits of meeting and working with professionals from different disciplines?

Ole Emil Andersen: “Working in an interdisciplinary team presents numerous advantages. For one, it leads to more robust and innovative solutions, as team members bring diverse expertise and perspectives to the table. For instance, in our project, the collaboration between data scientists and clinicians allowed us to tackle the problem from both a technical and a medical standpoint. Also, such collaborations expand one’s network, thereby laying the groundwork for future interdisciplinary projects. Lastly, these partnerships serve as a catalyst for individual growth, as they expose team members to new skills and viewpoints they might not have encountered otherwise.”

First author: “To ensure safe systems, we need expertise from both disciplines”

Andrea Storås, whom Ole mentioned earlier, is the first author of the article Usefulness of Heat Map Explanations for Deep-Learning-Based Electrocardiogram Analysis. She is doing a PhD in computer science at Simula Metropolitan Center for Digital Engineering (SimulaMet) and Oslo Metropolitan University, working with explainable artificial intelligence (XAI) in medicine.

“The intersection between medicine and technology is something I find very interesting. As part of my projects, I meet and discuss AI models and model explanations with healthcare personnel,” says Andrea Storås.

Andrea also found great value in meeting researchers from mixed backgrounds at the Data Science Spring School.

“I met researchers from the medical field and computer science field and learned a lot from both domains. This was also true regarding the group project. In my group, there were three medical doctors and one computer scientist besides myself. I really felt that we were able to supplement each other and come up with a solution together,” says Andrea Storås.

When asked about the possible benefits of meeting and working with researchers from different disciplines, Andrea answers: “It boosts the learning experience and can make you see a problem from a different perspective. By combining knowledge from several disciplines, we can solve problems we would not be able to solve on our own. I believe data science and medicine are good examples where this is the case. To ensure safe systems that solve real challenges in the clinic, we need expertise from both disciplines.”

Read more about the Data Science Spring School experience here.

Usefulness of Heat Map Explanations for Deep-Learning-Based Electrocardiogram Analysis

Authors: Andrea M. Storås, Ole Emil Andersen, Sam Lockhart, Roman Thielemann, Filip Gnesin, Vajira Thambawita, Steven A. Hicks, Jørgen K. Kanters, Inga Strümke, Pål Halvorsen and Michael A. Riegler.

Diagnostics 2023, 13(14), 2345; https://doi.org/10.3390/diagnostics13142345