Research Session · Analysis 1.0

Accompanied by Dr. André Weinreich, Head of Research & Science from emolyzr/Humboldt University, I’ve run several sessions during the winter semester 2016/17. We now have a gathered some data for the movies which had have enough attendees.

The numerical data provides two main starting points: either analysing the questionaire or the plain heart rate data. We check several options to find some correlations between them. The first obvious result is the relation of mean heart rate and liking. At present we focus on cluster analyses from both “ends”: the average rating as well as the individual pulse.

These are the mean heart rates of each screened movie in alphabetical order. In this earlier post some reasons are given, why these films were chosen. Most movies have been screened in a lecture hall, some at a regular cinema, which clearly caused immersiveness at different levels.

Some clues to read the data:

  • Have a look which approx. mean heart rate the movie evokes
  • Watch for immediate changes (up or down)
  • Look for sections which continously differ from mean heart rate
  • The dynamic range, the film has caused in general, is an indicator as well
  • Can an overall trend be identified in individual sections or the entire film?

American Psycho

Mary Harron · 2000 · n=16 (Berlin, Lecture Hall)

American Psycho Poster

American Psycho Graph


Awakenings

Penny Marshall · 1990 · n=16 (Lemgo), n=14 (Berlin) both Lecture Hall

Awakenings Poster

Awakenings Graph


Deadpool

Tim Miller · 2016 · n=16 (Lemgo, Lecture Hall)

Deadpool Poster

Deadpool Graph


Delicatessen

Jean-Pierre Jeunet & Marc Caro · 1991 · n=14 (Lemgo), n=16 (Berlin) both Lecture Hall

Delicatessen Poster

Delicatessen graph HR mean


Doctor Strange

Scott Derrickson · 2016 · n=16 (Lemgo, Cinema)

Doctor Strange Poster

Dr Strange Graph


Fantastic Beasts and where to Find Them

David Yates · 2016 · n=15 (Lemgo, Cinema)

Fantastic Beasts … Poster

Fantastic Beasts Graph


Gone Girl

David Fincer · 2014 · n=15 (Berlin, Lecture Hall)

Gone Girl Poster

Gone Girl Graph


Heil

Dietrich Brüggemann · 2015 · n=16 (Berlin, Lecture Hall)

Heil Poster

Heil Graph


Her

Spike Jonze · 2013 · n=15 (Berlin, Lecture Hall)

Her Poster

Her Graph


Labor Day

Jason Reitman · 2013 · n=16 (Berlin, Lecture Hall)

Labor Day Poster

Labor Day Graph


Legend

Brian Helgeland · 2015 · n=16 (Lemgo, Lecture Hall)

Legend Poster

Legend Graph


Passengers

Morten Tyldum · 2016 · n=14 (Lemgo, Cinema)

Passengers Poster

Passengers Graph


Robocop

José Padilha · 2014 · n=10 (Lemgo), n=16 (Berlin) both Lecture Hall

Robocop Poster

Robocop Graph


Schönefeld Boulevard

Sylke Enders · 2014 · n=15 (Berlin, Lecture Hall)

Schönefeld Boulevard Poster

Schoenefeld Boulevard Graph


Stereo

Maximilian Erlenwein · 2014 · n=15 (Berlin, Lecture Hall)

Stereo Poster

Stereo Graph


Zootopia

Byron Howard/Rich Moore · 2016 · n=14 (Lemgo, Lecture Hall)

Zootopia Poster

Zoomania Graph


The analysis’ goal is to find some structure in observing the heart rate of movie goers. One outcome might be the relation between the heart rate and the likness. Because we are still in the process of applying different approaches, we are not yet ready to publish the whole data.

If you are interested to get more details of this movie analysis study, it’s progress in analysis, or how Movie Pulse can be used for analyzing any feature length movie yourself, get in touch.

The Rocky Horror Picture Show seen by five study volunteers

“The Rocky Horror Picture Show” · Lecture Record #3

This has been the most “shocking” movie for my students so far. At least three of them could not stand it’s b-movieish style, story and characters. But most of them watched it completely and some of them had have fun. But I assume this has been the minority. The concept of the movie will not work nowadays: to board a loved genre of the former generation (musical) and to whisk contemporary no-no’s, controversial moral issues and sexual frankness into a hym of individuality. My subjective appraisal on this consists of two motivations: 1st – this is not their music, therefore it’s just another musical for them; 2nd – society tends to prefer conservative ideas at present, even the younger are part of this trend. The sexual individuality and freedom of personal choice is still not guaranteed – even 41 years! after RHPS has been released.

We still have not achieved absolute pleasure.

Little Conformity

Although this is still not a valid test scenario in terms of quantity – I’d like to apply the former developed strategies for data comparison to this little data set again. This will help to achieve a valid outcome on a later larger scale of study volunteers.

The Rocky Horror Picture Show · Amount of aberration
The amount of aberration has been quite massive (light grey) and the less than average aberration little (dark grey)

Compared to former records this movie has raised very divers reactions and a small amount of similar tendencies. Just to recall: high values of the light grey depict regions where the subjects tend to have no comparable heart rate. The dark grey regions show the time spans of more similar reactions.

The Rocky Horror Picture Show · Accordance of 100%
Accordance of 100% percentage in red, calculated average in grey.

The absolute identical reactions of alle participants in terms of above or below the base line therefore have been very short and few (8,9 %).

Majority will make the Difference

The Rocky Horror Picture Show · Accordance of majority
The accordance of the majority (3 out of 5) compared with 100% accordance (black)

Five participants can have a majority of 60%. So 3 out of 5 show more similar reactions in high pulse (red) and low pulse (green). The 100% accordance is included in black, since this is a subset of the majority calculation.

Gender Comparison

The Rocky Horror Picture Show · Comparison of gender accordance
Comparison of gender accordance (male blue and female orange)

This comparison is new – almost new. I used it in “The Angry Birds Movie” to differentiate between the parent and the children reactions. I will make this a standard comparison: building subset groups of the study volunteers – the most obvious is gender. As long as we will have a fifty-fifty arrangement. And of course: this is only a valid result, if based on a relevant group size. Having 2 female and 3 male will not really suffice. But for building the tool set, we’ll use it for now. 54,4% of the female and only 28,7% of the male have had identical reactions within their group.

The Rocky Horror Picture Show · Gender difference and accordance
Accordance of all study volunteers (green) and regions where female and male reacted differently compared to the other gender (red)

This will improve the depicting of the differences for sub group comparison: you can easily compare the green amount of the movie, which caused similar reactions on all participants (8,9 %), with the amount of differences between the two groups: red shows the parts (51,7%) where female and male reacted similar within their own group (sex), but different to the other gender.

Visualization

The video depicts again the average threshold of the majority (60% this time) – the image becomes tinted in green and red. To make this work more obvious I have destained the movie beforehand. Additionally the 100% subset of all four test users has been included as well. To distinguish both data visually, a hint for the 100% subset has been added in white. If one image shows a white frame: this means all viewers have reacted the same way, not only the majority. The graph below the timeline shows this subset in white as well.

"The Angry Birds Movie" · A Family Record

“The Angry Birds Movie” · A Family Record

We had a long awaited trip to the cinema with our two boys, age 6 the little and the older one became 8 years old just recently. They have watched so many Angry Birds Shorts and now wanted to see the feature movie. When I asked them why, they could not really state more than “it’s fun”. I’ve read a very short review stating it’s a sequence of gags only. Not a good starting point for parents.

But to make an ease start: we’ve had fun watching it. Recording the pulses of all family members had brought some prove that these kind of movies are build on two major layers (if we disregard the fact that we four are not a qualified test screening quantity).

I won’t go into detail on the movie itself. It’s a merely simple world and characters, compared to “Zootopia” for example. But the richness of references to contemporary and history of media has a certain quality. We, the parents, had often laughed out loud: Terence can be seen as a revenant of Chief Bromden from One Flew Over the Cuckoo’s Nest” and you can meet the female twins of “Shining” as well as you can party with Daft Punk. Not to mention that Rick Astley’s “Never Gonna Give You Up” will be played at a central moment for the (former) hero Mighty Eagle. And the main character Red has a interesting personality and a goal; presumably this movie has a story after all.

100% accordance (green) · 100% accordance of the parents (grey) · 100% accordance of the childs (blue) · discrepancy of parents vs. childs (red)
This is my major outcome of inspecting the results of the Family Record: red
depicts the time children and parents reacted different while watching the movieThe following steps will explain how this result has been achieved. If you’re not familiar with the workflow I developed so far, I recommend reading the “Citizen Kane” post first, because I’d like to refer to some steps with not going into detail again.

All four records stored in the Movie Pulse app
All four records stored in the Movie Pulse app

All records as relative values on their main average, which refers as base zero.
All records as relative values on their main average, which refers as base zero
All of the four records of the family members are changed to relative values, based on their single average. Parents records have a warm color, the records of the children are shown in cold tones.

The aberration of all family members (light grey). Higher values show greater discrepance. Dark grey depict areas of less than average aberration.
The aberration of all family members (light grey) and less than average aberration (dark grey)
Higher values show greater discrepancy. Dark grey depict areas of less than average aberration, which means the majority of viewers tend to have the same reactions, either above or below their main average.

Accordance of 100% percentage in red, calculated average in grey.
Accordance of 100% percentage in red, calculated average in grey.
The red line shows the moments of the film when the complete family has reacted in the same manner: all four have had either a raised pulse above zero, or had have a calm pulse below zero. The amount of accordance over time is 17,6%. But I now start a little clustering and interpretation.

The accordance of the majority (3 out of 4) compared with 100% accordance (black)
The accordance of the majority (3 out of 4) compared with 100% accordance (black)
As I stated in the beginning: I refer to the same procedure I developed for “Citizen Kane”: besides the finding of 100% accordance, I am looking for the areas of conformity of the majority of viewers (3 out of 4). This shows a greater pattern of resemblance over time: a high result of 68%!

But this time I will try something new: I’d like to match the differences of the children’s outcome with those of the parents!

The results are divided into the subcategories “Parents” and “Childs” and their results are computed separately. This proofed the idea that watching “The Angry Birds Movie” triggered both groups in different ways.

100% accordance (green) · 100% accordance of the parents (grey) · 100% accordance of the childs (blue) · discrepancy of parents vs. childs (red)
100% accordance
 ·100% accordance of the parents ·100% accordance of the children ·discrepancy of parents vs. childrenThe green row is the result we’ve had in different styles beforehand: 100% accordance (17,6 %) of all family members. Grey is used to mark the timespan when both parents reacted similar (astonishing 57,3%), blue shows the conformity of both boys (an even higher result of 58,8%) and finally red discloses disparity between the parents and children’s reactions (50,4%).

Citizen Kane · Orson Welles · 1941, Watched and recorded with 4 test persons

“Citizen Kane” · Lecture Record #2

We are one step closer to our general aim. For identifying structures within movies we analyze the heart rate of the viewers with our Movie Pulse app. Because of a generous support we’re now able to record up to 5 viewers in parallel! For a first start with multiple Apple Watches we’ve had four volunteers watching “Citizen Kane”. All Watches are paired to a single iPhone 5 and committed their data without any hassle. The data shows no gaps, missing periods of recording. Only a small exception of being perfect: one record stopped prior the end of the movie. Perhaps the user inadvertently pressed the stop-button. Therefore the last 13 minutes are not having the base of 4 recordings and are less significant.

Having watched “Citizen Kane” is an important requirement in terms of understanding the history of cinema. That motion pictures are able to condense what went wrong in an entire single life with a single ordinary object – this really strikes me every time I see this movie. I don’t want to go into details here. “Rosebud” is all I have to cite.

Raw data of four test persons watching "Citizen Kane" the same time
Raw data of four test persons watching “Citizen Kane” the same time

If we now have a look onto the raw data of our Movie Pulse volunteers – my first reaction has probably been the same than yours: Am I on the right track? For heaven’s sake, how could this mess matter in any meaning?

They all must have seen another movie or the whole Movie Pulse enchilada is for the birds. Wait. The data needed some nurture.

Absolute figures are nonsense

We need to have a look at each recording individually. Each person has his/her own resting heart rate. We don’t know this rate, but we can calculate the overall average for the movie records of this person. This figure along with the minimum and maximum pulse in this period of time is the starting point of all our computations. In the Movie Pulse app there is a far more sophisticated algorithm which computes not only the overall average, but the average changing within a specific time frame. This gives the nice up and down trend line. Having here the plain average will be sufficient for now.

The same data all vertical aligned at their average. Note the absence of absolute figures of the heart rate.
The same data vertical aligned at their average. Note the absence of absolute figures of the heart rate. The colors have been altered to a warm tone for female, cold for male test users.

Adjust the base

What formerly has been the average will now serve as the base – zero null. The numbers of units from average to the lowest will represent the minimum range of units the person has reached. The maximum units are defined the same way above zero. Instead of having a heart rate, we deal with a range of units at a base of zero – the threshold of maximum and minimum our test person has reached at a particular time within the movie.

Amount of deviation

A first attempt is to identify the amount of difference at each recording time. Let’s say at time 0:32:12 person 1 has a value of 4, person 2 a value of 6, person 3 a value of 7 and person 4 a value of minus 4. Three people are above their individual average and one person is below it. The deviation is 4,43…. somehow complicated, therefore computers come in very handy. Two more simple examples: p1 value 1, p2 value 1, p3 value 1 and p4 value 2 will have a deviation of 0,5. Yes 1,1,1,1 will gain 0,0 – no difference at all. Low deviation will mean great conformity – which, in our analysis, is the biggest proof of having viewers responding similar.

Determining the deviation compares the data and will give a guidance: low deviation means users reacted more similar.
Determining the deviation compares the data and will give a guidance: low deviation means users reacted more similar.

To make this graph more readable I added a simple TRUE value for those recordings which have a small deviation (average or below), which is represented by the darker gray. This is a first hint for periods of time which have gained equal reactions.

Direction is all we need

But there is one problem: How about the differences the test users have in terms of range of threshold? Of course, according to my theory, a very high or low value will state either high attention or intense state of being calm. But this is the peak we might come closer soon; for now it’s more elementary and useful at which moments the viewers responded in the same way, the same direction!

The simple overall average of all records is depicted in grey. The red line shows the period of time all four users had have data within the similar area: either above or below their own zero base.
The simple overall average of all records is depicted in grey. The red line shows the period of time all four users had have data within the similar area: either above or below their own zero base.

I came to the conclusion that having an overall average of all four users might not sufficient and will alter the data in a wrong way: Imagine one person reacting completely different. With pure math average his or her data will be taken into account nevertheless it might be completely different than the majority. This is what we’re after: majority.

The resulting graph shows data where the majority (three out of four, 75%) has had similar data either above (red) or below (green) the zero base. 100 % similarity is shown in black.
The resulting graph shows data where the majority (three out of four, 75%) has had similar data, either above (red) or below (green) the zero base. 100 % similarity is shown in black.

For this reason I computed the graph depicting the average raise or descent of A) all four test persons B) of the majority of 75% of the test persons which is shown in red and green. Therefore A) is a subset of B) and is marked in black on this image (white in the video).

Again the movie is boiled down into 2 minutes excepts, placed beside each other; one row representing 30 minutes of the movie.

Those 100% and 75% conformity results have been used in the “Citizen Kane in 2 minutes” movie. The overlay of green and red color is applied at the same rules as the prior published “Metropolis in 2 minutes”: high units above the average will tint the image in red, low units in green. This time the color is multiplied with the image, not overlayed.

The video takes the average threshold of the majority (75%) into account – again the image becomes tinted in green and red. Additionally the 100% subset of all four test users has been included as well. To distinguish both data visually, a hint for the 100% subset has been added in white. If one image shows a white frame: this means all viewers have reacted the same way, not only the majority. The graph below the timeline shows this subset in white as well.

Please remember that this first multiple recording unfortunately is not complete approx. 13 minutes prior the movie ended. So “n” has changed from four to three persons at 1:39:12. There is always room for improvement. Any sugestions are welcome too! 

Metropolis · Fritz Lang · 1927

“Metropolis” · Lecture Record #1

This time I’d like to start to refer to movies which are important in film history and have a certain age. “Metropolis” is the mother of science fiction – besides the Georges Méliès Le voyage dans la lune” (1902!). Numerous experts have already debated about it’s value, impact and whatsoever outcome in vision, theme and cinematic morphology. One thing I find interesting myself is the fact that the author Thea van Harbou has had a different aim than the director Fritz Lang: I remember a statement on the extras of the restored version of Enno Patalas – Lang was mainly fond of the human-machine setting; of machines in general, whereas van Harbou focused the idealistic approach that the heart has a mediator function between hands and the mind.

Metropolis · Fritz Lang · 1927
Metropolis · Fritz Lang · 1927

I showed this movie to my students and picked a random volunteer wearing the Apple Watch for running Movie Pulse for the length of the film.

Again: this is a individual result and can not validate the general impact of the movie.

But beside watching and exploring heart rates of movies myself, giving lectures, thinking of app improvements and features, raising comparable results of the same movie –  I think it’s quite important to achieve some visual experiments on depicting the heart rate graph along with the movie itself. Without legal infringements.

A timeline of "Metropolis" along with the Movie Pulse result of a volunteer.
“Metropolis” timeline along with the Movie Pulse result of a volunteer. (Click to enlarge)

This approach seems very inefficient, besides the less detailed graph and the quite small images. So why not combining the result with the movie itself?

  1. combine the heart rate with the images of the movie
  2. doing this for the length of the movie
  3. depict the change of beeing calm/exited
  4. provide an overview

Those ideas led to the 2 minute movie you can see here:

Example: Finding the sequence at 0:39
1st row = 30 minutes + 2nd row 5th image = minute 0:38 – 0:40

Each row shows 30 minutes of the movie. 2 minutes excerpts are placed side by side, so 15 excerpts are depicting 30 minutes, 4 rows showing the almost 2 hours. The graph below the images is just an orientation – since the green and red tint has been rendered slightly different. I determined an overall average – this refers as base zero for red and green. The max value yields 100 % red, min value provides as 100% green.

Heart rate of the movie "The Hateful Eight" by Quentin Tarantino · 2015

“The Hateful Eight”

I have been persuaded by friends of mine – this movie was not my choice. The trailer already gave me the feeling that this would be a long and boring time in cinema. And I should go with my instincts.

But after having recently so many movies I was fond of – I have to admit it was somehow curiosity. If things go that bad: How will the Movie Pulse look like, of a film I don’t like? According to my feelings I went near sleeping pulse after the opulent and impressive introduction of the characters and the setting. Only the visuals by Robert Richardson kept my attention, I was basically trying not to be disturbed by the loose tongues of the cast. I slightly had the impression having a bath tube shape of heart rate: great expectations, low outcome in the middle and an effectuated raise at the end.

Btw. What did Tarantino say to Tim Roth? Look at Christoph Waltz characters in my recent movies and copy what you like? But please don’t be that good?

If we now have a look at the graph, one must say I was somehow wrong, but not completely. There is a longer increase time at the beginning, a longer decrease time at the end and you can see the valley in the middle, which is not that extraordinary as expected. But the “waves”, the trend line my heart rate performs, could be interpreted as turning points: having a raise until approx. 0:47 – then going down until 1:30 to catch my attention up to 2:20. The “Schlachteplatte” at the end could not attract interest for the rest of the movie. 2 hours and about 45 minutes of my lifetime I could have spend with movies which really matter. Again: the DOP helped making this bearable.

This is a single, subjective and yes, prejudiced interpretation, which points out the future of Movie Pulse: there must be a solution to anonymously compare graphs of other people which have seen the same movie. A solution to render an average graph of multiple sources and compare your own with that.

How would a Tarantino enthusiast’s graph look like, in opposite?

Heart rate of the movie "The Hateful Eight" by Quentin Tarantino · 2015
The Hateful Eight · Quentin Tarantino · 2015

First Comparison · Star Wars 2015

Here comes a first result to compare the heart rate of two different people watching the same movie. Both persons have seen the movie in different locations and at different times (Person 1 & 2).

At first I put graph one onto the other – regardless of the absolute heart rate, which resulted in a centered position in y, which is the pulse (Person 1 +2 Overlay). Because of the visual confusing outcome I decided to identify similarities via simple color multiplication (Similarities [Overlapping]).

This is my theory what this result can tell: In those regions which have overlapping color (red + red or green + green) both persons have had the same kind of pulse: either above the trend line (red) or below (green). This means both persons have been emotionally touched in the same manner. So the bars in grey depict those time frames, where both individuals have been responded similar – although they have seen the movie in different locations and at different day time (and of course at different dates).

Person 1 fails keeping attention after 30 minutes – it’s me, folks and this is why: I went to the cinema in the afternoon, when I usually have my energy drop.

Comparison of two people seeing Star Wars: Episode VII - The Force Awakens · J.J. Abrams · 2015
Comparison of two people seeing Star Wars: Episode VII – The Force Awakens · J.J. Abrams · 2015