Tag: research

Methodology of a visualization

Introduction

Visual representations of data offer a quick way to express a lot of information. As the old adage goes, a picture is worth a thousand words. One of the facets of digital humanities research is providing information in the form of visuals: graphs, maps, charts, etc.

I was already writing up some notes on a visualization I was creating for the dissertation when I read this excellent blog post by Fred Gibbs (a version of a presentation at the AHA 2015). In this essay I think Fred accurately identifies the digital humanities field as one in need of stepping up to the next level. It is no longer enough to present visuals as humanities research, but it is time to start critiquing what is presented, and for researchers to start explicitly explaining the choices that went into creating that visualization.

With those thoughts in mind, I present the methodology, the decisions, and the visualization of over 200 deaths at the KZ Porta Westfalica Barkhausen, during a one year period.

A change is happening (at least for me) in how data is analyzed. I have a spreadsheet of over 200 deaths, with various information, death date, location, nationality, etc. The desire to create a visualization came from wanting to understand the data and see the commonalities and differences. The first question I had was how many nationalities are represented, and which countries. The second question is what is the distribution of the deaths by month.

The following is how I came to a visualization that answers the first question.

Data Compilation

Data is taken from two locations and merged.

  • The first set of data is a large spreadsheet obtained from the KZ Neuengamme Archiv containing all of their data on the prisoners that died and were at KZ Neuengamme or one of the satellite camps. This file contains 23,393 individuals.
  • The second data set is another set of files from KZ Neuengamme Archiv, but is derived from a list compiled by French authorities. It is available online at: http://www.bddm.org/liv/index_liv.php. The files were split into three sections listing the dead from Barkhause, Porta Westfalica, and Lerbeck. These files contained a total of 177 individuals.

Combining just the individuals matching those who were in a Porta Westfalica KZ from both sets of data left around 280 individuals.

Data Cleaning

There were a number of steps needed in order to have useful information from the data.

  • First of all, the data from the French archive was highly abbreviated. For example, the column containing the locations of internment were two or three letter abbreviations of location names. Elie Barioz, for example, had the locations “Wil, Ng (Po, Bar)” which, when translated, turn into “Wilhelmshaven, Neuengamme (Porta Westfalica, Porta Westfalica-Barkhausen)”
    • The process of translating the abbreviations was quite labor intensive. First, I had to search on the French site for an individual: http://www.bddm.org/liv/recherche.php
    • Search for ‘Barioz’. image-of-searchingNote: The Chrome web browser can automatically translate the pages on this site.
    • The correct individual can be determined by comparing the full name and the birthdate. The citation to the location in the book is a hyperlink to that record (ex. Part III, No. 14 list. (III.14.)).image-of-matches
    • The abbreviations for this individual’s interment locations are hyperlinks to more information, part of which is the full name of the location. Clicking on ‘Wil’ results in a pop up window describing the KZ at Wilhelmshaven and information about the city.
      image-location-pop-up
    • After determining that ‘Wil’ meant ‘Wilhelmshaven’, all occurrences of ‘Wil’ in that column can be changed to ‘Wilhelmshaven’.This process is repeated until all of the abbreviations have been translated.
  • Remove extraneous asterisks. It was quite frustrating to note that the French site did not include information on what the asterisk and other odd symbols mean. (Another odd notation is the numbers in parenthesis after the birth location.) I had to simply just delete the asterisks, losing any possible meaning they might have had.
  • Combine duplicates. Keep as much information from both records as possible.
  • Fix dates. They should all be the same format. This is tricky, in that Europe keeps dates in the format MM-DD-YYYY. For clarity sake, it would be best to use “Month DD, YYYY”. I left them as is for now. Editing 280 dates is not fun…
  • Fix nationality. The Tableau software references current nations. The data in the spread sheets uses nations current to the time of creation. For example, some individuals were noted with the nationality of ‘Soviet Union (Ukraine)’. These needed to be brought to the present as ‘Ukraine’. More problematic were the individuals from ‘Czechoslovakia’. Presently, there is the Czech Republic and Slovakia. The question is, which present day nationality to pick. There is a column for birth place which potentially solves the issue, but this field is for where the individual was born, wich, in the case of Jan Siminski, is seen. He was born in the Polish town of Obersitz (German translation), so the birth place can not clarify his nationality as Czech or Slovakian.
  • This brings up another issue, the translation of place names. City names in German, especially during the Third Reich, are different than current German names for the city, which are different than the English name of the city, which are different than what the nation calls the city. I need to standardize the names, picking, probably English. Tableau seemed to have no problem with the ethnic city names, or the German version, so I left them as is.

 

Tool Picking

I used the free program, Tableau Public: http://www.tableau.com/

This allows for very quick visuals, and a very easy process. The website has a number of free tutorials to get started. http://www.tableau.com/learn/training

Map

The first visualization I wanted to make was a map showing where the prisoners were from, their nationality. The map would also show the number of prisoners from each country. (This is not a tutorial on how to use Tableau, but a walk through of the pertinent choices I made to make sense of the data, it is methodology, not tech support. 🙂 )

Using the default settings (basically, just double clicking on the Nationality field to create the map) results in a dot on each country represented in the data.

image-blue-dots-map
This can be transformed into a polygon highlight of the country by selecting a “Filled Map”.

image-filled-map
Next step was to apply shading to the filled map; the larger the number of prisoners who died from that country the darker the fill color.
image-filled-map
The default color was shades of green. I wanted a more dull color to fit in with the theme of the visualization, “death”. I picked a light orange to brown default gradient, separated into 13 steps (there are 13 countries represented).

Table

While just a filled map with gradient colored countries is helpful, the information would be more complete, more fully understandable, with a legend. This can be created by using a plane table listing the countries and the number of dead from that country. Each row is color coordinated with the map by using the same color scheme and number of steps as with the map.

image-table

 

Dashboard

In Tableau, you create a dashboard to combine the different work sheets, maps, tables, graphs, etc. In this case, a full page map, with the table overlaid completes the visualization.

Result

The result is a very simple map, created in about ten minutes (after a few video tutorials to refresh my memory on how to create the affects I wanted).

(See a fully functioning result below this image.)
image-final-result

 

Benefits of Tableau

Tableau has some limitations. The results are hosted on their servers, which has the potential for lock down. They use proprietary, closed source code and applications.

But there are many benefits. The default visualizations look great. It is very easy to create simple and powerful visualizations. The product is capable of producing very sophisticated statistical representations. You can use the free and open source stats program R. The visualizations are embed-able in any website using Javascript.

The biggest benefit of using Tableau is the automatic link back to the original data source. I think the most needed shift in humanities (particularly the history profession), and the biggest benefit of “digital” capabilities for the humanities, is the ability to link to the source material. This makes it infinitely more easy for readers and other scholars to follow the source trail in order to provide better and more accurate feed back (read critique and support).

To see the underlying data in this visualization, click on a country in the map or the table. A pop up window appears with minimal data.

image-pop-up-info

Click on the “View Data” icon.

image-view-data-icon

Select the “Underlying” tab and check the “Show all columns” box. Voilà!

image-underlying-data

Behold the intoxicating power of being able to view the underlying data for a visualization!

Digital Humanities Improvement Idea

Imagine, if you will, the typical journal article or book, with footnotes or end notes referencing some primary document or page in another book or article. With digital media, that footnote turns into a hyper-link. A link to a digital copy of the primary document at the archive’s site, or the author’s own personal archive site. Or it links to a Google Book site with the page of the book or journal displayed. Now you have the whole document or at least a whole page of text to provide appropriate context to citation.

Way too often I have been met with a dead end in following citations; especially references to documents in an archive. Not often, but archives change catalog formats, documents move in an archive, they no longer are available to researchers, etc. It would be so much easier to have a link to what some researcher has already spent time finding. Let’s build on the shoulders of each other, rather than make each scholar waste time doing archival research that has already been done.

I think it incumbent upon all researchers to provide more than a dead-text citation to their sources. In this digital age, it is becoming more and more trivial to set up a repository of the sources used in research, and the skills needed to provide a link to an item in a repository less demanding. Here are some ideas on how to accomplish this already.

  • Set up a free, hosted version of Omeka at http://omeka.net. Add all of your source material to Omeka. Provide a link to the document in Omeka along with your citation in the footnote or end note.
  • Create a free WordPress account at http://wordpress.com. Add a post for each source document. Provide a link to that post in your citation.
  • Most universities have a free faculty or student web hosting environment (something likehttp://univ.edu/~usrname/). Dump all of your digital copies of your documents in that space (nicely organized in descriptive folders and with descriptive file names–no spaces in the names, of course). Now, provide a link to that resource in your citation.
  • Set up a free Zotero account at http://zotero.org. Set up a Group Library as Public and publish all of your sources to this library.

I intend to take my own advice. I have an Omeka repository already set up, with a few resources there already: NaziTunnels Document Repository. Once I start publishing the text of my dissertation, there will be links back to the primary document in the footnotes.

I would love to see this type of digital citation become as ubiquitous as the present-day dead-text citation.

I have not addressed Copyright issues with this. Copyright restrictions will severely limit the resources to be used in an online sources repository, but there are certainly work ways to work around this.

If hosting the sources on your own, one quick fix would be to put the digital citation sources behind a password (available in the book or journal text). Another option might be to get permission from the archive if only low quality reproductions are offered.

End

Let me know if you find the live-text or digital citation idea viable. Do you have other ideas for providing a repository of your sources?

Drop me a note if you want more detail on how I created the map in Tableau. I’m by no means proficient or in no way the technical support for Tableau, but I’ll do what I can to guide and advise.

Finding sources

53616116There are many ways and means of finding sources. Even though I should be just in the writing stages by now, I’m still finding sources to add to my collection. I was reading in Pierre Bleton’s book “Das Leben ist schön!” (no relation to the film of the same name, this book came out first), and a footnote here and there got me off on a web search for things. I don’t know why I have not searched the USHMM catalog until now, but it proved useful. I already knew of the three or four books that come up with a search for “Porta Westfalica”, but the 76 survivor interviews were a new find. I have transcripts from about 25 survivors that I found in the Neuengamme archive. Only one of those is listed in the results from USHMM. That gives me 100 survivor accounts. Of the estimated 3,000 prisoners in four camps at Barkhausen, Hausberge, and Lerbeck/Neesen, that gives me a view of 3% of the prisoners. Further looking into the records of the interviews, showed that not all of them were in English; some were in Danish, Hungarian, Hebrew, and Slovak. Of the 76 listed, only 45 were in English. IMG_0928So that gives me a total of 70 survivor interviews to use for my dissertation. I’ll have to compile some data points to draw up some graphs and what not (maps, info-charts, etc) about the survivors, and see what that can tell me. I also need to find some time to go down to the USHMM so I can view the interviews which are provided by the USC Shoah Foundation and only viewable from select locations.

Porta Westfalica: Second Week

2013-06-29 04.11.31
Me, being lonely.

My first week in Germany was very lonely. I’m such an introvert, so it’s my own fault, but still. That’s the reality.

On Saturday I took the train to Minden. I dropped my things off at the hotel, then took the train further to Nienburg. I served there as a missionary for my church 16 years ago. I had contact with friends there, still, so I stayed there Saturday night and went to Church on Sunday. That was great fun. Sunday evening I was back in Minden.

2013-06-29 03.25.07
Train station

On Monday, I met with Dr. Gerhard Franke. He is a retired school teacher and teacher instructor (which he did for over twenty years). He was interested in this topic because he grew up in Hausberge, was four years old when all of the tunnel stuff was going on, and is interested in history. He was a great help. He had contacted the city of Minden archives and was notified of several people in the area who were interested in the topic. So, being the great person that he is, he made appointments for us to visit with them.

The first stop on Monday was with a Herr Münstermann. He was a teenager during the war and experienced the issue first hand. He also later became a city official for many years. I wasn’t prepared to even meet with anyone that had lived through this time period, so I didn’t complete any paperwork at GMU to allow me to use personal interviews in my research. But nothing says I can’t listen anyways. He told us a little about the situation, and most helpfully, explained why it came to the point where forced labor and underground factories were necessary in the first place. Basically, after “Big Week” (a massive, multi-day bombing raid on Germany by the Allies), the Ambi-Budd company in Berlin was nearly completely destroyed. Numerous other business suffered a similar fate. They began the process of requesting increased protection from the military and help from the government. The idea was then, to disperse factories into smaller sub-factories where products could be produced in part, then brought together at yet another facility to be assembled.

Neat old building in Minden.
Neat old building in Minden.

In 1944, the decision was to move dispersal to underground facilities to further keep the factories secret and protected. Herr Münstermann had several copies of documents created by Rainer Fröbe and others. It would have been great to get copies of some of these documents, and Dr. Franke asked. Herr Münstermann, a very serious but kindly man (he is a widower and we could tell he was pleased to have us as company, he even put on a tie, and his suit coat when we sat down to talk), but he never gave out his things, because they had a tendency to not be brought back. We sat in silence. He capitulated, based on the good relationship Dr. Franke and he had. After our meeting, we immediately went to a copy shop, made double copies, and took the originals back to Herr Münstermann. Thank you, Herr Münstermann! (As it turns out, I found all of those documents in the Neuengamme archive, but you never know, so get copies while you can!)

Former local government building, until WWI. Current location of local archives
Former local government building, until WWI. Current location of local archives

Our next visit was with Thomas Lange, someone I ended up having much in common with. He was my age, for one. Second, he did his masters thesis on this topic. Thirdly, he is into technology. He’s actually a sound technician now, after finishing his Masters in History. Dr. Franke very kindly invited us all to his house for the discussion. Thomas, also very, very generously, provided Dr. Franke and I with nice, bound copies of his Master’s Thesis. He was thrilled that someone was interested in his work, doubly so that some crazy American was interested. We had a good talk about the subject, what sources he found and where, and things like that. Good stuff.

On Tuesday, we traveled all over Porta Westfalica (which consists of 15 districts which used to be individual villages, including Barkhausen, Hausberge, Neesen, and Lerbeck). We visited the Wilhelm Monument, one of the entrances on Jakobsberg (apparently we missed some huge cement oil tanks that can still be seen), and one entrance on Wittekindsberg, under the monument and behind the current lodgings of the Schützenverein Barkhausen. Dr. Franke knows lots of people and was able to work out some amazing opportunities.

One such opportunity was to meet with Wolfgang Walter who was very young in the military during WWII. He ended up serving in the West German military in the Porta Westfalica area, and thereby developed a great interest in the tunnel systems near by. He was a joy to talk with. He even gave me a large, 2 foot by 3 foot map of the area, so as to properly get a sense of my surroundings and landscape. Thank you, Herr Walter!

Schützenverien (shooting club) Barkhausen
Schützenverien (shooting club) Barkhausen

Dr. Franke had called the local newspaper and set up an appointment for a photographer to come take pictures. So on Wednesday just before I left, we (Dr. Franke and I) and the chief of the Schützenverein had our picture taken by the old entrance. It was blown up in 1947, so the only thing left to see are huge concrete blocks.  Dr. Franke is very interested in turning the tunnel system into some kind of memorial or museum, and is using me (coming all the way from America, showing interest in this place) as an example to local government officials and what not, that this is something important, and worth spending money on. A great thing, I think.

My time in Minden/Porta Westfalica was much too short. I could have used another day in the archives, and another day taking pictures and meeting people. But all in all, a great experience.

 

Staatsarchiv Detmold: First week in Germany

Well, here I am. In Germany again. The overriding thought for this trip is not the awesome opportunity to be in Europe, to see wonderful cities, meet amazingly friendly people, or finally be able to get into the “meat and potatoes” of this darn dissertation. No, I’m much more practical than that. My overriding thought, is… I sure miss my family. How can I be away for a whole month? My baby girl won’t even remember me, will she? Think about that next time you think going off to Europe to do research sounds so cool.

Hotel Nadler, home for a few days.
Hotel Nadler, home for a few days.

That’s the reality of the situation. Now on to the academic and other sides of things. The first stop on my research trip is Detmold. It is a very pretty city, so I’ll intersperse this post with pictures. Here I will be looking for anything in any way related to the tunnels at Porta Westfalica. I’m staying at the Hotel Nadler, a quaint little Fachwerkhaus turned into a restaurant and hotel. It’s right on the outskirts of the city center, where all the action is. I picked this location for it’s closeness to the city center and because it’s not too far from the archive. Just a 10-15 minute walk. I have done that for all of the locations except Berlin. That makes me walk so I get some exercise before and after sitting at a desk looking at old papers all day.

I got to the city too late to get to the archives the first day, so I went the second day I was there. And the two following days there after. That was Wednesday, Thursday and Friday. The first day made the trip seem a bit worthless. Of the fifteen or so folders of material that I had to go through, I got through about seven of them on Wednesday, and there was nothing worth wile in them, at least not for me. I did almost want to change my topic to something about how to prepare your house or building for bombing raids. There were some cool brochures and books on that. I later saw a portion of a documentary on TV that showed some training videos on what to when bombed by the British fire bombs. That seems like it could be a good research project; focusing on the literature and other forms of educating the populace on how to survive bombing raids. Anyhow, I digress.

This building is a bit off-kilter
This building is a bit off-kilter

On Thursday and Friday I hit the proverbial jackpot. Not for documents relating to the building and use of the tunnels during the war, but what was done with it afterwards. Most of the works out there all close their research with liberation by the US or British. I want to write about what happened after that. How did the people in the area deal with all of those former prisoners? Where did the former prisoners go? What did they do with these huge holes in their mountains? What happened with the equipment? Who was punished?

What I found in the archive were loads of documents that dealt with this post-war period. Unfortunately the archive follows the arcane tradition of not allowing users to maker their own copies of their documents. If I had a whole month, or $1000, I could have got all of the information. But I will have to be satisfied with what I could transcribe into my computer. One folder was full of tabulations of the weekly hours worked in the tunnel site in dismantling hardware and machinery, and preparing the site for demolition. Another folder was full of correspondence to those in charge of the post-war tunnel and the companies and firms that had contracts for building during the war. They apparently felt they should still be paid for work done. That’s something I had never considered before. Companies that contracted with the National Socialist Government to build and design, were promised money. When the war was lost, the National Socialist Government dissolved. Well, did that dissolve the contracts as well? Were the companies to lose out on the money owed them? I’m not sure what the answer on that one is. But I found a bunch of complaints and claims from building companies and architecture firms that wanted payment from somebody.

Detmold Church
Detmold Church

One final thing that I found in Detmold, was the correspondence between the town of Hausberge and the occupying British Army. The British plan was to blow up the whole of the tunnel system due to the possibility of the location being a highly usable military compound. The Allied Occupying forces want to completely wipe out any German military compounds.

My time in Detmold was a bit too short, but I may be able to swing a day on the way back if all goes quickly in Berlin. But I doubt I’ll ever be back, unless some other generous organization would like to pay for another research trip.