Perceptibility: Can I see it?

The algorithmically produced nature of modern-day data visualizations allows for a new form of visual language that was not possible when charts and graphs were drawn by hand or by templatized tools like Microsoft Excel. While this is exciting as new forms of expression are developed, it equally creates challenges where the technology is highlighted over the human ability to visually perceive these new types of visual marks. A noticeable pattern across visualizations is to use microscopic visual elements that are too small to easily see what they are when initially encountering a visualization.

Figure 3: Citeology, Justin Matejka (2012).

Figure 4: In Obama's Words, Juan Osborne (2011).

Figure 5: An Emergent Mosaic of Wikipedia Activity, Bruce Herr (2007).

An initial state of confusion occurs when words are perceived as wispy lines or literal screen grabs are seen as miniature tiles. Gestalt laws of organization (Wertheimer 1938) tell us that our eyes tend to see the larger entire image rather than the individual visual elements. The burden is then on the viewer to use the zoom-in features of the viewing device to further decipher what the visual elements actually are. If the viewer is not aware that the visual mark is not actually what he perceives it to be, then that extra step is omitted. In other words, if the viewer perceives the line as a line and not a string of small text, that level of understanding would be completely bypassed.

Pre-knowledge: What is this?

The predominant use of visual symbols in data visualization require explanation on the part of the designer to a) describe what the viewer is seeing and b) instruct how to read and how to use the visualization, if interactivity is involved. Occasionally there also is citation of the data source or background information explaining the reasoning for why the designer created such visualizations.

This type of contextual content can appear in various forms: an introduction window that takes advantage of content loading times, surrounding text when visualizations appear in the designer's blog, content provided when the "i" or "about" button is clicked, or tacit tutorials where the initial view automatically includes some form of guide to assist the viewer (Segel and Heer 2010, 1145).

Figure 6: MusicMap, Musicovery (2011).

Figure 7: People movin, Carlo Zapponi (2011).

Figure 8: Nobel Prizes and Laureates 1901-2012, Giogia Lupi (2012).

When new visual forms or interaction methods are presented to the viewer, these types of considerate explanations can alleviate the initial "What do I do now?" sensation and allow for an easier transition to engage with the visualization.

Once the visual elements of a visualization are perceived (or not) and the viewer can examine the visualization, multiple factors come into play simultaneously: comprehension, interpretation, utility, and engagement.

Comprehension: Can I understand it?

Being able to successfully perceive the visual elements of a visualization is not a guarantee that the viewer will understand it. Specialized visualizations of a particular domain require viewer knowledge of the content. Aside from viewer knowledge level, deciphering spatial and visual encodings (Bertin 1983, 60), understanding scales and measurements used in the visualization (Yau 2013, 109), assessing whether visual conventions can be leveraged (Kostelnick and Hassett 2003), etc. are cognitive activities that the viewer must perform.

Several other issues arise under this area that are worth mentioning. Visual metaphors are often used in visualizations for semantic or aesthetic purposes. In such cases, the viewer will also need to evaluate the semantic connections that the metaphor is trying to leverage. If the metaphor can aid the viewer to better understand the content of the visualization, then that would be considered a successful application. Careful matching and crafting of the metaphor is required.

Figure 9: GIS Map of Geography, Andre Skupin (2005).

Figure 10: Themescape, Thomson Reuters (2008).

Mismatches between data type and visual structures can be seen in visualizations. The fetishism towards circular visualizations often forces visualizations that use circles inappropriately (Few 2010). The cyclical nature of circles can support repeating events well, such as the lifecycle of certain living organisms over several years (Frankel and DePace 2012, 57). If visualizations are to be effective informational artifacts, the data type must map to the visual structure appropriately (Dastani 2002, 603). The following example attempts to visualize when certain words appear in the novel Les Misérable. A story that follows a temporal structure is mapped onto a circle with no clear indication of the beginning, middle, and end; one must search for them.

Figure 11: Novel views: Les Misérable, Jeff Clark (2013).

Visual conventions can be used effectively, but they can also be challenged in this new era of data visualization. The various ways in which time is represented is a good example. In traditional graphing language, time is typically expressed as the x-axis, running from past on the left to the future on the right. A quick search reveals several other ways to display the notion of time, such as using the radius as showing time (center is past, periphery is future) or time being expressed as age (past 5 years, past 2 weeks, yesterday, today.)

Figure 12: Imports and exports to and from Denmark & Norway from 1700 to 1760, William Playfair (1786).

Figure 13: W3schools.com's Historical Browser Statistics, Axiis (2009).

Figure 14: Seismic Monitor, IRIS (2012).

Computer-drawn visual elements can convey more dimensions than traditional methods. For example, lines traditionally embodied a constant dimension from point to point (e.g., solid, dotted, red, narrow). Visual elements that are computer-generated can be much more finely articulated and dynamic within a singular element. Take Aaron Koblin's Flight patterns, for example. A dynamic map of flight paths over the U.S. is represented by highly articulated lines that change brightness based on the altitude of the aircraft (Koblin 2011). The line is brighter when the aircraft is lower, creating dense areas of white line clusters where hubs are located. In addition to the conventional use of line as a path connecting two geographical points, Koblin has embodied the simple line with multiple meanings.

Figure 15: Flight Patterns, Aaron Koblin (2005).

What this does is requires viewers to be cognizant of a new way of reading a visual element, in this case the line, in order to appreciate the full depth of the designer's intention.

Utility: Can I use it?

Data visualizations, which typically represent relationships between many variables, benefit from interactivity and all the affordances that this provides. Commonly known methods of direct manipulation (Tidwell 2011, 293; Schneiderman, 1997, 33) need to be obvious to the viewer along with any interactive features, functions, or actions that accompany the visualization. When designed effectively, complex information represented through a visualization can gradually unfold, providing progressive disclosure to the reader in a way that alleviates cognitive load.

Experimentation is to be expected in a new domain. However, visualization designers should leverage design knowledge and principles that already exist in the fields of interaction and interface design to create effectively usable digital artifacts.

Figure 16: CNN Ecosphere, Heimat Berlin and Minivegas (2013).

Interpretation: What meaning do I make from it?

The reason we change modalities of the data, typically from text to graphic, is to engage our eyes in the thinking process (Few 2009, 30) and to help "amplify cognition" (Card, Mackinlay and Schneiderman 1999, 1). To do this, there are two related issues here: the production method of the visualization itself and the resulting inquiry the analyst can make.

The production method used to create the visualization needs to support discovery of the data by allowing the designer and the analyst to adjust the visual representation. Adjustments to visual attributes like colors to provide more contrast or scales to show micro-/macro-activities can help to reveal patterns, should such patterns exist in the data. Using visualization tools that have customization features gives control to the analyst to engage with the data further and discover the visual patterns.

After observing a data visualization, the analyst should be able to make subsequent inquiries: ask more questions, relate the immediate observations to larger knowledge that she may have, create a hypothesis about the data and its attending topic area, extrapolate ideas and make connections with other pieces of knowledge, generate "what if" scenarios, etc.

These various cognitive activities can be supported better by additional data sets and different types of visualizations. Let us use the following visualization (Figure 17) to understand this further.

Figure 17: Vehicles Involved in Fatal Crashes 2010, Nathan Yau (2012).

By using a familiar visual structure, i.e., a calendar, the barrier to understanding the visualization is minimal and the viewer can turn his attention to the data, which is the number of fatal crashes in 2010 in the U.S. This structure allows for comparison among the various time units: days, weeks, weekends, months, and seasons.

To gain further insights and a larger contextual understanding from this examination, one can ask questions like:

• Why are there more fatalities on the weekends between April and October? Is it the weather? (hypothesis testing)
• Are there more people out driving on these weekends? Does the fatality rate increase or is it the same? (understanding whether these are outliers or the norm)
• What time did these fatalities occur? (augmenting with more specific facts)
• Judging from the higher incidents around New Year's Day and Memorial Day weekend, is alcohol involved? (hypothesis testing)
• What about holidays that involve longer travel distances, like Thanksgiving? (seeking correlation and analogies)

The analyst would benefit by augmenting this particular visualization with datasets related to meteorology, number of drivers, travel distances, and alcohol consumption along with visualizations that would easily promote correspondence and comparison against the dates of interest.

Engagement: What is making me stay?

In the digital environment, where everything is a click or two away and the experience is ephemeral, creating the draw to stay with a visualization becomes an important consideration to the designer. Factors that keep the viewer anchored to a visualization can be based on issues such as the relevance of the topic area (is it of interest to me?), the ability it provides to learn more (I can get more detail here), the novelty of the visual (I've never seen anything like this before), the lack of alternative availability (this is the only place I can get this information in this form). The following is a non-exhaustive list of such factors:

• topic relevance
• personal relevance
• usefulness
• timeliness/immediacy
• learning
• extending/detailing
• visual novelty
• direct action to take
• motivation
• alternative availability

Figure 18: What percent are you? White, Gebeloff and Fessenden (2012).

Figure 19: Crisis map of Yolanda typhoon, Standby Taskforce (2013).

Figure 20: Jamn, George and Jonathan III (2014).

Once the viewer completes viewing the visualization, he enters the last phase of the experience, the "extend" phase, which translates to "outcome" in our framework.

Outcome: What do I do next?

If we are to consider the engagement with a data visualization to be an integral part of the experience, then what is the resulting outcome of that experience? Can it help us connect to a larger context in our lives? Are there new actions or behavior we can take with this new knowledge? If there is new learning to be gained from this experience, how can it supplement what we already know in order to enhance our overall understanding? What subsequent higher-level cognitive tasks will this allow us to perform?

Figure 21: San Francisco Crimespotting, Stamen Design (2009).

Figure 22: SOPA Opera, Dan Nguyen (2012).

Purpose: What is this visualization for?

One factor that permeates across the entire experience is that of purpose. Data visualizations range from mundane to amusing to informational and even to life-saving. Some have a strong "so what?" factor while the value of others is immediately apparent—the seminal Ushahidi, an open source mapping tool to report violence in Africa, for example.

All of the seven factors described previously can be tempered by understanding the purpose of the visualization. Is it to inform, persuade, stimulate, explore, or share? If it is meant to be entertaining, then comprehension and interpretation is not necessarily an important factor. If the designer wants the visualization to have a strong exploratory component, then including features that allow the viewer to perform deeper visual analytics of the data, such as filter, change views, or gain detail, become imperative (Heer and Schneiderman 2012). Viewers would benefit from knowing the underlying purpose of the visualization in order to evaluate it and form expectations for the ensuing experience.

Figure 23: Tweetures, Kunal Anand (2011).

Figure 24: Global burden of death cause patterns, Institute for Health Metrics and Evaluation, University of Washington (2013).