New Media for Scientific Data Visualization

New MediA for scientific data

visualization

Andrea Fasolo Rao

New Media for scientificdata visualizationAndrea Fasolo Rao279245

Supervisor:Marco Ferrari

A.A 2014/2015

Sessione di LaureaMarzo 2016

Università Iuav di Venezia

Dipartimento di Progettazione e pianificazione in ambienti complessi

Corso di Laurea Magistrale in Design del Prodotto e della Comunicazione Visiva

Curriculum Comunicazioni visive e multimediali

6 New Media for Scientific Data Visualization

AbstractItaliano

Questa tesi affronta il tema dell’interpretazione e della visualizzazione dei dati scientifici, su come si è sviluppata negli ultimi decenni, il suo legame con la tecnologia informatica e la posizione nei riguardi dei principi del design dell’informazione più tradizio-nale, oltre all’enorme influenza che il fenomeno Open Data sta avendo sul mondo della visualizzazione in generale. Verranno analizzati inoltre vari progetti basati sulla visualizzazione dati, ma che la affrontano da punti di vista diversi: quello puramente scienti-fico, quello del design e quello artistico. Quest’ultimo tipo di progetti potrebbe essere fondamentale per l’evoluzione della visualizzazione scientifica e di dati in generale: mettendo al primo posto valori estetici piuttosto che informativi esplora nuove modalità di approccio ai dati precedentemente non prese in con-siderazione Inoltre, l’utilizzo di strumenti considerati innovativi (video, new media, interattività, suono) può dare nuova vita alla divulgazione scientifica e alla visualizzazione di dati. La complessità e la vastità dei fenomeni e dei dati coinvolti, applicando algoritmi di “traduzione” in immagini e suoni, può contribuire a costruire esperienze di valore sia divulgativo che es-tetico. Sfruttando questo valore estetico si può avvici-nare il pubblico a tematiche apparentemente comp-lesse ma affascinanti, che normalmente passano per forme di divulgazione più tradizionali che, negli anni a venire, rischiano di perdere “appeal” agli occhi di un pubblico sempre più esigente, in termini di entertain-ment e “spettacolarità”.

7

AbstractEnglish

This thesis deals with the interpretation and visualization of scientific data, on how it has devel-oped in recent decades, its connection with computer technology and its position regarding the principles of the more traditional information design, and also the huge influence of the Open Data phenomenon is having on the visualization world in general. Vari-ous data visualization projects will be analyzed, that approaches it from different angles: the purely sci-entifically one, the design one and the art one. This last kind of project could be fundamental for the sci-entific, and data too, visualization evolution: with its primary focus on aesthetic values rather than infor-mative it explores new ways to approach data; in ad-diction the use of innovative instruments (video, new media, interactivity, sound) can breathe new life in science popularization and to data visualization. The complexity and breadth of the involved phenomena and data, together with appropriate “translating” al-gorithms to sounds and images, can create informa-tive, popularizing and aesthetic experiences. Lever-aging this aesthetic value, it can be possible to bring the public closer to apparently complex but fascinat-ing themes, since the more traditional popularization and communication means will lose of “appeal” to an always more demanding and difficult to astound pub-lic, that is becoming used to always more spectacular and entertaining means of communication.


Contents

1. Introduction

Why to investigate Scientific Visualization

2. Scientific Visualization

What is Scientific Visualization

Historic Scientific Visualization

Scientific Visualization, computers and computer graphics

3. Open Data

Beginning of the Open Data era

Early Open Data examples

The birth of public open data

The future: Linked Data

4. Art, Science and Design

Separation of Science and Aesthetics

Evaluating attractiveness

The role of art

10

12

14

16

18

24

32

34

36

37

40

44

46

48

50

9

5. Case Studies

Case studies selection

Ryoji Ikeda, Supersymmetry

Nathalie Miebach, Weather scores

Alex H. Parker, Worlds

Aaron Koblin, Nik Hafermaas, Dan Goods, eCLOUD

Soo-in Yang, David Benjamin, Living Light

Lorenzo Pagliei, Les Invisibles

JeongHo Park, LHC CERN Data Audio Generator

Jan Willem Tulp, Goldilocks

Kurzgesagt, In a Nutshell

Nasa Scientific Visualization Studio

Nasa Hyperwall Visualizations

6. A Design-centered evolution

Design as the unifying force

Usage of new media

Coverage of contemporary and popular science topics

Usage of datasets and real-time data sources

Keeping the data understandable

Application of design reasoning and verifiable methodology

7. Project: Sounds from the Sun

Sound and Psychoacoustics

Application to data

The Solar wind and its consequences

Solar wind observation

Sound rendition of solar wind data

Technical aspects

Testing process

Resources

Conclusions

Thanks

Colophon

52

54

56

60

64

66

68

70

72

74

76

78

82

88

90

91

93

94

95

96

98

100

102

103

104

106

108

110

114

118

121

122

1. Introduction


One of the most rapidly growing field of design in the last decade has surely been data visualization. This happened, and is still happening, for a variety of reasons, including the always-increasing availability of any sort of dataset, the rise of visual journalism, but also the incredible computational power that we can have inside the few kilograms of a nowadays laptop computer, when compared to a 20, or even 10, years ago workstation.

Thanks to these few conditions, we have the possibility, power and will to create progressively more useful and appealing visualizations that can fit in different aspects of our lives.

Parallel to design-driven and “artistic” data vi-sualizations, in the last three decades another field of representation was born and growing exponentially, both in usage and importance: Scientific visualiza-tion, or SciViz.

This particular kind of computer graphics is traditionally aimed at rendering three-dimensional phenomena of various kind, with an emphasis on the “realistic” representation of features like volumes or surfaces, depending on the aim of the piece. This kind of artifacts, however, are usually developed in highly specific scientific or academic environments, focusing on very different parameters from the artworks we are more used to see, aiming at a very small group of experts from a specific field and often created with-out the design knowledge involved in the generally more accessible works of data visualization.

However, things are rapidly changing: in the last years the open data revolution caused a great number of research centers and agencies to make their datasets available to the public. For the first time in history each one of us have the possibility to work on the very same numbers analyzed by the top scientists and researchers of NASA or CERN, just to mention two of many. This created a lot of interest in various design and art communities, and made pos-

Why to investigate scientific

visualization

13Introduction

sible some extremely interesting works of “data ren-dition”, ranging from static visualization to real-time immersive interactive installations, covering the vast arc that goes from design to art.

This could turn out to be a great possibility for Scientific visualization: it has the chance to learn from the great mass of projects involving open data. It could incorporate, for example, a more conscious and consistent design process, to improve usability and readability for the specialists, but also to look at the more experimental and aesthetics-oriented works of art, that could be made into powerful popularization instruments for the general public.

Scientific Visualization2.


If Scientific visualization is approached, the first thing that should be made clear is if and how it is dif-ferent from what we usually call “data visualization”: is it a subfield, a completely different thing or some-thing in between?

Traditionally, Scientific visualization has always been considered a group of techniques to transform some kind of science-based data into visual informa-tion through a reproducible process, with the funda-mental difference from data visualization that the data is intrinsically spatial and physically based. In other words, in Scientific visualization the spatial co-ordinates are part of the data, usually being between the independent variables; while in information and data visualization designers often deal with abstract data, so they can freely decide how to manage the two or three spatial dimensions.1 The same consid-eration is for time: in Scientific visualization it’s often given and intrinsic to the data points, while in data visualization it’s another parameter that the designer can choose to manipulate.

This difference can be further clarified when the most common visualization techniques used in SciViz are taken into account, considering data of different nature:2 bi-dimensional dataset usually represents scalar or vector fields; in the case of scalar data (for ex., temperatures) color mapping, wireframe plots, scatter plots or contour lines can be used, while vector fields (like winds or magnetic fields) take ad-vantage of streamlines or line integral convolution;

For tri-dimensional data the subdivision be-tween scalar and vector fields still adapts: in case of scalar values, volume rendering or isosurfaces can be used, while vector data uses streamlines, line integral convolution, particle tracing and topological meth-ods. To visualize tensor fields instead, which are more complex, a technique denominated “hyperstream-lines” were developed.3

What is Scientific Visualization

1: T.M. Rhyne, North Carolina State University, “Information and Scientific Visualization:

Separate but Equal or Happy Together at Last”, 2011

2: The Ohio State University, “A Critical History of Computer Graphics and Animation”

https://design.osvu.edu/carlson/history/lesson18.html

3: T. Delmarcelle, L. Hesselink, “Visualizing second-order tensor fields with hyperstreamlines”, Computer

Graphics and Applications, IEEE, 1993

17Scientific Visualization

Example of line integral convolution applied visualizing heat movements in a hot water pot

Example of hyperstreamlines applied to tensor field visualization

Example of mesh visualization of a 3D function


For how SciViz is defined and for its mechan-ics, it is tightly bounded to computers: it requires to elaborate and translate to images always increasing amounts of data, or to run complex simulations that may take up to years of calculations on huge super-computers. Therefore, it may be difficult to imagine Scientific visualization before the digital revolution that happened in the ’80. In fact, usually we con-sider SciViz to be born in the early ’80, and becom-ing prominent at the end of the same decade, after a 1987 report released by SIGGRAPH for the National Science Foundation, “Visualization in Scientific Com-puting”. That report was crucial for the history of Sci-entific visualization, bringing to the newborn field a lot of attentions, that concretized themselves in new funding, workshops, conferences and publications in the following years.

Before the NSF paper there are anyway some remarkable examples of visualizations, developed beginning from the ’60, and there are even more im-pressing examples of “pre-digital” Scientific visual-ization: in fact, the century from 1850 to 1950 gave a great push to the developing of various data represen-tation techniques. The most important example, but also quite singular, is considered to be the Maxwell’s thermodynamic surface, a clay sculpture created in 1874 by the mathematician James Clerk Maxwell. He is considered the father of electromagnetism, but he was active also in other physics and mathematics fields.

His surface was, in fact, a three-dimensional ab-stract plot of the thermodynamic states of a fictitious substance, similar to water, putting in relation its vol-ume, entropy and energy on the x, y and z axes, with isothermal and isopiestics lines traced on the surface. With Maxwell’s own words, it allowed “the principal features of known substances [to] be represented on a convenient scale”4 : usually, in thermodynamics, when it is required to analyze the state of any fluid

Historic Scientific Visualization

4: J.C. Maxwell, “Letter to James Thomson”, The Scientific Letters and Papers of James Clerk

Maxwell, volume 3, p. 230-231

and to extract a parameter like pressure or tempera-ture, engineers and scientist have to rely on a number of different bi-dimensional diagrams, each one plot-ting just two parameters at a time, like pressure and volume, or temperature and entropy, and so on. Those diagrams can be considered, in fact, just various bi-di-mensional projections of a Maxwell’s thermodynam-ic surface. While the practicality of this artifact is low, due to the obvious difficult to extrapolate or read single values from it, it offers an extreme degree of synthesis, allowing a novel and encompassing view on various phenomena of fluid thermodynamics, of-fering so a major insight value over the traditional bi-dimensional graphs.

Going back two decades, another remarkable example must be taken into consideration, this time in the field of public health. During 1854 there was a cholera outbreak in London, in the Soho district: it was a particularly unfortunate area of the city, since

Maxwell thermodynamic surfaceCavendish Laboratory, Cambridge, England.


Example of Pression / Entropy thermodynamic diagram for water

Example of Entropy / Enthalpy thermodynamic diagram


the sewer system had still to reach it. This wasn’t, anyway, a big concern to the population: at the time it was a wide spread idea that infectious diseases spread through some kind of miasma, or “bad air”, therefore most people had no idea of the importance of proper sanitary services. What really changed the situation, saving many other people from the infection and bringing a new understanding about hygiene, was the work of John Snow, a physician: he started trac-ing on a map all the cases of cholera he was hearing about from the population, and thus noticing that most cases were localized in a precise area. Further investigations brought him to conclude that most of the infected had been drinking water from a par-ticular street pump. His research led to the closing of that pump, that later was discovered to be polluted by a nearby cesspit, and created in the population the awareness that some diseases are carried by infected water, thus anticipating Louis Pasteur’s germ theory

Snow’s original cholera spreading map


by 7 years.5 It’s an incredible milestone for scientific visualization: it was one of the first time, and a proof, that visualizing a phenomena on a map in a useful way can cast a new light on events that may seem unconnected between them.

Another notable example that are considered precursors of SciViz is the “Diagram of the causes of mortality in the army in the East” by Florance Night-ingale, from 1873. She was as a British nurse manager during the Crimean War, founder of the first secu-lar nursing school in London and also a proficient writer, with various publications to spread medical knowledge. She also worked on and popularized the graphical presentation of statistical data, leading to a breakthrough in scientific visualization. During her career she used to produce various diagrams, with a designed that she created and called “coxcombs”, or nowadays polar area diagrams, that she often used to present statistical data and observations to peo-ple who may had difficult to understand traditional statistical reports, for example for her reports to the members of the Parliament on the state of the medi-cal care in the Crimean War.

One of these diagrams visualized the seasonal mortality by cause of her patients, noting that a big number of deaths was linked to the bad state of Brit-ish army camps in India, particularly: bad drainage, contaminated water, overcrowding and poor ventila-tion. So, she started a campaign to popularize the idea

Florence Nightingale’s “Diagram of the causes of mortality in the army in the East”

5: Steven Johnson, The Ghost Map: The Story of London’s Most Terrifying Epidemic – and How it

Changed Science, Cities and the Modern World, Riverhead Books, 2006


that it was necessary to improve the sanitary condi-tions in the country to increase the life quality and expectance for both the Indian population and the British army.

With support from the Royal Sanitary Com-mission, she convinced the minister responsible to require every house owner to connect its building to the main drainage system, greatly enhancing sani-tary conditions.

As in Snow’s example, also this case shows how a proper and well-design way to represent abstract data can help to establish connections that are other-wise difficult to be noted, and to spread knowledge on problems that are difficult to understand for whoever may be a stranger to a particular field of knowledge.


One of the most remarkable example of early computer-generated scientific visualizations, and also the first computer graphics film, is “A two gyro grav-ity gradient attitude control system”. Dated 1963, it comes from the Bell Laboratories, at the time owned by AT&T. It was created by Edward E. Zajac, an en-gineer working on various long-range, trans-oceanic and planetary communications projects.

At the beginning of the sixties Zajac was work-ing on Bell’s communication satellites program, in-volving a new kind of satellite equipped with two gyroscopes, that would have solved some stability and control problems that emerged with the previous satellite designs. While the mathematics required to prove its theory are thoughtfully explained in its pa-per “A Two-Gyro, Gravity-Gradient Satellite Attitude Control System”, (J.A. Lewis, E.E. Zajac, The Bell sys-tem technical journal, 1964), he felt that the data, the graphs and the static images in that dissertation were not really enough to explain the complex rotations that its theorized satellite would have. So, together with his colleague Frank Sinden, they started to work on an innovative visualization technique: a perspec-tive video simulation, programmed in FORTRAN and finalized using a custom-made software called ORBIT, designed for this very purpose.

The realization process is to be considered in-teresting, for today’s standards: the original computa-tions were fed via punch cards to a IBM 7090 super-computer, and then printed directly to microfilm with a General Dynamics Electronics Stromberg-Carlson 4020 microfilm recorder.

While the animation looks clear but extremely elementary, each minute of the resulting 16 FPS-vid-eo rendering took up to eight minutes of computation on what was one of the most performing supercom-puter at the time.6

In those early days of computing resources were incredibly strict, thus the aesthetics of most visual-

Scientific Visualization,

computers and computer graphics

Composition of frames from“A two gyro gravity gradient

attitude control system”

6: Steven Johnson, The Ghost Map: The Story of London’s Most Terrifying Epidemic – and How it

Changed Science, Cities and the Modern World, Riverhead Books, 2006


izations were tied just to black and white, since color would have consumed significantly more memory. The first example of the use of color came neverthe-less in 1969, by Dr. Louis Frank, who was working with the Space Plasma Physics Research group of the University of Iowa on the data collected by the LEPE-DEA instrument of a satellite.7 He plotted time on the x axis, while the energy spectra in on the y axis and the different color hues represents the amount of par-ticle hits per second, effectively summarizing three dimensions of data in two-dimensional plot.

Going back again to animation, in 1977 the Computer Graphics Research Group (CGRG) of Ohio State University, with Bob Reynolds and Wayne Carl-

IBM 7090 supercomputer

Dr. Frank’s time / energy graph

7: http://www-pi.physics.uiowa.edu/~frank/


Still frame from “Interacting Galaxies”

The Mice Galaxies


son, created a milestone visualization of interacting galaxies that was also featured on the Carl Sagan Cos-mos TV show.8 The importance of this work lies in the fact that it was a display of a very complex nu-merical simulation, that actually was repeated many times with different parameters to show how differ-ent types of interaction can create very differently shaped galaxies, explaining for example the tidal tails formations we observe in the Antennae Galax-ies and in the Mice.9 The simulation was programmed in FORTRAN by Dr. Alan Toomre, and even if quiet simple for the number of particles simulated in the system, proved how this kind of simulation were al-ready a valuable research tool in this particular field: in fact, when dealing with this kind of dynamics, sim-ulations are the only way to visualize the evolution of galaxies, since observation can show us only “stills” of galaxies in various stage of their life, considering the time involved in these processes. For example, his simulations led Toomre to suggest that elliptical galaxies are remnants of the major mergers of spiral galaxies,10 which became a widely accepted idea from the scientific community.

Visualization and animation proved its value also with another historic example, this time in the field of programming. One of the classic problems posed to computer science students is the implemen-tation of sorting algorithms, which are various ways to put in order a big series of randomly – distributed

Still frames from “Sorting out Sorting”

8: http://excelsior.biosci.ohio-state.edu/~carlson/history/ACCAD-overview/overview-old.html




numbers. Mathematics and researchers created in the years a lot of different sorting algorithms, reach-ing an incredible level of efficiency, but also involv-ing very complex mathematics processes that make very different to imagine how an algorithm actually works and to estimate its efficiency. With this idea in mind, Ronald Baecker from the University of Toronto created in 1980, and presented at Siggraph in 1981, “Sorting out Sorting”, the first computer-generated process-visualization animation that showed the ap-plication of different algorithms in real-time, making their inner workings evident, with clear comparisons on time and efficiency.11 The video was a great suc-cess, and it’s still used today to illustrate that kind of processes.

The fact that really pushed forward the at-the-time young Scientific Visualization field was the pub-lication of a report compiled for the National Science Foundation from a SIGGRAPH panel in 1987, called “Visualization in Scientific Computing”: first of all it acknowledged and defined “officially” Scientific Vi-sualization, poetically describing it as “a tool for ap-plying computer to science, [that] offers a way to see the unseen”,12 and then proceeded to explain the role of visualization in science, which processes are involved, how it should expand and even mentioned the already underrated importance of data, especially from high-volume sources. The paper identifies Visu-alization as a multidisciplinary field, that at that time involved computer graphics, image processing, com-puter vision, computer-aided design, signal process-ing and user interface studies, but also recognizes the need to expand that, including engineering and scientists dedicated to develop new visualization sys-

Still frames from “Sorting out Sorting”

11: R. Baecker, “Sorting out Sorting: A Case Study of Software Visualization for Teaching

Computer Science”, Software Visualization: Programming as a Multimedia Experience, MIT

Press, 1998.

12: Bruce McCormick, Thomas DeFanti, Maxine Brown, Visualization in Scientific Computing, p. vii


tems, artists, for their education in visual communi-cation and inventive about visual representations, cognitive scientist and psychologists to design better interfaces and try new concepts and steer research in new directions.13

Throughout the paper it’s underlined many times how visualization in general is fundamental to human beings, and that a lot of different kinds of information are better communicate in a visual way: for example, when dealing with big amounts of data that the brain cannot process in numerical form, or for structures like DNA, molecular models or other kinds of simulation. An interesting fact is that the authors note the need for interaction in Scientific vi-sualization: at the time most elaborations run on su-percomputers via batch processing, so the user could only start the process and wait for the result, with-out knowing what the calculator was doing. They theorized Interactive visual computing, where scientist could “communicate with data by manipulating its vi-sual representation during processing. The more sophis-ticated process of navigation allows scientists to steer, or dynamically modify, computations while they are occur-ring. These processes are invaluable tools for scientific discovery.”14 This change should be implemented de-veloping and giving high-quality visualization tools directly to the researchers, via a proposed federally-funded initiative. The first step should be to abandon the supercomputing model, in favor of “workstations, minicomputers and image computers […] that should be placed on the desks of each and every researcher” with specialized graphics processors, that could be much more efficient, with each of these machines costing between 5,000 $ and 100,000 $. Along with

Still frame from the last section of “Sorting out Sorting”

13: Bruce McCormick, Thomas DeFanti, Maxine Brown, Visualization in Scientific Computing, p. 11

14: Bruce McCormick, Thomas DeFanti, Maxine Brown, Visualization in Scientific Computing, p. 5


this new hardware, research centers should provide dedicated software to develop visualizations, easy enough to use to not require the user to know pro-gramming or have particular graphical knowledge.

The SIGGRAPH report apparently got its point, causing the interest of the research world, attracting funding agencies and triggering conferences, work-shops and various publications on the topic. Most importantly, various software developers proposed a variety of specific and generic visualization soft-ware environments, for example IBM Data Explorer, Wavefront Advanced Visualizer and SGI IRIS Explor-er. Most of them did not use code or textual instruc-tion, but rather developed a dataflow paradigm that did not require programming or graphics knowledge: the user had to connect modules between them, each one performing a specific operation. It was however possible to create custom modules with traditional programming languages like C or FORTRAN.

Screenshot from the Wavefront Advanced Visualizer


Screenshot from the IBM Data Explorer software

Screenshot from apE from the Ohio Supercomputer Graphics Project at Ohio State

Open data3.


A phenomenon that is slowly influencing and changing various aspect of our lives, even if most of us are still not really aware of it, is Open Data: with its idealistic push and novel way of seeing information we can consider it to be the most recent revolution in the history of knowledge.

For centuries data, particularly governmental, was considered something to keep away and even dangerous for the public security, until 2008, when the Obama administration realized that opening up their data to the public could bring to more participa-tion and trust in the government.

But if we consider the attitude towards the openness of scientific data and results, it has always been more positive than in the political world, with various researchers theorizing that every scientific result should be as more widespread and accessible as possible. Even with this disposition, until the 1990 de-cade there wasn’t really any active projects to publish or share scientific data.

The specific term “Open data” was first men-tioned in May 1995, in a proposal for the Congress of the World Meteorological Organization, concerning the disclosure of geophysical and environmental data. From the same paper we can read: “Atmosphere data transcends borders, they promote open exchange of information between countries, a prerequisite for the analysis of this global phenomena”. While this seems almost obvious nowadays, at the time even the shar-ing of “naturally global” atmospheric and weather data was not really taken into consideration.

But the concepts behind Open data are way old-er than that, always considering the scientific world: from a theoretical point of view, in 1942 Robert King Merton, considered one of the fathers of sociology, wrote: ““Communism,” in the nontechnical and extend-ed sense of common ownership of goods, is a second integral element of the scientific ethos. The substantive findings of science are a product of social collaboration

beginning ofThe Open Data era

Robert King Merton

35Open Data

and are assigned to the community. They constitute a common heritage in which the equity of the individual producer is severely limited. […] Property rights in science are whittled down to a bare minimum by the rationale of the scientific ethic. The scientist’s claim to “his” intel-lectual “property” is limited to that of recognition and es-teem which, if the institution functions with a modicum of efficiency, is roughly commensurate with the signifi-cance of the increments brought to the common fund of knowledge. […] The institutional conception of science as part of the public domain is linked with the imperative for the communication of findings. Secrecy is the antith-esis of this norm; full and open communication its en-actment. […] The communism of the scientific ethos is in-compatible with the definition of technology as “private property” in a capitalistic economy”.15

While it does not clearly mention Open Data in the way we consider it nowadays, he is endors-ing it way ahead of its time, and even touching the theme of intellectual property, a topic that will be very dear to the Open source movement in the last decades of the century, that was also a great promoter of the Open Data ideals. Another important theoreti-cal endorser of Open Data was the American politi-cal economist and Nobel prize Elinor Ostrom, that in 2009 stated that information commons are similar to public goods, because their use by one do not impede the use by other. Instead, information commons can be considered of a new kind: their use does not de-plete the common stock, but rather it enriches it.16

Elinor Ostrom

15: Robert King Merton, “The Normative Structure of Science”, The Sociology of Science: Theoretical and Empirical Investigations

16: Charlotte Hess, Elinor Ostrom, “IDEAS, ARTIFACTS, AND FACILITIES: INFORMATION AS A COMMON-POOL RESOURCE”, 2003


One of the first notable example of the applica-tions of some Open Data principles is from the 1950 decade: the International Council of Scientific Unions created the World Data Center system, in preparation for the International Geophysical Year of 1957/58. Several World Data Centers were established, to min-imize the risk of data loss and maximize data accessi-bility. However, since there was no Internet, that was more like a distributed storage system, rather than an active data sharing platform.

The first really Open Data-oriented project was the Human Genome Project, which is still the biggest collaborative biological project. Its planning started in 1984, the development begun in 1990 and was com-pleted in 2003. It was carried out by twenty universi-ties in United States, United Kingdom, Japan, France, Germany and China, under coordination and fund-ing from the US government, with the ultimate goal of mapping the whole human genome, sequencing multiple variations for each gene.17

Early open data examples

International Geophysical Year 1957/58celebrating stamp

Human Genome Project logo

17: Kathering Harmon, “Genome sequencing for the Rest of Us”, Scientific American

37Open Data

Going back to consider Governative Open Data, the most important event on this side was the Sebas-topol meeting: a reunion of thirty activists and think-ers near San Francisco, aiming to define the concepts of Open public Data, hoping to make it adopt to the US presidential candidates. Between them there were various important figures: Tim O’Reilly, most known for popularizing the concepts of open source and of Web 2.0; Lawrence Lessing, a professor of Law at Stanford Univeristy and founder of the Cre-ative Commons licenses; Aaron Swartz, hacktivist and inventor of RSS, sadly known for his suicide in 2013 after being found guilty of wire fraud for trying to download and share a great number of academic journal articles.

The Sebastopol meeting defined that public data should be considered a common property, just like sci-entific ideas, and it should be of interest to everyone for it to be shared and used. This concept did not come out of nothing, but it is strongly based on the concepts of open source software: openness, participation and collaboration. Coming from that environment most of the participants were aware of how good this model is working for free software development: program-mers collaborate and are invited to publicly share their code, both to let others learn from them, and to create a collective exvvpertise. There are not really hierarchical rules, and the work between developers is based on a peer-to-peer collaboration model that promotes competence and reputation. Synthetizing, Tim O’Reilly said that “we must apply the principles of open source and its working methods to public affairs”.

The meeting achieved even more than what they were expecting: in 2009 the newly elected presi-dent of the US Barack Obama signed three presiden-tial memoranda about open government and Open Data, taking the open source culture at the heart of public actions, stating that “Openness will strengthen our democracy”. Precisely, he defined that the Govern-

The birth ofpublic open data

Aaron Swartz at a Creative Commons event, 2008


ment should be transparent, to promote accountability and because information is a national asset; participa-tory, to enhance public engagement, soliciting public input, thus enhancing the government effectiveness and improving its decision-making; collaborative, us-ing innovating tools to create collaboration between Government, public and private sectors.18 This had a huge impact also for scientific data, because it trig-gered the opening of NASA and other governmental agencies datasets to the public.

The situation of Open Data in governments is positively evolving and many states worldwide are moving towards policies similar to the one used in the US, even if the cultural changes required for it may be difficult to accept in some context. The benefits are anyway vastly superior that any possible concern, both in politics, allowing better transparency and citizenship participation, and economics terms, hav-ing created a whole new branch of possible activities based on data.

What is incredibly relevant about Open Data for the Scientific Visualization field is that making scientific datasets available for the public created a great interest in designers and artist: being able to ac-cess that amount of data made possible to create en-tirely new data-driven concepts and projects both in art and design, that are creating fascinating ways to make use of those numbers, often crossing with fields like new media, interactive and installation art.

Anyway, Open Data is not spreading as fast as it could, and it is facing some issues: the most important

Open data scores per country, evalued from the Global Open Data Index - http://index.okfn.org/

18: Barack Obama, “Transparency and Open Government - Memorandum for the Heads of

Executive Departments and Agencies”, https://www.whitehouse.gov/the_press_office/

TransparencyandOpenGovernment

39Open Data

being the lack of interoperability and the heteroge-neous demand. The first problem can be summarized with the lack of a universal Open Data standard: it is stated that data needs to be machine-readable, but not how, and so dozens of different standards and formats are used around the world. This does not just regard the file formats themselves, but also column subdivision and labeling, precision in number repre-sentation, and many more similar details that a devel-oper must face when approaching different datasets.

Open Data theorists themselves, after their great initial success and the general social acceptance of their ideals, also are slightly changing their points of view: for example, Lawrence Lessing dissociated himself from the ideas of radical transparency of the movement. Even Beth Noveck, that took part in the creation of the Open Data policies of the Obama ad-ministration, is not sure anymore that Open Data is being enough to improve the governance of public affairs. Other participants of the Sebastopol meeting even ended up supporting WikiLeaks.

These minor difficulties do not mean that Open Data should be abandoned: they just mean that that was a first step, and we must proceed towards always better models of data sharing. Tim Berners-Lee is pro-posing, for example, Linked Data.

Beth Noveck


Tim Berners-Lee is an incredibly important fig-ure, that influenced the very way we communicate and perceive information nowadays: in March 1989, while working at CERN he defined and prosed a new information management system that became, in the November of the same year, the World Wide Web that we all know, successfully implementing the first HTTP communication.19 Currently he is director of the World Wide Web Consortium (W3C) and founder of the World Wide Web Foundation.

Starting from 2009, he worked with the British government to support their Open Data project, that concretized itself in the data.gov.uk portal.

Tim Berners-Lee research nowadays is focusing toward the future of Open Data: he is proposing the “Linked Data” standard, as a mean to solve the prob-lems that traditional Open Data are facing.

Linked data must respond to four simple rules: 1- Use Uniform Resource Identifiers (URIs) as

names for things, e.g. http://dbpedia.org/resource/Brus-sels can be used for referring to the city of Brussels.

2- Use HTTP URIs, so that people can look up those names.

3- When someone looks up a URI, provide useful information, using the standards (i.e. RDF, SPARQL).

4- Include links to other URIs, so more things can be discovered, e.g. from http://dbpedia.org/resource/Brussels a link is available to http://dbpedia.org/re-source/Belgium.20

To make clear and “rate” how much Linked Open Data is considered good and useful, a rating sys-tem has been developed:

1 Star: The Data is available on the Web with an Open license.

2 Stars: Data is in a machine-readable format, like an Excel file instead of a scanned image.

3 Stars: Data is in a non-propertary format, like .csv instead of Excel.

4 Stars: Data uses a W3C open standard to iden-

The future:Linked Data

Tim Berners-Lee

19: https://www.w3.org/History/1989/proposal.html

20: https://joinup.ec.europa.eu/community/semic/news/understanding-linked-data-

example

41Open Data

tify things, making the data easily linkable.5 Stars: Data is linked to others dataset to provide

context.21

This system is addressing mainly governmental agencies and any other entity that can produce and spread Open and Linked Data, to make clear which is considered to be the best and most useful way to share their data, focusing on shareability and possi-bility to link different contents.

The most notable example of Linked Data now-adays is DBpedia22: it is a project born in 2007 from the Universities of Berlin and Leipzig and in collabo-ration with OpenLink Software, aiming at extracting all the structured content from Wikipedia, to make it available on the web and semantically searchable.

This last aspect is a major difference over more “traditional” forms of Open Data: semantic research allows to query for associative and contextual in-formation, both explicitly and implicitly, allowing to provide a single, precise piece of information, but also answer to open questions through techniques like pattern matching and digital reasoning. With this kind of technology, the value of information grows exponentially when adding new connections, al- DBpedia mapping in 2014

21: https://www.w3.org/DesignIssues/LinkedData.html

22: https://www.ted.com/talks/tim_berners_lee_on_the_next_web/transcript?language=en#t-485180


lowing each dataset to be interconnected, effectively building a much more effective network of knowl-edge than what is possible today.

To give an example, as of February 2016, the English DBpedia describes 5.9 million resources, with 4.3 million abstract, 425 thousand geo coordinates and 1.45 million depictions. 4 million resources are part of a consistent ontology and consists of 2.06 mil-lion persons, 682 thousand places, 376 thousand cre-ative works, 278 thousand species and 5 thousand diseases. There are 128 languages in which DBpedia is localized, for a total amount of 38.3 million entries. All of this data is automatically selected, extracted, categorized and linked together by automatically scraping Wikipedia.23

Another remarkable example of Linked Data is represented by GeoNames, a geographical database containing 10 million geographical names, corre-sponding to more than 7.5 million unique features24; and UMBEL (Upper Mapping and Binding Exchange Layer), a knowledge graph of 35 thousands concepts that can be used in information science to relate in-formation from different sources. UMBEL in particu-lar makes heavily rely on the semantic research and interoperability for concept search, concept defini-tions and ontology consistency checking.25

23: https://www.ted.com/talks/tim_berners_lee_on_the_next_web/

transcript?language=en#t-485180

24: http://www.geonames.org/

25: http://umbel.org/resources/news/umbel-v-1-20-released/

43Open Data

art, scienceand design4.


Observing both the historic examples and the more contemporary case studies taken into account from scientific visualization, it can be noted that they focus mostly on solving functional requirements, of-ten overlooking on concepts like aesthetics or user experience. Also, they are often created by and aimed to experts in a particular scientific sector, requiring some level of familiarity and knowledge with the topic of the visualization.

On the opposite side, the increasing public avail-ability of datasets, APIs, and other forms of Open Data is creating a kind of “parallel universe” of artistic proj-ects involving visualization, centered mostly on the aesthetic value, reversing the more traditional utili-tarian approach, and attracting much attention from a new and wider kind of public, that maybe never re-ally approached science and are just generally inter-ested in art and culture: in this way, suddenly, they are experiencing a work of art but also dealing with subatomic particles, pulsars and weather data. “Come for the art, stay for the science”, it might be said.

A cause of the low consideration of aesthet-ics in SciViz resides in the perceived impossibility to evaluate and rate objectively the attractiveness of any visualization, privileging different parameters, specificially utility and soundness. These three prin-ciples are considered the elementary requirements of design, and were identified for the first time in 25 BC, by the Roman architect Vitruvius in his book “De Ar-chitectura”. Relating these principles to information visualization:

Utility is associated with functionality, effective-ness and efficiency, and can be considered the opti-mization of a visualization, presenting in the clearest way as much information as possible within the giv-en constraints. It is often the main focus of academic research, since it can be evaluated by comparison of different methods and concepts.

Soundness is a quality related to the algorithm

separation of science and aesthetics

47Art, Science and Desig

used to generate a visualization, mainly to its reli-ability and robustness; but also takes a wider sense, including how efficient the code is, if it can take dif-ferent kind of data, or tolerate errors. In an academic environment it is expected that the algorithm behind a visualization is clearly explained, and must fulfill some elementary stability and reliability require-ments. It can be evaluated both with mathematical and empirical methods.

Attractiveness refers to the aesthetic quality of a visualization, relating primarily to the appeal or beauty, but can be extended to consider other aspects like originality, novelty, and can also encompass the whole user experience matter. The problem with this principle is that, at least traditionally, there is no rec-ognized way to objectively evaluate or it, therefore academic research in scientific visualization does not have any expectation in that regard26.

This kind of reasoning has a fundamental flaw anyway: while it is obvious that “beauty” cannot be measured, it is also true that various studies proved that, when dealing for example with computer in-terfaces, the aesthetic quality can support utility, im-proving the user’s performances and satisfaction27.

26: Andrew Vande Moere, Helen Purchase, “On the role of design in information visualization”

27: D. Norman, “Emotion & Design: Attractive Things Work Better.” interactions 2002


It has been proved that the perceived usability of an interface is related to the subjective aesthetic judgment by the user28; another study revealed that users tend to spend more time using a software with a more appealing interface and are also more for-giving in case of errors or bugs29.Another study em-pirically demonstrated that “visual embellishments” considered non-utilitarian, like metaphors, positively influence long term recall in infographic charts30.

Transferring the same reasoning to visualiza-tion, a highly valuable aesthetic applied to data may compel the user to engage with it, giving a higher val-ue to the information presented and enabling a better communication.

When it comes to evaluate attractiveness for vi-sualization, some theoretical efforts to identify some kind of criteria have been made, most notably from Lau and Vande Moere. They identified three possible characteristics that determine the engagement of audiences on a visualization: design quality, gener-ally including visual style and user experience; data focus as the core of the visualization is to communi-cate meaning instead of simple facts or trends; user interaction, that involves the aspects related to fluid-ity, engagement and social collaboration31.

Another effort is from David Chek Ling Ngo, that proposed 14 different metrics to quantify the aesthetic layout of an interface: balance, equilibrium, symmetry, sequence, cohesion, unity, proportion, simplicity, density, regularity, economy, homogeneity, rhythm, order and complexity; backed by mathemat-ical models and empirical analysis32.

This extremely rational approach of course is not shared by everyone: others researchers believe that aesthetics can be evaluated without any measur-able characteristics, and is ultimately founded in the human experience. So people can find an artwork or a visualization aesthetically pleasing without know-ing “why”, in a non-discursive way33.

Evaluating attractiveness

28: M. Kurosu and K. Kashimura, “Apparent Usability vs. Inherent Usability: Experimental

Analysis on the Determinants of the Apparent Usability”. Conference on Human factors in

Computing Systems (CHI’95)

29: N. Tractinsky, D. Ikar, “What is Beautiful is Usable”, Interacting with Computers

30: S. Bateman, R. Mandryk, C Gutwin, “Useful Junk? The Effects of Visual Embellishment on Comprehension and Memorability of Charts”.

Conference on Human Factors in Computing Systems (CHI’10)

31: A. Lau and A. Vande Moere, “Towards a Model of Information Aesthetic Visualization.“

International Conference on Information Visualisation (IV’07)

32: David Chek Ling Ngo, Lian Seng Teo, John G. Byrne, “Modelling interface aesthetics”

33: Oliver Conolly, Bashar Haydar, “Aesthetic Principles”

49Art, Science and Desig

While each and every project has its own equi-librium of art, design and science, nowadays it can nevertheless be clearly distinguished if they are ad-dressed toward aesthetic value or scientific clarity, creating a sort of “buffer zone” that sits right in-be-tween them and is currently extremely small popu-lated, if not at all. Design could move in this zone, somehow bringing those two directions together and blurring the line to create a kind of visualizations that fuses them, maintaining readability of the data and also bringing in aesthetic value. This could improve the experience both of experts, creating better tools and practices to handle and work with data, while also creating new artworks to popularize science among the public, that readability and instructional value.


The role of Art About the role of art and artist, there is a very interesting case study from Daniel F. Keefe, David B. Karelitz, Eileen L. Vote, and David H. Laidlaw at the Brown University Visualization Research Lab, where they investigated both the potential role of artists in designing new visualization techniques and in the impact that virtual and augmented reality technolo-gies can have on the future of visualization. They got together programming, art and design students, and a team of scientists to explain the various tasks and topics they had to cover. They started from simple 2D fluid flow visualization, to scale up with increas-ingly difficult task as 3D and visualization in a Vir-tual Reality environment. It quickly became clear that artists can be very useful in the initial design and conceptualization stages, as they often provided new insights to the team. Working together with sci-entists they achieved a very effective feedback loop with them, also influencing the way data was col-lected to better suit the artist’s proposals. Cooperat-ing with designers and programmers was also very proficient for they artists, since they often lacked the technical knowledge to convey their concepts. A use-ful tool that brought all the participants together was the critique and discussion moments, both on a visual and on a scientific standpoint.

Proceeding in the work, some difficulties emerged when approaching VR: the main obstacle with it is the lack of prototyping tools that allow non-experts to work with it. But since VR is such a unique media, it is almost impossible to evaluate and critique a design proposal without trying it in the appropri-ate environment, and nowadays this requires days of specialized programming work, thus breaking the possibility of quickly testing, alter or discard concepts, a methodology widely used and of proven effective-ness in the nowadays design practice. To work around this difficulty, the team realized a quick augmented reality sketching system for the artists, that involved

51Art, Science and Design

a fictional paintbrush that traces “virtual 3D strokes”, that effectively sped up the work and allowed for a better evaluation and refinement process.

Working with VR and data has proven to be a delicate work of balance, because while it allows to represent much more information in an amazing im-mersive environment, it must be clear from the be-ginning how a design idea will work with the actual data to display: it can happen to prototype a concept based on a personal perception of the data to find out that it seems effective, to alter discover that that concept does not go well with the actual data once it’s implemented, and considering how VR is difficult to implement right now it may turn out to be a consider-able waste of time.

In conclusion, this experiment shows a few im-portant points on the relationship of art, science and design: artists proved that their creativity can be very useful in the conceptual and initial designing process to create new techniques, better if in a mul-tidisciplinary team. When it come to virtual reality and similar novel and specific medias it also must be noted that the developing of rapid prototyping envi-ronments should be a priority in the next future, to allow for the experimentation of novel visualization techniques34.

Virtual Reality bloodflow visualization, created with the visualization prototyping system developed at Brown University’s Visualization Research Lab

34: Keefe D, Karelitz D, Vote E, et al. “Artistic Collaboration in Designing VR Visualizations”. IEEE Computer Graphics and Applications 2005

CASE STUDIES5.


There is currently a notable number of data-based projects that crosses ways with art, design and science, using both more “traditional” forms of ex-pression (graphics, video, sculpture…) and new and “non-conventional” medias (sound, interaction, instal-lations). There is a selection of works in different field, including some “institutional” projects created direct-ly by research center and agencies, and some works that does not directly involve data, but acts as means of science popularization, with a very good response from audience. Each project has tags to distinguish its field and features:

case studies selection

1 - R. Ikeda, Supersimmetry - Art, Installation, Particle physics, Music, Projections

2 - Miebacht, Weather Scores - Weather data, Sculpture, Music, Art

3 - A. Parker, Worlds - Video, Planetary Science, Design

4 - eCloud - Weather data, Installation, Art

5 - Living Light - Weather data, Installation, Design

6 - L. Pagliei, Les Invisibles - Music, Particle physics, Art

7 - J. Park, LHC CERN Data Audio Generator - Music, Particle physics, Application, Design, Science

8 - J. W. Tulp, GoldiLocks - Planetary Science, Web, Design

9 - In a nutshell - Web, Video, Animation, Design

10 - NASA Scientific Visualization Studio - Weather data, Planetary sci-ence, Science, Map

11 - NASA Hyperwall Visualizations - Planetary science, Science, Weath-er Data

Please note that some tags will be counted more than once for tha last two cases, which embed

more small projects.

55Case Studies

10

1102

04

06

08

03

05

07

09

01

Design

Art Science

Case studies mapping

Tags recurrencyScience

Weather Data

Planetary Science

Animation

Design

Art

Music

Map

Installation

Particle Physics

Video

Projections

Sculpture

Application


Ryoji ikedaSupersymmetry

Ryoji Ikeda is a Japanese sound and visual art-ist, whose musical interest is mainly in the most “raw” states of sounds, like fundamental tones and various kinds of noise, that he often uses to play with the limits of human earing, with extremely low or high-pitched sounds. His visual aesthetics is based on similar principles, often visualizing the sound he pro-duces with massive, abstract and geometric black and white projection. He often used data, mathematics and physical concepts as “sources” for the composition of his works, and is considered a pioneer of minimal electronic music35.

His project “Supersymmetry” was developed during his residence at CERN in Geneva in 2014 and 2015, where he had the chance to cooperate with the researchers working on the supersymmetry theory: at the time they were running experiments at the LHC particle accelerator to find some yet-undiscov-ered fundamental particles, like the Higgs Boson, that would prove the reliance of the Standard Model. Even if this work does not include data directly from the LHC experiments, it is trying to describe in an ar-tistic and “experiential” way what these physics theo-ries are and imply, so it is more of a conceptual project than an actual data visualization system.

The installation itself is constituted of two sec-tions, named “experiment” and “experience”: the first one revolves around observation of natural phenom-ena, which is exemplified with a number of small

Ryoji Ikeda during a performance

Audio visual installation, 40 projectors, 40 computers w/ monitor, speakers,

2014

35: http://www.ryojiikeda.com/biography/

Supersymmetry (experience), whole view

Supersymmetry (experiment), particular

pellets of different materials, moving on three light-ened surfaces that are constantly changing their in-clination. The pallets move following the angle of the surfaces, colliding with each other. A laser scanning system detects the position of each particle and sends it to the second section, “experience”, where sound and images are produced according to movements and collisions detected between the pellets. This cre-ates a kind of detachments in the user’s perception: while they try to follow and understand at least one of the events, hundreds of other events are happen-ing each second around them, in a way simulating the enormous amount of data and complexity that particle physics researchers have to face with each experiment36.

Supersymmetry (experience), one of the monitors

36: http://special.ycam.jp/supersymmetry/en/work/index.html

Supersymmetry (experiment)

Supersymmetry (experience)


Nathalie’s work focuses on the intersection of art and science, taking data from astronomy, ecol-ogy and meteorology and transforming it into woven sculptures: in this way she uses the natural “grid” that basket weaving creates as a coordinate system to dis-place three-dimensional data. In Nathalie’s opinion, her work pushes the boundaries of traditional ways to interpret and visualize data, playing with the ex-pectations of audience about what is considered to be “science” and “art”37.

Additionally, since 2009 she collaborated with musicians and composers, to transform a series of sculptures she based on weather data into musical scores for them to play: this both gives a diverse emo-tional level to her primarily visual work, and might also reveal patterns in the data that may otherwise remain unnoticed.

Her work can be considered quite rare in data art, mixing together media that are not really much used in the field, thus creating a unique way to see and experience the datasets she uses. Her work also maintains a very “analogic” feeling, relying on an art that requires a lot of manual work and skill as basket weaving and preferring traditional scores and hu-man musicians for her music, rather than relying on electronics music or computer-guided composition as most sound/data artists uses to.

Nathalie miebachWeather Scores

Scuplture, music

Nathalie Miebach

37: http://nathaliemiebach.com/statement.html

61Case Studies

The “Weather scores” exposition

Musical score for “Duet of Blizzards”


“External Weather, Internal Storms” 2009

63Case Studies

“O Fortuna Sandy Spins,” 2013


Alex H. ParkerWorlds

Alex Harrison Parker is a planetary astronomer at Southwest Research Institute, currently collaborat-ing with NASA’s New Horizons mission38. He is active also as a science popularizer, creating illustrations, video and musical pieces based both on scientific data and on topics from his research.

For example, “Worlds: The Kepler Planet Candi-dates” is an animation video he created were he visu-alizes all the Kepler planet candidates around a single star, each one of them with their real diameter, orbit, and speed. It was awarded with the CinèGlobe “Data Visualization Award” in 201339.

The value of this work resides in the very straightforward and clear way it presents such a rich topic: it can be considered a very good work of design rather than art, but nevertheless very interesting, giving in little more than 3 minutes a good perspec-tive on the incredible amount and variety of exoplan-ets that the Kepler space telescope is discovering.

Other projects from him include “Painted Stone”, a visualization of 100,000 asteroids and orbits, ob-served by the Sloan Digital Sky Survey, with the color representing their material composition; “Kepler 11: A Six-Planet Sonata”, a sonification of the transits of the six planets in front of their star in the Kepler 11 plan-etary system, that demonstrate through sound the orbital resonances that system; “Beyond Neptune”, an animation about trans-Neptunian objects, visualizing when they were discovered, their size and orbit.

Dr. Alex H. Parker

Animation, Music

38: http://www.alexharrisonparker.com/

39: http://www.alexharrisonparker.com/datavisualization/

Worlds

Painted Stonemid: Beyond Neptunebottom: Worlds


Aaron Koblin,Nik Hafermaas,

Dan GoodseCLOUD

eCLOUD is a dynamic sculpture made from suspended polycarbonate tiles that can change from transparent to opaque, to simulate the volume and behavior of an idealized cloud. It was created in col-laboration with NASA’s Jet Propulsion Laboratory and the Art Center Collage of Design. The system takes real-time weather data from NOAA and simu-lates how a cloud would behave in the conditions of the picked location40.

The project includes a dynamic display system that shows a preview of the simulated cloud behavior, the current location, and details about the weather data received.

eCLOUD was created for and is installed at the San José International Airport as a centerpiece art-work, as part of the Art & Technology program41.

This is a valuable example of the possible inter-sections of data, art and new media: it uses an inno-vative technology to display real-time data through a very effective visual metaphor, that people passing by can easily understand, together with a notable and positive aesthetic value.

Dynamic sculpture, 2010

40: http://www.ecloudproject.com/tech.html

41: http://www.ecloudproject.com

67Case Studies

eCloud installed in the San Jose Airport

eCloud dynamic displaymid: eCloud installed in the San Jose Airportbottom: demonstration of the liquid crystal tiles technology


Soo-in YangDavid Benjamin

Living Light

Living Light is a functional art installation lo-cated in Seoul, created to inform the citizens about the air quality in the city. The canopy has a city map etched on its glass surface, and is divided in 27 differ-ent blocks representing the different areas of the city where the air monitoring stations are situated. When an improvement in the air quality of a district is de-tected, the corresponding block of the system lights up; otherwise the system is automatically updated every 15 minutes, lighting up all the blocks in order, from the one with the best air quality to the worst one. The structure also responds to text messages, blinking and then sending the data about the specific location that the user requests. “Living Light” is fully integrated with the already existing air-quality moni-toring system active in Seoul, receiving data from its sensors network42.

This is a notable example of public art: its beau-tiful design became an attraction for the city itself, and it is also very useful for the population, inform-ing and raising awareness about the condition of air and pollution in the city.

Public art installation, 2009

42: http://www.interactivearchitecture.org/living-light-2.html

Living Light installed at Peace Park, Seoul


Lorenzo PaglieiLes Invisibles

Lorenzo Pagliei is an Italian composer, electronic musician, pianist and orchestral conductor, currently living and working in Paris at IRCAM as composer and researcher. His work focuses on sounds not re-alizable with acoustical instruments and innovative ways to interact and control sound43.

In 2001 he took part in a collaboration effort made from the Centre des Arts of Paris, aiming at making scientists and artists work together on a piece of art involving concepts from physics and nanotech-nology: Lorenzo worked with the Centre national de la recherche scientifique – CNRS, that was currently working on nuclear magnetic resonance. It’s a new technique to investigate the properties of molecules and nanoparticles, that works by analyzing the par-ticle’s oscillations when hit by an electromagnetic pulse. This kind of data is almost directly translatable into sound, since particles and molecules threated this way behave like regular objects that get hit, vibrating and thus generating sound. Analyzing this oscillating response from molecules allow to deduce its composi-tion and various other properties.

Each particle generates a different sound or rhythm, that were used as the composing material for a 1-hour music piece created for the exposition. “Les Invisibles” was performed again in October 2015, in the Crypt of the Ville d’Orsay, this time as a real-time performance controlled live by the composer44.

Sound Installation, 2001 / 2015

Lorenzo Pagliei

43: http://www.lorenzopagliei.com/site/bio.html

44: Personal interview with Lorenzo Pagliei, February 2016

mid: Setup for the live performancebotton right: Developing of the sound synthesis systembottom left: Visualizations of the results of some experiments

Sonogram of a section of “Les Invisibiles”


JeongHo ParkLHC CERN Data

Audio GeneratorData analysis Software

Jeongho Park is a developer and audiovisual artist that created various data-oriented and interac-tive projects, applications and installations45.

This project is a software application that gen-erates a tridimensional visual models and a sonifica-tion from event data of the LHC particles accelerator at CERN in Geneve. Some events are bundled with the application, and new ones can be download from CERN website, since the application is natively com-patible with their databases.

Each line of the visualization is the path of a particle, and when an event is “played” also a sound from each one of them is generated, with a different frequency according to its mass and properties.

Tools like this prove the didactic and research value that an aesthetic approach can have, giving a useful and explorable 3D view of an event, while the sound gives information in an intuitive way, making easy to spot the general layout of an event, or quickly pinpointing abnormal or strange particles, that will stand out for their different tone.

45: http://jeonghopark.de/z

46: https://github.com/jeonghopark/lhcCernMac

73Case Studies

Screenshot from the application

Interface of the application


Jan Willem Tulp Goldilocks

Data Visualization, website, 2015

The “Goldilocks” name comes from the hom-onym effect, that states that some phenomena must fall within certain margins rather than reaching extremes: this principle is applied to astrobiology to determine when a planet could theoretically support life, since it depends on a series of parameters that must fall within certain boundaries47.

This interactive visualization displays all the known exoplanets (1942 confirmed, at October 9th, 2015) and their host stars. Size and orbital parameters of the planets are based on the observed data, as well as star size, color and position.

Alongside the main visualization, displaying all planets around their stars in their respective posi-tions, there are five other visualizations, more data-oriented, focused on: distance of each planet from their star, underlying the ones in their system’s hab-itable zone; Earth Similarity Index; atmosphere and composition compared to Earth; temperature and mass; temperature of their stars and amount of en-ergy they receive from them.

47: http://tulpinteractive.com/goldilocks/

Various visualizations fromhttp://goldilocks.info/


kurzgesagtIn a Nutshell

YouTube channel

“In a nutshell” is a YouTube channel started in 2013 by a Munich design studio, working with ani-mation and information design48. Their videos deal with various topics, like physics, sociology, computer science, biology, astronomy and politics, and always explain a particular subject related to some contem-porary matter, to mention the most recent topics: the war on drugs, what are red dwarf stars, what is a quantum computer, how Facebook steals video views, and the refugee crisis. It’s always some kind of de-manding argument, that may somehow “scare away” people looking for entertainment: but the channel has nevertheless almost two million subscribers and is getting close to a hundred million visualizations, thanks to the their engaging style of drawing and an-imations, and for the easy but informative way they deal with these topics49.

Very little of their content is related with data itself, even if they use data visualization from time to time, but the value of this case is in the way they are doing science popularization and information, in a way demonstrating that it is possible to talk to regular people about “difficult”, niche and specialized arguments with the right design language.

48: http://kurzgesagt.org/profile/

49: https://www.youtube.com/user/Kurzgesagt

77Case Studies

Various still frames from “In a nutshell” videos


Nasa Scientific Visualization

Studio

NASA Scientific Visualization Studio works to popularize various contemporary science topics cov-ered at NASA, like Earth’s climate or planetary ob-servations. They also cover various techniques, from simple picture elaborations or combinations to ani-mations and various data visualizations.

Projects between the most popular include:Human Fingerprint on Global Air Quality:

thanks to to high-resolution global satellite maps, NASA tracked air pollution trends in the last decade around the world. They compiled a map summariz-ing that data, and a video to explaining how and why air quality is changing aroud the globe.

This project demostrates how valuable emission control laws are being in the United States and in Eu-rope, observing how general air quality improved in these regions in the last years, while emerging and less regulated countries like China or India are seeing a decrease in air quality. Air pollution is linked to oth-er factors too: for example since the beginning of the civil war Syria is decreasing in population due to emi-gration and most economical activities are shutting down, thus, ironically, leading to a better air quality.

Data is collected with the Ozone Monitoring System aboard the NASA Aura satellite50.

50: https://svs.gsfc.nasa.gov/cgi-bin/details.cgi?aid=12094

79Case Studies

“Human Fingerprint on Global Air Quality” map


Dial-a-Moon: This project is mainly a picture vi-sualizer, that takes a date and time as input and out-puts an image of the Moon in that moment, plus some textual informations about phase, apparent diameter, distance and precise positioning. The very high reso-lution and detail of the pictures comes from the fact that they are taken from the Lunar Reconnaissance Orbiter, a NASA probe orbiting the Moon, that pairs its LROC camera to a laser altimeter, LOLA, allowing for a precise mapping of our satellite’s surface. It is also possible to download a high resolution map for each picture, with the name of each location written on it.

Together with the visualization are presented some video explaining related topics, for example how the Moon rotates around Earth and what its li-bration is52.

Five-Year Global Temperature Anomalies from 1880 to 2015: This project from NASA Scientific Visu-alization Studio shows the temperature anomalies on Earth’s surface from 1880 to 2015: this means that the colors does not represent the actual temperature, but the difference from the normal temperature.

Data is taken from the GISTEMP (GISS Surface Temperature Analysis, NASA Goddard Institute for Space Studies) analysis system, combining data from NOAA GHCN v3 (meteorological stations), ERSST v4 (ocean areas), and SCAR (Antarctic stations)51.

Frames of “Five-Year Global Temperature Anomalies from 1880 to 2015”


81Case Studies

Space Weather: This animation shows the hy-pothetical voyage of a photon of light from the center of the Sun to the Earth’s atmosphere. The video dis-plays also various phenomena that are encountered in such a trip: it shows nuclear fusion and ignition processes inside the Sun’s core; loops, flares, promi-nences and coronal material ejections on the surface; and then follow the solar particles until they reach Earth’s magnetic field, causing aurora53.

Still frames from the “Space Weather” video


53: https://svs.gsfc.nasa.gov/Gallery/SpaceWeather.html

Nasa Hyperwall Visualizations

The HyperWall is a visualization system with 8x16 = 128 LCD modular screens, backed up by 128 computational nodes and connected to the Pleiades supercomputer (a 11312 multi-CPU nodes calculator).

NASA internally develops various kind of me-dia and simulation for this system, leveraging both on the huge screen size available and the massive com-puting power behind it.

The visualizations include:Dionysius Crater: It’s a series of pictures taken

from the Lunar Reconnaissance Orbiter of a crater in the Sea of Tranquility, interesting for non-uniform and striated materials of which the crater seems to be composed.Dionysius Crater’s rim

83Case Studies

Pluto’s surface composition: A series of picture of the nano planet Pluto, taken with various instru-ments aboard the New Horizons probe. Together with regular pictures, other instruments toke images in other wavelengths to investigate Pluto’s surface composition. Also, High-resolution b/w images and low-resolution color images where combined to ob-tain some of the most iconic pictures released.

NASA’s Hyperwall

Observations on Pluto’s surface

85Case Studies

GEOS-5 Nature Run Collection: This series of massive visualizations displays data from various simulations about weather and changes in Earth’s atmosphere, run with the NASA Goddard Earth Ob-serving System Model (GEOS-5). Topics covered in-clude the evolution of surface temperatures, clouds formation and movements, presence of water vapor, dispersion of aerosol from dust, biomass burning, fos-sil fuel emission and volcanoes, and winds.

The full simulation took 2 years to complete, running on 3750 processors on the Discover super-computer at the NASA Center for Climate Simulation, producing 400 terabytes of data.

On the other page, top to bottom: Earth’s surface temperature (colors) and outgoing radiation (white);surface (white) and upper level (colors) winds;Dust (red), sea salt (blue), organic/black carbon (green), and sulfates (white).On this page, top: total precipitable water (white) and rainfall (colors, red is highest);middle: aerosol and human-initiated burning simulation.


NOAA Coral Reef Watch 2015: A series of-animated Data Visualizations showing the real time conditions of reef environments, helping to identify where the corals are losing their symbiotic algae, so losing their color and bleaching: a condition that heavily endangers the survival of entire colonies. Data is taken from a variety of satellites, including the polar orbiters Suomi-NPP/VIIRS and MetOp-B/AVHRR, and the geostationary satellites MSG-3, MT-SAT-2, GOES-East, and GOES-West. Data itself consist mainly of temperatures, from which various other indexes are calculated, according to average or in-stantaneous values.

top: sea-surface temperatures visualizationmid left: sea-surface temperature anomalies

visualizationmid-right: accumulated thermal stree visualization

87Case Studies

Solar Wind Strips the Martian Atmosphere: While today Mars is covered by a dry desert and has a very thin atmosphere, scientists agree it used be quite different, with a thick atmosphere and probably bod-ies of liquid water like seas, lakes and rivers, similarly to modern Earth. The change happened because the atmosphere dispersed into space, and the most prob-able cause is solar wind: in this video the simulations made to verify this theory are confronted with the MAVEN probe’s observations of ions leaving Mars’ atmosphere to disperse in space.

top: Solar wind hitting Marsmid left: Mars atmosphere stripped awaymid-right: Ions speeds observed by MAVEN

a design-centered evolution6.


Design as the unifying force

Analyzing projects from both scientific visu-alization and data-driven art, looking at their his-tory and at some case studies, it emerges that, while these disciplines underwent a great evolution and has reached very remarkable levels of advancement, there is still room for notable improvement, especially if we consider the role that design methodology could have in the future of these disciplines.

Nowadays, in fact, SciViz and data art projects are running in two different and almost parallel di-rections, with almost none contact points; but they could both benefit a lot from some kind of interac-tion between them. Design could be the force driving them to cooperate, being almost “equidistant” from both of them, but still close enough to be able to in-teract with both. Recognizing the potential of design for these fields and involving more designers into the developing processes of scientific visualizations and data art would bring a more reliable design method-ology, that would improve, for scientific visualization, important aesthetic qualities like the overall experi-ence and the design of interfaces, easing the work for researchers and allowing the creation of better popu-larizing and generally public-distributed material; at the same time data-oriented art projects could rely on a more solid work methodology and on a good tech-nologic know-how.

There are various examples of designers and re-searchers endorsing the potential of art: to mention one, in the report made for a project experimenting visualization techniques for Virtual Reality, Daniel Keefe and his colleagues stated concluded that “artists [have] key roles in working closely with scientists to design novel visual techniques for exploring data and testing hypotheses”, and strongly suggest a interdis-ciplinary approach to visualization, “Teaching art to computer scientists, computer science to artists”.

91A Design-centered evolution

With the technological evolution of the last decades and the popularization of developing and easy-to-use creative tools, it’s always more common to see projects that involve interaction, immersive video projections, sound and similar state-of-the-art features. In this way the general audience is becom-ing more and more acquainted to this forms of media, and always more difficult to impress.

This implies that any forward-looking project must take into serious consideration the usage of the most interesting and appealing of these technol-ogies: it must not be modernism just for the sake of it of course, but it must be taken into account during the design process that it may be more useful and far more compelling for the audience to rzepresent a certain phenomenon, for example, with sound, or giving the possibility to explore a particular dataset through virtual or augmented reality.

It’s obvious that a weak design idea won’t magi-cally acquire value thanks to the use of any tech-nology, but the right combination of a well-thought concept and the appropriate technical mean will much more easily appeal and attract the public, and the more time they will spending interacting or ob-serving the project the more it will possible for the to become interested in the underlying information and

Usage of New Media

Example of projection mapping, a new media that can be seen more and more in public events


topic.It must also be considered the value of new me-

dia for researchers and professionals, for example: in the medical field the use of virtual reality has already proven its outstanding value both for surgical train-ing and as assistance during surgeries54, and some theorize that technologies such as Microsoft Hand-pose, an extremely precise hand-tracking system, will be paired with already-existing robotic preci-sion surgical arm to use during operations, allowing much more accurate actions and preventing human mistakes55.

The use of sound is also proving its usefulness in the field of astrophysics research: NASA is using data sonification through a software called “Xsonify” to translate observations made in various fields into sound. For example, they translated x-ray observa-tions, radio signals from Jupiter and micrometeorites impacts on the Voyager II probe56. The project was born to help visually impaired researchers, but it has been found to be useful also for normally seeing in-dividuals: for example, for x-ray observations data was usually visualized, but that method proved to be scarcely effective due to the high level of noise and the difficulty to visually identify events and patterns. Moving to a sound-based system proved to be a bet-ter solution to pinpoint different kinds of events, that presents themselves as different timbres, or patterns, audible as rhythms57.

Interactive virtual reality simulation of a brain tumor removal surgical operation.

54: Teodor P Grantcharov, “Is Virtual Reality Simulation an Effective Training Method

in Surgery?” - http://www.medscape.com/viewarticle/575183

55: https://virtualrealityreporter.com/virtual-reality-surgical-medical-application

56: Robert M. Candey, Anton M. Schertenleib , Wanda L. Diaz Merced, “Xsonify sonification

tool for space physics”

57: Wanda Diaz Merced, “Sonification for the exploration of 1D Astrophysics data” - South

African Astronomical Observatory Colloquium, 17 April 2014


Coverage of contemporaryand popular science topics

Considering projects directed at the general public also the choice of the topics to cover can be rel-evant: while most of the population has only a gen-eral and superficial science knowledge, often as old as their high school years, most of them have heard from media of various relevant events in contempo-rary science. For example, they might know about Higgs Boson observation at CERN, the New Horizon probe reaching Pluto, the Zika virus, the discover-ing of gravitational waves, or got interested in fields touched by movies like Interstellar: this can be made use of to engage people in a project and going deeper on the same topic.

It may also be interesting because most of con-temporary science topics are full of open and unan-swered questions, and being presented with an in-development topic can be far more engaging, pushing the audience to wonder and make hypothesis, rather than dealing with a fully known and already dis-closed argument. These kind of discoveries are also changing our point of view on reality itself and may make way to life-changing technologies or scenarios: this also is a possible engagement factor, and at the same time a knowledge to spread in order to limit misinformation and anti-scientific propaganda.

Still frame from a video from “Kurzgesagt – In a Nutshell” explaining quantum computing


Usage of datasets and real-time data

sources

Progress in data-collection techniques in the last years has been incredibly fast and prolific, cre-ating the necessity of always more refined and pow-erful data analysis and visualization algorithms. So, doing research and experimenting new techniques with big datasets and real-time data sources, even if only for aesthetic purposes, can lead to the creation of novel systems that may be useful also in the scientific research field.

Also using real-time data sources can increase the user’s engagement in case of popularization-aimed or exposition projects, creating always-changing sce-narios that can also bring useful information to the public, like in the “Living light” project by David Ben-jamin and Soo-in Yang, “eCloud” by Aaron Koblin et al. and other public data display projects, that serves both as beautiful “public art” or “urban furniture” and as providers of useful and meaningful informations.

NASA Deep Space Network Now visualizes in real-time all communications with far away

spacecrafts and probes


Keeping the data understandable

This is of course the most important require-ment for a project aimed at pure research and in-vestigation purposes, which would be useless if the data is not somehow clearly readable and under-standable. Data can be accessed even if an indirect way: an easy expedient that a lot of works use is to present a general overview of the whole involved data in the main screen, while allowing the user to select one or more data points to access the details about it.

This point may be important also for projects directed to the general public, allowing for a more precise understanding of the presented phenomena and maybe also giving the piece an instructional value. If the data is difficult to understand, maybe because of its unit of measure, or for the very wide order of magnitude, a number of devices can be used to overcome the problem, for example translating the raw numbers with some metaphor related to some more understandable measures, or related to everyday objects. Instead of using magnitude for the brightness of a star the value can be related to the brightness of a far-away light bulb, or power con-sumption/absorption can be related to the amount used by home appliances.


Application of design reasoning

and verifiable methodology

A fundamental step in any data visualization project is to map data to visual attributes and struc-tures. This procedure, to be effective, requires a high degree of design reasoning, especially if it involves the creation of new techniques: in this case it is re-quired to invent an original visual metaphor that fits well with the abstract structure of the data. Consider-ing the three space dimensions, time, all the possible transformations in positioning, size, color and so on, there are almost infinite possible combinations of data elements with visual parameters and attributes, so data-to-form decisions need to be explicit. Even if design reasoning is always present, it may be un-conscious, and if this happens the designer must pay attention to reconstruct and document the path that brought him/her to those particular decisions, and try to objectively evaluate or even test them. Even if there is no “right” or “wrong” in this field, there are various ways to support a design choice: from es-tablished empirical results, like the Gestalt laws, to various studies on human perception, or by feedback from other designers or the users themselves. Build-ing up this kind of objective documentation on visual mapping decisions, including the use of new technol-ogies, will increase the amount of good examples and best practices that future projects will be able to build upon, in a very open-source manner.

While this aspect may be already of use for some or a minor addition to some other’s designer workflow, it may a different matter for the art uni-verse: since it’s a personal form of expression from the artist to the world, it’s not obvious that the author has the intention to explain the thought process behind his/her artwork, or it can even be interested in giv-ing the idea that the inner processes behind an art-work are just purely irrational, or dreamt up, to leave the work open for different possible interpretations. However, if a piece is dedicated to a purpose as sci-ence popularization or raising awareness about some

3D visualization design for the bat flight problem inspired by the Miró painting The Gold of the

Azure, one of the results of Keefe’s project

97A design-centered evolution

phenomenon like global warming, it should be con-sidered in a slightly different way. Artists working in this particular sub-field should be aware that the value of their work can be also in the research and in the thinking they made and in the rationale in their decisions; and that their work has the possibility to ease scientific research if they invent a valuable data rendition technique.

There are some cases of good documentation of the design processes in the last years: for example, Daniel F. Keefe et al. explains in a IEEE Computer Graphics and Applications article the process and the results of a project that involved both professors and students to develop new visualization techniques in a virtual reality environment. This kind of docu-mentation can be extremely valuable: VR as a work-ing technology is still in its first years, so there is no consistent literature or examples about possible uses in design and visualization; in fact, also Keefe and his colleagues highlighted the complete lack of design tools to rapidly sketch and prototype58. Also Byron and Wattenberg explained in detail in a paper the design rationale behind their Streamgraph, an new kind of stacked graph, from the algorithmic level to every visual choice they made59, creating a collection of generalizable knowledge that can benefit other de-signers and developers who want a similar approach.

Together with the current approach of inter-action design, were “aesthetics to be an integral part of functionality, with pleasure a criterion for design equal to efficiency or usability”60, these and other ex-amples show and explain the potential of integrating design thinking in the research process, how aesthet-ics is valuable to any visualization, and how it can be somehow estimated and evaluated.

Byron’s Streamgraph example

58: Keefe D, Karelitz D, Vote E, et al. Artistic Collaboration in Designing VR Visualizations. IEEE Computer Graphics and Applications 2005; 25(2): 18–23.

59: Byron L and Wattenberg M. Stacked Graphs - Geometry & Aesthetics. IEEE Transactions on Visualization and Computer Graphics 2008; 14(6): 1245–1252.

60: Gaver B, Dunne T, and Pacenti E. Design: Cultural Probes. interactions 1999;

Project: Sounds from the sun7.


Vision is widely considered to be the most im-portant and defining sense of human beings: it is in-deed true that the ability to see colors, for example, gave a great evolutionary advantage to our ancestors; and it is also one of the rarest and most powerful fea-tures between mammals, reserved almost only to hu-mans and some great apes.

At the same time, the hearing system of man-kind is considered to be less evolved and refined than the one of other species of mammals: most of them have a wider range of audible frequencies, are more sensible and can better localize the source of sounds around them. The hearing system of humans was, and still is, nevertheless fundamental for the survival of our specie, being the most effective way to detect and localize possible dangers: a sudden noise startles us, and triggers in the brain and in the body a chain reaction that gets us ready either for fighting or for fleeing.

The effectiveness of this system is due to a series of different reasons:

-Hearing is much quicker than seeing. Vision is a slow chemical process: molecules in the retina change their characteristics when struck by light, this change is transformed into electrical signals that the brain has to deeply elaborate to transform them in the images we see. This process is considered to take between 1/30 and 1/20 of a second, resolving in the general knowledge that we can see no more that 20/30 different pictures per second. We can express this as a value with a unit of measure: 30 Hz. Mean-while hearing is based on a quick mechanical pro-cess, with the hearing bones vibrating and exciting a nerve connected to the brain. In result this process is much quicker, allowing mankind to process sound with a resolution 1.000 times greater, around a maxi-mum value of 20.000 Hz. In other words, the shorter event that the eyes can perceive has to last at least 50 milliseconds, while the shortest hearable event can

Sound and psychoacoustics

101Project: Sounds from the Sun

last around 0.05 milliseconds.- Vision is focused in front of us, while we can

hear all around us. Humans, mammals and other ani-mals can only see in front of them, and checking all around us require to move our whole body, and this process of course takes some time. We also have to focus on what we’re seeing to recognize if something may be of our interest. On the other hand, hearing is always working in every direction around us, and it usually takes way less than a second to identify the location, possible source and our possible interest of a sound.

- We can close our eyes, but never our ears. We can effectively “shut down” our seeing in any mo-ment by just closing our eyes, becoming visually iso-lated from the surroundings, but there is no effective way to shut down our hearing. Even while sleeping, it is easier to be awakened by a noise than by a change of light. This is linked to other interesting features of our hearing system: for example, it is very easy to be-come acquainted to, ignore, and do not hear anymore the majority of background noises. A constant sound is taken for “granted” from our hearing system, and so ignored to focus on more interesting sounds, while we instantly notice any new sound that appears around us, or even any relevant change in the sound scape surrounding us.


The peculiarities explained, together with oth-ers of minor relevance, can have some interesting consequences. For example, the extreme precision of the hearing system enables us to easily detect both established patterns and irregularities in them: for example, we instinctively detect and recognize tem-poral patterns as rhythms, often following them with some kind of body movement. In the same manner we can notice if something breaks out of an estab-lished pattern, breaking a rhythm.

The mechanisms applies in the frequency do-main: we instantly distinguish an harmonic sound (where all sounds have a frequency multiple of the same number) from any kind of non-organized sound, that we call noise. If any kind of “harmonic context” is established, like an orchestra playing a classical piece, any kind of sound incoherent with that background pops up, for example a badly tuned instrument.

This kind of ability has, in fact, already prov-en its value for data investigation: NASA is using it with its “Xsonify” software, to translate observations made in various fields into sound. For example, they translated x-ray observations, radio signals from Ju-piter and micrometeorites impacts on the Voyager II probe. The project was born to help visually impaired researchers, but it has been found to be useful also for normally seeing individuals: for example, for x-ray observations data was usually visualized, but that method proved to be scarcely effective due to the high level of noise and the difficulty to visually identify events and patterns. In this specific case, the ability of our hearing system to ignore background noise made easier for researcher to isolate interest-ing events, while temporal patterns in the data that corresponds to repeating or cycling phenomena are easier to find, also quickly spotting irregularities that are much more difficult to identify with traditional visualization.

applicationsto data


the Solar Wind and its consequences

The Sun is Earth’s source of energy, in the form of light and heat, but its activity does not limit to that: it also emits every other form of electromagnetic ra-diation, from radio to gamma waves, together with a massive stream of plasma and charged particles, like protons, electrons and alpha particles. Its magnetic field extends well beyond the eight planets of the So-lar System: this is constantly changing due to Sun’s rotation and activity, and has a notable influence on each planet.

The variations in these phenomena leads to peak values that can influence our life on Earth: the most harmless and spectacular being aurora borealis and australis, caused by the interaction of high amounts of charged particles from the Sun with Earth’s mag-netic field. The most powerful emissions, classified as Solar flares, Coronal mass ejections and Geomag-netic storms, in order of magnitude, can have far less desirable effects on our technological lives: they can disturb and create malfunctions in the communica-tion and geolocation satellites in geostationary orbit; on Earth they can disturb and interrupt wireless net-works, like cellphone network, influence the behav-ior of compasses and other location and orienting de-vices, and even induce huge electrical currents in the Earth’s crust. To mention the most catastrophic event linked to solar activity, a geomagnetic storm occurred on March 13 in 1989 caused first the lost of control of various satellites in polar orbit around Earth, inter-rupted the communication with GOES weather satel-lites, caused anomalous operations to NASA’s TDRS-1 communication satellite, and after only 90 seconds broke the majority of power breakers in Quebec pow-er grid, causing a massive 9-hours blackout on a large area of Canada61.

Above: a mid-level solar flare emitted on Jan. 12 2015, pictured by NASA’s Solar Dynamics Observatory

61: Morin, Michel; Sirois, Gilles; Derome, Bernard (13 March 1989). “Le Québec dans le noir” (in French). Radio-Canada: http://archives.radio-canada.ca/environnement/catastrophes_naturelles/clips/14150/


Seeing the consequences that Sun can have on Earth, the close observation of its activity has grown in interest in the last decades: since 1995 NASA and ESA launched 4 different satellite probes to moni-tor solar wind: Solar and Heliospheric Observatory (SOHO), Global Geospace Science (GGS) Wind, Ad-vanced Composition Explorer (ACE) and Deep Space Climate Observatory (DSCOVR). Observatories in outer space are much more effective than Earth-based ones: our planet’s surface is almost completely shielded from solar wind by its magnetic field, thus making impossible any kind of direct observation, and only some indirect evaluations can be made for example observing some atmospheric phenomena. All of these probes are orbiting between Earth and Sun at a distance around 1’500’000 kilometers from our planet: their signals take around 5 seconds to reach us, while solar wind has a largely lowest speed and takes around one hour to cover the same dis-tance62. This gives us at least a minimum time to get ready for exceptional solar activities, to minimize the consequences on our technological infrastructures, since as of today we have no model or way to fore-cast solar activity: in fact, this is another challenging

Solar Wind Observation

ACE satellite during preparation and testing phases

62: Morin, Michel; Sirois, Gilles; Derome, Bernard (13 March 1989). “Le Québec dans le

noir” (in French). Radio-Canada: http://archives.radio-canada.ca/environnement/catastrophes_

naturelles/clips/14150/


objective for astrophysicists, that hope, with the data collected by these probes, to design a model of solar dynamics good enough to forecast solar wind and its peak activity.

The forecast of dangerous events in solar wind is currently based on the ACE Real Time Solar Wind program, which issues warnings and alerts for any-one with systems sensitive to high levels of magnetic activity. Four of the instruments on board of ACE produce a low-resolution continuous stream of data that can be received from different stations on Earth, the stations forwards the data to the Space Weather Prediction Center in Colorado, which delivers it to other centers around the world and is also published on the World Wide Web. This whole process takes 5 to 10 minutes, from the measurements took by ACE instruments to the to the publication on the SWPC website, available on http://www.swpc.noaa.gov/products/ace-real-time-solar-wind.

The solar arrays on NOAA’s Deep Space Climate Observatory spacecraft, or DSCOVR, are unfurled in the Building 1 high bay at the Astrotech payload processing facility in Titusville, Florida, near Kennedy Space Center.


Considering on one side the particularities of our hearing system, the differences with visual per-ception and, on the other, the continuous and real-time nature of the data taken into account, it can be interesting to consider a different approach to the rendition of solar wind data, that leverages on dif-ferent mechanisms of perception rather the purely visual one; and since this kind of data needs to be constantly monitored, but at the same time is almost constant or with small variations of little interest, it can pair well with the innate abilities of our hearing system to ignore constant background sounds, while promptly reacting to changes in it, without requiring the specific attention and focus of the listeners, that can, in this way, prioritize other more demanding tasks.

So, in my project I took into account the differ-ent phenomena and their parameters monitored by the ACE Real Time Solar Wind program: energy of charged particles stream, components of the solar magnetic field and density, speed, temperature and density of the plasma stream.

Each phenomenon is translated with a specific sonification system that reflects its characteristics:

The particle stream is rendered by a discrete stream of microsounds, where the energy is mapped to the frequency of the sounds;

Sound rendition of Solar wind data

Data about Magnetic Field & Solar Wind Electron Proton Alpha Monitor from the SWEPAM instrument, as presented on http://www.swpc.noaa.gov/products/ace-real-time-solar-wind


the plasma flux is rendered by a noisy, continu-ous sound, where the temperature is mapped to the timbre of the sound, the density to width of the noise and the speed to the central frequency;

the magnetic field is rendered by an harmonic sound similar to a chord, where the total magnitude is the fundamental pitch and each component of the magnetic field corresponds to the volume and beating cycle of each harmonic.

These choices are not casual, and are made for different reasons: first of all, metaphorical, their per-ceptions mimic the nature of the phenomenon they replicate, and the sound parameters are mapped tak-ing into consideration which one of them may better represent their physical correspondence.

Another aspect taken into consideration is if the sounds can be identified even if played together: in this case the sharp attacks of the particles is always clearly distinguishable from the other continuous sounds; while these other two are distinguishable from two different aspects; they take different regis-ters of the audible spectrum, and also the harmonic nature of the magnetic field rendition stands out from the noisy sound generated by the solar wind plasma.


The data is published in form of plain text on the http://services.swpc.noaa.gov/text/ web direc-tory, in three different text files according to the in-strument that generates the data, and is updated ev-ery minute for solar wind plasma and magnetic field, and every five for the electron and protons flux. Data is retrieved from those files within thirty seconds from a Processing script, that translates the plain text in scientific notation into the regular positional nota-tion, labels and then sends them via OSC to a Max/MSP patch. This passage is required because Max/MSP cannot directly use values in scientific notation written as strings in plain text, while Processing can translate them in a very efficient way.

In Max the values are routed to the proper sound generator, that adapts the data to the appropri-ate ranges of values, and then transforms them into parameters for the sound synthesis engines.

Techical aspects

Source data in raw format, as presented on the SWPC web directory


SoftwareFlow Chart

Data on SWPC website

Download on local computer

Conversion to positional notation

Labeling

OSC send

Values to sound parameters mapping

Sound synthesis

Spatialization

Output

Wait 30 seconds

Processing

Max/MSP


The described system has been tested for some days, both from myself to refine the initial calibra-tion, and from ten non-trained users with only a gen-eral knowledge of the involved phenomena, to verify the capability of the system to effectively translate into distinguishable and perceivably different sounds the original data.

The system that proved most quickly its effec-tiveness was the “particle synthesis” applied to the proton and electron stream: the only adjustment re-quired was to double all to frequencies generated, to create a clearer difference between the low-frequen-cy sounds associated to the low-energy particles and the highest pitched sounds related to the high-energy particles. For most of the time the stream is constitut-ed mainly of some thousands of low-energy particles per second, with some dozens of high-energy parti-cles. Occasionally this balance changes, and the solar wind composition shifts, with the most energetic par-ticles increasing their density by ten or more times.

After applying the frequency range adjustment, I requested to the users to say whenever they could notice this shift while I was checking the real-time data: the change was correctly noticed in almost ev-ery occasion, both if the subject was paying attention and if they were involved in casual chatting.

The subtractive synthesis engine working on the solar wind plasma data required some more test-ing and refining: the speed and density parameters, mapped respectively to the center frequency of the filter and to its bandwidth proved to be quite effective and describing, with the changes being almost imme-diately noticed by the test subjects, even the less rele-vant ones. The last parameter, the temperature of the plasma mapped to the timbre of the noise, proved to be more difficult to render effectively, mostly because of the wide range of possible values, from 1’000° K to 1’000’000° K. Small changes, around 50% or less of the previous value, can be difficult to detect without

TestingProcess


some focus; anyway this parameter has often very noticeable and sudden changes, with the value dou-bling or increasing even more, often in less than a minute: these shifts became clearly noticeable with a careful calibrating of the related parameters, allow-ing to pin down most of the relevant changes in the phenomenon.

The “harmonic synthesizer” connected to the magnetic field parameters proved to be the most dif-ficult to map, and various different approaches were tried before getting to a satisfying one. In the defini-tive system the total magnitude of the magnetic field is considered to be the most relevant, and thus is con-nected to the fundamental frequency of the sound generated, which is the most clearly distinguishable parameter. The three components, x, y and z, of the magnetic field are mapped to the amplitude of har-monic sounds multiple of the fundamental: in this way the overall character of the sound changes with the “balance” of the magnetic field, exposing the dom-inant component. In this way sudden changes both in the magnitude and in the rotation of the magnetic field became clearly audible.

Sonograms of the three sound synthesis systems, top to bottom: particles, solar wind plasma, and magnetic field

ReferencesConclusionsThanksColophon

114

BibliographyRONALD BAECKER, Sorting Out Sorting: A Case Study of Software Visualization for Teaching Computer Science, “Software Visualization: Programming as a Multimedia Expe-rience”, MIT Press,1998,pp. 369 - 381

LEE BYRON, MARTIN WATTENBERG, Stacked Graphs – Geometry & Aesthetics, “IEEE Transactions on Visualization and Computer Graphics” ,Vol 14 Issue 6,2008,pp. 1245 - 1252

ROBERT M. CANDEY, ANTON M. SCHERTENLEIB, WANDA L. DIAZ MERCED, Xsonify Sonification Tool For Space Physics, 12th International Conference on Auditory Display, London, UK, June 20-23, 2006

CHARLOTTE HESS, ELINOR OSTROM,IDEAS, ARTIFACTS, AND FACILITIES: Information As A Common-Pool Resource, “Law And Contemporary Problems”, vol 66, Winter/Spring 2003, pp. 111 - 145

DANIEL F. KEEFE, DAVID B. KARELITZ, EILEEN L. VOTE, DAVID H. LAIDLAW, Artistic Collaboration in Designing VR Visualizations, “IEEE Computer Graphics and Applications”, March/April 2005, pp. 18 - 23

DANIEL F. KEEFE, DANIEL ACEVEDO, JADRIAN MILES, FRITZ DRURY, SHARON M. SWARTZ, DAVID H. LAIDLAW, Scientific Sketching for Collaborative VR Visualization Design, “IEEE Transactions On Visualization And Computer Graphics”, Vol 16 n. 4, July / August 2008, pp. 1 - 13

ANDREA LAU, ANDREW VANDE MOERE, Towards a Model of Information Aesthetics in Information Visualization, Information Visualization IV, 2007

J. A. LEWIS, E. E. ZAJAC, A Two-Gyro, Gravity-Gradient Satellite Attitude Control System, “The Bell System Technical Journal”, November 1964, pp. 2705 - 2765

LIPYEOW LIM, HAIXUN WANG, MIN WANG, Semantic Queries by Example, 2003

DAVID CHEK LING NGO, LIAN SENG TEO, JOHN G. BYRNE, Modelling interface aesthet-ics, “Information Sciences”, 152, 2003, pp. 25 - 46

BRUCE H. MCCORMICJ, THOMAS A. DEFANTI, MAXINE D. BROWN, Visualization in Scientific Computing, “Computer Graphics”, ACM SIGGRAPH, July 1987

115

ROBERT K. MERTON, The Normative Structure of Science, “The Sociology of Science: Theoretical and Empirical Investigations”, University of Chicago Press, 1973

A. MICHAEL NOLL, Computer-Generated Three-dimensional Movies, “Computers and Automation”, Bell Telephone Labs., November 1965, pp. 20 - 23

A. MICHAEL NOLL, Computers and the Visual Arts, “Design and Planning 2: Computers in Design and Communication”, Hastings House, 1967, pp. 65 - 79

THERESA-MARIE RHYNE, Information and Scientific Visualization: Separate but Equal or Happy Together at Last, “IEEE Visualization”, 2003

WARREN SACK, Aesthetics of Information Visualization, “Context Providers: Conditions of Meaning in Media Arts”, Intellect Ltd, 2011, pp. 123 - 144

ANDREW VANDE MOERE, HELEN PURCHASE, On the role of design in information visualization , “Information Visualization”, 2011, pp. 356 - 371

FERNANDA B. VIÉGAS AND MARTIN WATTENBERG, Artistic Data Visualization: Be-yond Visual Analytics, “Online Communities and Social Computing”, 2007, pp 182-191

E. E. ZAJAC, Computer-Made perspective Movies as a Scientific and Communication Tool, “Communications of the ACM”, Vol 7, Num 3, 1964, pp. 169 / 170

116

SitographyAT&T Archives, “Simulation of a Two-Gyro Gravity-Gradient Attitude Control System” - http://webcache.googleusercontent.com/search?q=cache:wlCuGlowlpgJ:techchannel.att.com/play-video.cfm/2012/7/18/AT%26T-Archives-First-Computer-Generated-Graphics-Film+&cd=1&hl=en&ct=clnk&gl=it

Ronald M. Baecker, “Sorting Out Sorting” Video - https://www.youtube.com/watch?v=SJwEwA5gOkM

BBC Data Art, “A Quick Illustrated History of Visualisation” - http://data-art.net/resources/history_of_vis.php

Tim Berners-Lee, “The next Web of open, linked data” - http://www.dailymotion.com/video/x8omty_tim-berners-lee-the-next-web-of-ope_tech#.UVWqxhnGrJw

Tim Berners-Lee, “Linked Data” - https://www.w3.org/DesignIssues/LinkedData.html

Jorge Camoes, “Infographics vs. Data Visualization” - http://www.excelcharts.com/blog/infographics-data-visualization/

Wayne Carlson, “A Critical History of Computer Graphics and Animation, Section 18: Sci-entific Visualization” - https://design.osu.edu/carlson/history/lesson18.html

Simon Chignard, “A brief history of Open Data” - http://www.paristechreview.com/2013/03/29/brief-history-open-data/

Centre des Arts, “Invisible et Insaisissable” - http://www.cda95.fr/content/invisible-et-insaisissable

Michael Friendly, “Milestones in the History of Scientific Visualization” - http://www.datavis.ca/papers/vita/Friendly08aaas.html

MARIEKE GUY, “OPEN ACCESS TO RESEARCH DATA – TIMELINE” - http://access.okfn.org/2015/04/30/open-access-to-research-data-timeline/

Jeremy Norman, “Edward Zajac Produces the First Computer-Animated Film (1963)” - http://www.historyofinformation.com/expanded.php?id=1002

Barack Obama, “Transparency and Open Government” Memorandum - https://www.whitehouse.gov/the_press_office/TransparencyandOpenGovernment

117

Alex Harrison Parkers’s website - http://www.alexharrisonparker.com

Lisa Charlotte Rost, “The Line between Data Vis and Data Art” - http://lisacharlotterost.github.io/2015/12/14/The-Line-between-Data-Vis-And-Data-Art/

Moritz Stefaner, “Little Boxes” - http://well-formed-data.net/archives/1210/little-boxes

Edward Tufte, “Scientific visualization: evidence and/or fireworks” - http://www.ed-wardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0002T8

Edward Zajac, “Simulation of a two-giro gravity attitude control system” video - https://www.youtube.com/watch?v=m8Rbl7JG4Ng

The objective of this thesis is to present some possibilities for the improvement of a field, scientific visualization, that has seen a steady increase in its importance since its beginning.

First of all, it has been necessary to make clear, both for me and for the possible readers, how scien-tific visualization can be defined, explaining why and how it is different from the most generic information visualization projects that we are more and more used to see almost every day, ranging from user-generated statistics to popular visualizations spreading through Internet about any possible field of knowledge or in-terest. The fact that it requires a deep understanding of the underlying scientific phenomena pushed it in a niche, effectively separating it from the more ge-neric fields of visual arts and design that nowadays are pushing experimentation in information design, especially thanks to the technological innovations of the last decades. These innovations, for example vir-tual reality, nowadays are almost confined in the field of entertainment, while they have a great underly-ing potential for visualization and design for a great number of different topics.

In fact, the interest for contemporary science topics is well alive, and the projects created by art-ists and designers in the last years, thanks also to the Open Data revolution, demonstrate this, together with the some of the possibilities that new forms of media could have to investigate even the most com-plex concepts related to science, medicine, physics, and so on.

There are nowadays some very prolific exam-ples of cooperation between scientists, artists, pro-grammers and designers, that created some new and novel ways to approach visualization, and more gen-erally rendition, of various data and phenomena, but they are still considered “extraordinary” and “experi-mental” examples, while my research suggests that that kind of approach should become more and more

Conclusions

119

common in the next years, involving artists and de-signers not only in the finishing phases of visualiza-tion projects but also in the concept and development parts: this could bring great positive effects both for researchers, that could have better instruments, in-volving new technologies and approaches to investi-gate their fields; and also for the general public, that could have access, for example, to pieces of public art with an informative value, supported by strong sci-entific principles.

The project I developed demonstrates how a new and different approach, that revolves around the conscious use of sound, can be an effective and useful way to to represent real-time data, even in a mission critical environment.

To conclude, it has been a difficult but satisfying work, that required a broad research and the concep-tual effort to put together many different fields, that I hope will at least contribute to open the discussion on how to reconsider the respective roles of science, art and design, how they could interact for us and how they could positively effect our lives in the future.

121

First of all I have to thank my family, that made this 2 years experience in Venice even possible, sup-porting me in all the ways possible for them.

A huge thank you goes to the colleagues I met here in Venice, the ones that became some of my best friends and really my other family, making this amazing city not only the place where I studied, but also my home and the place I loved the most to live in. Your caring support made this experience a wonder-ful one, that made me grow way more that you all can imagine.

A special thank goes to my relator, Marco Fer-rari, that really guided me in the right way to face this thesis project, starting from my initial vague and unclear concepts about the themes I wanted to inves-tigate. I realize that my previous formation and my way of working may not entirely fit in this field of design, but I guarantee that I absorbed the most that I could from each project I realized and each course I took here, and for that my thank goes to Marco and all the other professors.

Thanks

122

ColophonNew Media for Scientific Data Visualization

IUAV University of Venice

Design and Planning in Complex Environments Department

Master Degree in Product and Visual Communication Design

AA 2014 / 2015

Supervisor:Marco Ferrari

Typefaces:Aleo, Alessio LasioBebas Neue, Ryoichi Tsunekawa

Software:Adobe InDesign CCAdobe Illustrator CCAdobe Photoshop CCProcessing 3.0Nodebox 3.0Cycling ‘74 Max 7

New Media for Scientific Data Visualization

Documents

Transcript of New Media for Scientific Data Visualization