Author Archives: Susan

Using Google Fusion Tables as a web application database

Databases are a great way to store data that’s used for visualizations, web apps, or any web-based program where you a) need to update the data on a quasi-regular basis and/or b) you need to be able to show only a subset of all the data you have. The trouble is, setting up an entire database system costs money and requires security, both at levels that may be overkill for individuals and small publications. Google Fusion Tables (through its API provides many of the features of a database, but without the setup and security costs. Even better, the main interface to your data is essentially a spreadsheet, meaning that it can be edited and updated by anyone, not just people who feel comfortable with coding and data formats.

This post is a basic walk-through of how to use Google Fusion Tables as a database for your visualization/webapp. For step-by-step videos of essentially this process, you can check out these videos.

Continue reading

Sound Advice: Embedded audio in Flash

There are generally two methods available for integrating media content – whether photos, audio or video – into Flash.

The first is to embed the media directly – that is, actually include it as part of your Flash file. The second is to load the media on demand from another location only at the moment (or just before) it’s actually needed.

The two methods impact your user’s experience differently. If you embed the media, all of it is instantly available as soon as your movie loads, guaranteeing a relatively seamless experience. The flip side of this is that embedding all that media can make your Flash (and swf) file much, much larger, making it take substantially longer to load in the first place. Generally, you only want to make users wait for content they’ve “asked for,” i.e. only load the necessary media until the user takes an action (clicking to another panel, for example) that requires more be loaded. The assumption is that users won’t really mind waiting for content they’ve indicated they want, but they’ll get annoyed if they have to wait for that 4-minute video interview to load if they’re only really interested in the overview. At the same time, a 3-second delay to hear a 10-second audio clip (especially if there are many of them) is frustrating, too. You’ll need to assess both the media you’re using and how it integrates into the overall interaction experience on a case-by-case basis to find the best solution each time. But you’ll also need to test frequently – and definitely on a web server: your users won’t have everything sitting on their desktop the way you do.

So, if you’ve determined that embedding your audio is the way to go, here are the steps for a setting up some basic play/pause functionality:

1. Import it! In Flash, choose File>>Import>>ImportToLibrary and import the mp3 you want to embed. It will appear in the Library panel under its file name, with a little speak icon next to it.

 


 

 

 

 

2. Export it! (for ActionScript) Right click on the sound in the Library panel and choose “Linkage…” from the dropdown. A window called “Linkage Properties” will pop up.

 

 

 

 

 

 

 

 

3. Click the “Export for ActionScript” check box (the third check “Export in first frame” will be selected automatically, this is fine). In the input box labeled “Class:”, give your sound a legal name – this means it can include no spaces, no periods, and cannot begin with a number. It’s ok that it won’t match what’s in the Library panel. You’ll want to copy the name you choose, as you’ll need it for the next step. Because my Library name began with a number I renamed mine to “ReginaSpektorFidelity”.

Now you need to need to describe your sound to Flash in code. To do this, we’re going to create a sound “object”, like so:

 

So what’s all this? When you use the keyword “var” (which stands for “variable”), you’re basically telling Flash to create a container in memory and label it so you can refer to it later. In this case, the label I gave the first container was “mySound”, and the second one is called “mySoundChannel”. But I can’t just say to the program “Hey, save some space”; I have to give it some idea of what type of thing is going to go in there. That’s the part after the colon: in the first case I’m saying “It’s going to be one of those sounds that I gave the name ‘ReginaSpektorFidelity'”; in the second case it’s going to be a SoundChannel (a special type defined by Flash – which we know because it turns blue). Finally, in the case of “mySound”, I’m actually going to tell it to put something in the container – in this case the sound itself. To tell the program to put something in a container, we use the equals (=) sign, and to tell it to make a copy of something in the Library, we use the “new” keyword.

Now that we’ve created our sound in code, we need to explain to our buttons how to control it. We’ll write event listeners (more on that here), and then we’ll put the code for controlling the sound inside them. For that, the code looks like this:

Some of this seems pretty obvious: when the user clicks “myPlay”, we tell “mySoundChannel” to play “mySound”; when the user clicks “myPause”, we tell “mySoundChannel” to stop. But what about the

if(mySoundChannel != null){

}

part? Look again at the second line of code, where we named the “mySoundChannel” container and told Flash that a SoundChannel was going to go in it. We didn’t actually put anything in that container. Nothing is in there until the line that says

mySoundChannel = mySound.play();

But what if someone accidentally clicks the pause button before they hit the play button? It doesn’t make any sense to try to “stop” a sound that isn’t there. So we check to make sure that the “mySoundChannel” container isn’t empty before we try to stop it. That’s what the “mySoundChannel != null” part checks for. “!=” is code for “does not equal” or “is not”. So in English what we’ve said is:

“If the ‘mySoundChannel’ container actually has a sound playing in it, stop that sound.”

In the previous event listener we use a similar statement to check if our sound is already playing, and stop it if it is. Otherwise, Flash will happily play the sound in an overlapping loop if the play button gets clicked more than once.

You may be asking yourself: ok, but why all this “SoundChannel” nonsense? Why can’t I just play my sound directly? The answer is: I have no #@?*$&! idea. It’s just the way Flash is set up. If  you ever find out, let me know :)

Exploring America: Visualizing Data with Google Fusion Tables

Interactive maps and charts can be a great way to add interest and visual appeal to a primarily textual work of journalism, and in a newsroom there are many times when it’s only after the article has been largely written that we think to create them. But visualizations can also serve as essential reporting tools in and of themselves, allowing journalists to see patterns in otherwise impenetrable data sets – patterns that can provide essential leads to finding interesting stories in the first place.

This habit of pattern-finding and data analysis is already well known in the area of journalism usually known as “computer-assisted reporting” (CAR), and where it exists is usually the purview of perhaps one or two specialist reporters. Typically, the tools associated with CAR have been both reasonably expensive and time-consuming to learn: database technologies like Microsoft Access; mapping technologies like ArcGIS. Few individuals and not all organizations could realistically afford the software, training and personnel required to do this kind of work.

At the infamously (and in this case, happily) exponential pace of technological evolution, however, there are now free, user-friendly tools whose power and versatility is rapidly surpassing the more “traditional” tools of CAR. A prime example of this Google Fusion Tables, which combines the essential functions of a database (large-scale data storage, powerful sorting and filtering, the ability to merge tables) with the robustness of Google’s mapping resources (and their many charting tools).

A quick walk-through of how to use the the merging and mapping functions is provided through the Fusion Tables Help, of which the below is an edited and annotated version. To follow along, you’ll need to be logged in to a Gmail account.

To start, click on each of the links below (these should come up in new browser tabs):

110th US Congressional District Outlines

2008 1-year American Community Survey (ACS) Data

You’ll notice that these pages basically looks like giant spreadsheets, and this is essentially what they are. Looking at the column headers, take a moment to notice that the format of the Outlines table’s “id” column and the ACS table’s “Two-Digit District” column is very similar.

Looking a little further, you’ll see that in the upper-right there is a count of the number of rows in the table (e.g. “1100 of 436″ for the ACS data); clicking on the “Next>>” link will show you the next 100 rows.

Just below the “Next>>” link, in the gray title bar, you’ll see a square button with a small triangle in it – this is the subtle clue you’re given that there are more columns of data than what you currently see. If you begin clicking this in the ACS data set, you’ll quickly discover the many, many columns that this table contains.

Browsing that many columns of data is tedious, though, and impossible to analyze. So let’s get it mapped instead so we can see really see what’s here.

In the Outlines table, click the “Merge” link in the blue bar. In the box on the left, you see radio buttons for each of the columns in that table, with “id” selected. Above the empty box on the right are the instructions: “Merge with” followed by an input box and a GET button. Ignoring the dropdown, paste the url of the ACS table (http://www.google.com/fusiontables/DataSource?dsrcid=237928) in the input box and click “GET”.

 

 

The right-hand box now contains radio buttons of every column in the ACS table – but what it wants you to do is tell it how to match up the information in the two tables. Remember How “id” and “Two-Digit District” looked pretty similar? Make sure those are the radio buttons selected and then type a name for your about-to-be-merged table into the input box labeled “Save as a new table named”. Finally, click the “Merge Tables” button.

 

 

 

Very often a red box with the warning “Could not merge tables” will appear for a moment. Ignore this and wait for the page to finish loading.

You’ll now be looking at a new Fusion Table with the name you entered earlier. The columns from the first table have a white background; those from the merged or “joined” table are pale yellow. Because we connect outline information with data, we can now see that data visualized by selecting “Visualize >> Map” in the blue bar. After several seconds, you’ll see a bright red Google Map of the U.S., with gray outlines marking the 110th Congressional District boundaries.

 

Click somewhere on the map. After a moment, a balloon pops up with a readout of the first 10 columns of table data for that area. That’s a little useful, but if we were interested in data only about one district, it would have been just as easy to read from the table. Instead, click the “Configure styles” link above the map.

In the “Configure map styles” popup, click on “Fill color” under the “Polygons” header at left. To the right, click “Buckets” and then select the “Divide into 2 buckets” radio button. First, click on the “2” dropdown and change your number of buckets to “4”. Open the “Column” dropdown below this and you can quickly scroll through all of the data columns available in ACS table. To start exploring, select one and click the “Save” button at the bottom. After a moment, a yellow “Map style saved” label will appear above the map, and it will be recolored according to your selection. To see how other columns of data map, simply click “Configure styles” again, select a different column of data from the dropdown, and click “Save”.

A few notes:

  • Keep in mind that the default “bucket” ranges (0-25, 25-50, 50-75 and 75-100) may not be ideal for the particular data column you’ve chosen. You may need to adjust these values in order for the map to be meaningful (or even show any color variation at all).
  • Also note that the default “bucket” colors should really be shades of a single color, rather than 4 distinct colors. Any time you are mapping the intensity of a single value, it should be indicated by intensity of a single color, not multiple colors.
  • Why doesn’t your chosen data point show up in the little balloon? You need to adjust its contents by clicking the “Configure info window” link above the map. There you can select the with check boxes the exact column information you want to appear. Select a few on the left, click “Save” and then click on the map again.

Having played around with the data for a little while, you’ve found some interesting data points. I’m always interested in rent burden and housing affordability, so I chose the very last column of data “Percent of Renter-occupied Units Spending 30 Percent or More of Household Income on Rent and Utilities”. After using the table view and (by clicking on the column header) sorting the data and find its minimum and maximum values, I adjusted my ranges to be 0-30, 30-45, 45-60, and 60-100, colored in shades of red. The result is a map that shows a few interesting things – at first glance, we note that Nebraska’s 3rd Congressional district is the only one in the country where more than 70% of people live in affordable housing, and two of the most rent-burdened districts are California’s 45th and Florida’s 25th – not in districts in New York, San Francisco, Los Angeles or other notoriously expensive cities. What’s going on here? This data alone won’t tell us, but it has given us a lead towards what might be an interesting story.

Once you’ve done the rest of your research and discovered some of the “why” behind your visualization, you’ll want to make sure your readers have your data at their disposal. To add the map to your page/site you’ll need to do 2 things:

1. Share it. In the upper-right corner of the map or table view, you’ll see a “Share” button that brings up a popup. In the bottom half of this window, three radio buttons list the “Visibility options”. To add your map to a webpage, you’ll need to make it at least “Unlisted”, if not “Public”.

 

 

 

 

 

 

2. Embed it. Click “Get embeddable link” above the map and a small scrolling window is revealed above the map with code that you can paste into an HTML. To it into WordPress, as above, you’ll need an iframe plugin like Easy iFrame Loader installed. Using the revealed code, follow the directions to add the map to your post or page.

If you want only to email your link to a few people (and not have others be able to view it) you can leave the “Visibility options” on “Private” and share it either through the “Share” popup, or else email them the link made visible when you click the “Get KML network link”.

So that’s a first round on using Google Fusion Tables to generate explore and share data sets through interactive mapping. There are many other features available here though, so there will undoubtedly be more Google Fusion Tables fun to come.

Listen Up: About Event Listeners

The so-called “event listener model” is an essential component of programming in both JavaScript and ActionScript, but can be a really difficult concept to get your head around. Essentially, when you set up an “event listener” you’re telling a particular object in your program how to react when a particular event takes place.

Probably the most common (and sensical) type of “event” that an interactive graphic or page will capture is a user’s mouse click. Let’s say you’re working in Flash, and you’ve got a button on the stage that you’ve named (on the stage, not in the Library) Button1.

Obviously, when a user clicks on this button, we want something to happen. But what? And how do we tell the button what to do?

First, select the frame that your button is on by clicking on the empty canvas, then click on the Actions panel. The code you’ll put into the Actions panel will look like this:

Button1.addEventListener(MouseEvent.CLICK, jumpToPanel2);

And here’s a bit of translation:

Great, except what’s that bit at the end about “jumpToPanel2″? Where is that? What does it mean?

The answer is: “jumpToPanel2″ doesn’t exist yet, because you haven’t written it. What “jumpToPanel2″ will be is the name of a code container that holds the set of instructions for what should happen when somebody clicks Button1. A named code container like this is called a function.

This sort of “refer to the instructions before you’ve written the instructions” is part of what makes event listeners so confusing. It feels like you’re telling the program to do something before you’ve told it what to do. Don’t worry. Flash doesn’t try to “read” any of those instructions until you publish (Ctrl+Enter or Apple+Enter) your movie.  So you’ve got plenty of time to create that “jumpToPanel2″ code container before Flash starts yelling at you. Usually the hard part is not letting yourself get caught up in this seemingly paradoxical logic. Forge on before you think too much about it.

So quick, let’s make that code container and put some instructions in it:

function jumpToPanel2(e){
gotoAndStop("Panel2");
}

And here’s another bit of translation:

Now the code makes a little more sense, and reads something like this:

“Hey symbol on the stage named “Button1″ – when you get clicked, follow the instructions in the code container named “jumpToPanel2″, which says to go to the frame named “Panel2″ and stop there.”

Of course, it will be hard to tell if you’ve made it to the frame called “Panel2″ if there’s nothing on there, so you’ll want to put something visual there first to make sure you can tell if your button click is working. You’ll also want to put a stop(); action on each of your content panels so that Flash doesn’t just run through them all when you publish your movie. If you’ve done all of that though, you should now have a button that jumps you from frame “Panel1″ to frame “Panel2″, and you’ve written your own event listener!

 

Google Gadget: “Geomap”

After Gmail,  Google is probably known best for mapping, and their “geomap” gadget is a great for generating quick world and region maps that either be embedded directly in a webpage to function interactively, or used statically via screenshot.

These maps work best at smaller sizes; even in the small image at left you can see that Florida looks a bit like a pointy pegleg hanging off the east coast, and the shapes  get only more geometric and slightly bizarre as the size increases.

All that being said, however, the geomap is a nifty little tool for quickly generating shaded maps of the world or world regions. It can map data directly from a Google Spreadsheet with a simple click – simply put the country name in the first column and its corresponding numerical data value in the column next to it, for as many countries as you have (note that you can import .txt or .csv files directly into Google Spreadsheets, which are common formats in which governments and NGOs make data available). Then simply copy the url of the spreadsheet from your browser and paste it into the “Data source url” field here, select the “Show Legend” check box,  and click  the “Preview Changes” button at the bottom of the page. Viola! Your data is mapped.

The tool does a fairly good job of generating an appropriate color ramp (the term for the range of colors in the legend or key), though the top and bottom numerical values can be a bit messy. Still, it provides options to change the size of the map and to focus on specific regions or countries (via the “Map” dropdown), which provides decent flexibility. Just remember to see each change you have to click the “Preview Changes” button again.

Whether you use it statically or interactively, you must credit your source for the map. Map outlines constitute information produced by someone other than yourself – and borders may differ from one source to the next – so the source you use must be cited. That means don’t crop out the little blue line of text at the bottom right that says “Gadgets powered by Google”. Either keep the original or add it back if you use an image of it. If you embed it directly in a page via the “Get the Code” button, this credit should be included automatically.

This gadget is best suited to give an overall impression of how countries or regions vary on some high-level metric, like GDP or population. Data with extreme outliers or where more granular geographic information is important are not good candidates for this; these will be better suited to some of the visualizations that can be created through Google Fusion Tables. But for simple data sets with reasonably limited variation, a quick geomap may be just the thing.

Fractured Feverlines

NYT's Budget Projection Graphic

Their page link suggests that the term for the New York Times’  budget projection graphic at left is a “porcupine” – though it’s not clear whether this could apply to visualizations other than feverlines.

Either way, however, this frizzy format is inspired, providing an insightful balance between mapping the actual data of annual budgets while retaining the impression of where analysts thought the trends were going in any given year.

There are certainly other ways that this data could have been presented; it could have been rendered as a bar chart with the range of projections for that particular year overlaid, for example. But the visualization as it stands elegantly solves several problems that the latter approach cannot: it retains the dimension of what year a projection was made as well as the year it was made for –  without relying on labels, rollovers, or – worse – some kind of maxed-out color code.

Because the saturation change and departure point clearly distinguish not only actual values from projections, but also maintain the continuity of the year in which those projections were made, the viewer can truly inspect the data and assess the projection trends over time. At this point of analysis the use of feverlines does us another favor, because their angles are informational and can be meaningfully compared.

This graphic’s innovative form is a credit to its author, Amanda Cox, and has deservedly garnered her the attention of even the inimitable Edward Tufte, who highlights another of her pieces on his own website, here.

Code Vocabulary

Below is a compilation of some fundamental terms you’ll need to get a handle on in order to work with almost any programming language – along with lay explanations of what the heck they mean.

The below list is geared towards ActionScript and JavaScript, which are the most common languages used to create interactivity on the web (and which, happily, both comply with ECMA standards – so get good with one and you’ll be comfortable with the other). This glossary is meant to be foundational, however, so many of them will be applicable to almost any programming language on the web (and elsewhere).

object: basically a synonym for “thing”. When you see “object” it can stand in for almost any of the terms listed below.

variable: a named container for information. The name can be any word or combination of words, as long as they’re not already used (“reserved”) by the programming language. In Flash, if your variable name turns blue when you type it, Flash is already using it – pick something else. And don’t try to include weird characters like &?@#*!. You are not a cursing cartoon character. Yet.

datatype: the category of stuff a particular variable “container” has or can have in it. The datatype or class (see below) of a variable gives you a sense of its general characteristics, and rules for what can be done to it or with it.

functions & arguments (to functions): functions are kind of like recipes. The arguments to the function are the specific ingredients to be used; the “operations” or steps within the function are the steps of the recipe.  Often at the end of a function, it returns a finished product to wherever in the program it was called from.

classes & instances (of classes): a class is like a do-it-yourself datatype, defining a new category of “object” (thing) that can be used by your program. An instance is a single copy of that object. The advantage of classes is they let you define a set of characteristics and behaviors that you want every copy of that class to share. Then you can make many instances (copies) of it, and know that they all have the same basic characteristics (acting like a button, for example).

In Flash, there is a visual corollary to this: anything that you make into a symbol (which then appears in the Library panel) is effectively a class; each copy you drag on to the stage is an instance. That’s why editing the Library symbol affects all its copies – when you change the visual characteristics of the class, it affects every instance.

A real-world analogy for classes and instances would be the idea of car models vs. actual cars. “Subaru Forrester” is a particular category of car (class), but there are thousands of individual vehicles (instances) on the road. Because they’re all Forresters, we expect them to be reasonably similar, though we also know that there are parts of them that can be customized (more on this later). Likewise, the way that individual Forresters can be identified by the name of their owner, each instance of a class should be given a unique name if you want to be able to do anything with it.

Get on the grid

Grid design, subtraction.comWhether you realize it consciously or not, every major website you visit (and every WordPress or other blog theme)  is laid out according to a grid pattern similar to the one at left (which comes from subtraction.com, the blog of Khoi Vinh, former design director of NYTimes.com).

Why does everyone use grids? A few reasons:

  1. It makes decisions easier. Without a grid, you’d have to make all kinds of design decisions on the fly – like how big to make a photo, or where to place a text well. Following the grid reduces the number of options so you don’t have to agonize over every single choice.
  2. It looks better. Almost any layout – unless created by a skilled artist, designer or illustrator – will look amateurish if it’s not based on some kind of grid. The grid provides an underlying structure and organization that translates to a polished look.
  3. It offers a way to make certain elements pop. Interest and surprise can be created by very occasionally and very selectively “breaking” the grid. However, this means that the element – usually an image or illustration – that does the breaking should not itself be a rectangle of some kind. Photo cutouts of people or objects are the best candidates for this.

A great walkthrough of grid design is the presentation made by Vinh and Mark Boulton (both of whom have now written books on the subject) at SXSW Interactive in 2007, called Grids Are Good.

There are also a range of grid generators and templates that can be found around the web:

  • gridsystemgenerator will create and display a custom grid according to values you enter, and (I have not tried these) will even generate s complete html/css template set for you.
  • 960.gs, by Nathan Smith, offers templates in a variety of formats (including Flash, which I’ve saved down to a CS3 format here). His system is the basis for the grids created with the generator above.

Inaccurate Inmates

Good Magazine inmate abuse graphic Good Magazine publishes a wide range of infographics produced by different folks (I think they get submissions and/or hire freelancers), many of which are lovely.

Some of them, however, are problematic, such as the example at left (click on the photo to view full size).

Though the subject matter is interesting, there is a fundamental scaling problem: each figure represents an arbitrary total percentage, making any visual comparison of the color segments inherently misleading.

This makes the graphic essentially inaccurate, despite the percentage labels – visual elements especially must be just as rigorously treated as “hard” numbers, because they can often register more powerfully (further explanation of this TK). Note that this graphic also lacks any kind of color key – another big nono when using color to ostensibly represent numerical values.

Kuler palette generator

Website for generating compatible palettesAn appealing color palette is an essential part of any design, whether for an entire website or an infographic. Kuler crowd-sources keyworded palettes that you can search or browse through, and also allows you to generate a custom palette from a photograph (it’s recommended to use photographs from nature, as naturally-occurring color combinations are almost always aesthetically pleasing).