Google's architecture through the eyes of a 4-year-old

My 4-year-old daughter drew this picture of the Google architecture today (I told her how to spell the words):
Here's the backstory that lead to this picture...

I started working at Google when my daughter was two years old. I would come home from work and I would ask her "What did you do today?" and she'd answer. Before long, she'd start asking me, "What did you do today?" Frequently, in my first year at Google (and still today) the answer was "I wrote a MapReduce." Fast-forward two years, and now both she and my son (age 2) often race to the door when I arrive home to see who can be the first to ask "Daddy, Daddy, did you write a MapReduce?"

She is also fascinated by "how things work" and will regularly request at bedtime an explanation of how something works. "How does the light work?", "How does your stomach work?" etc. Not too surprising for the child of two engineers. As a brief diversion, I had another "I live in the future" moment the other night: My daughter asked at bedtime "How does your eye work?" I explained in as much detail as I could and she was intrigued, and asked "Can we see it?", so I put my eye close to hers and explained what she was seeing. She replied, "No, I mean the inside!". An idea dawned on me. I pulled out my G1, spoke the words "diagram of an eye" (Voice Search is awesome), and a few seconds later a beautiful diagram of an eye was on the screen. My daughter was satisfied, but didn't even blink at what had just happened, while I felt like I was in a remarkably banal Star Trek episode.

In any case, yesterday I arrived home, greeted by the chorus of "Did you write a MapReduce?" to which I replied "Yes, I wrote a MapReduce." She asked what it did, I explained that it organized some information about books in a Bigtable, which is a useful place to store large amounts of information. She declared, "At bedtime tonight, I want you to explain how a Bigtable works." At bedtime, she did in fact ask, and I did my best to explain it at a 4-year-old level and she seemed to absorb it alright. After she went to bed, in a combination of amusement and fatherly pride, I twittered about it.

This morning, I told her that some of my friends were very impressed that she wanted to know about Bigtable. So, she decided she would draw a picture. The picture shows, on the Bigtable side a table (notice the legs) with a bunch of boxes (bigtable cells) organized in rows and columns. On the MapReduce side, the box between "Map" and "Reduce" is the mapper, while the big swirly cloud connected to it is the shuffle phase, and then the nice neat lines connected to that is the reducer. I'd love to say the big scribble at the top is the "cloud" from cloud computing, but she informs me that's the ceiling.

(Actual details about various parts of Google's architecture are available at Google Research's site, including papers on Bigtable and MapReduce.)

Mozy review: It doesn't work

I tried out the free version of Mozy in the fall of 2007, having heard good things about it. It quickly backed up over a gig of data and I decided to sign up. From here it went bad. I knew the full upload was going to take a while, but based on how it worked during the free period, it looked like it should take 4-8 weeks to back up my 100G of data. It slowed down dramatically, in two senses:

• The raw upload rate got a lot slower
• It would regularly get "stuck" and have to be manually killed and restarted, sort of defeating the purpose of the whole background thing.

Further, it would regularly (many hours a day) use 100% of CPU, rendering the machine unusable. Changing the set of things to be backed up would always kick it into this state, and would take hours and hours to finish, meaning there was no easy way I could tweak the backup set to just backup the most important stuff first.

So, I contacted support. They were very friendly and encouraged me to upgrade to the latest and greatest client, which I did. They extended my membership by one month for free. But, none of these upgrades worked. I asked that they cancel my account and they instead kept suggesting I try other things. I tried several of them and nothing worked. At this point, several gigs of data had been backed up, but my machine was still almost unusable while a backup was running. So,
I decided to try a test restore. I went to their web UI, which took tens of minutes to load, and in the end gave me a "No backups found" error or a generic "An error has occurred" message. I again contacted support, and they replied "well, it works for me", which was singularly unuseful. I asked again to cancel, and again got the "well, upgrade to the new version". I gave up. Nine months later, the 100% CPU problems were still occurring, though with less frequency,
and it finally "finished" the backup. I tried another test restore, and it still didn't work. I gave up and turned off Mozy.

(A couple other things worthy of note: This was all on a MacBook Pro, and I lived in two different places during this time, with very vanilla network configuration, using two different ISPs, so I'm doubtful the problems were particular to my situation)

Then, this past fall, I got auto-renewed. I sent an email making it clear I wanted my money refunded and my service canceled in no uncertain terms. I canceled and they refunded all but $5 of my renewal fee. So, in the end:

• The software was slow to the point of being unusable.
• The software had a detrimental effect on my usual use of my machine
• The restore functionality never worked
• Despite multiple requests, they refused to cancel my account in the first place
• On cancellation, they didn't even refund the full charge

Recent sci-fi reading

Over on Chris Shabsin's blog he posted his sci-fi reading list and I posted some comments (I've read just under half of his list, and I should harvest the remainder of). I thought I'd repeat some of those comments here and expand on them a bit.

Last year, I raved about Spin and thought Axis was pretty good, and this year I finally got around to reading some more Robert Charles Wilson. So far this year I've read Darwinia, Blind Lake, and The Harvest. None were as good as Spin, but all were good, and I especially liked Darwinia.

Other sci-fi authors I'd recommend:

• Kurt Vonnegut (Cat's Cradle and Slaughterhouse Five, in particular, but the latter is barely sci-fi)
  
• Stanislaw Lem (Cyberiad and The Futurological Congress were favorites when I was a teenager)
  
• Timothy Zahn (almost everything)
  
• Larry Niven (I remain impressed with how well most of his work has aged)
  

You can read my full comment on Chris' blog.

I'd also recommend goodreads.com or librarything.com as good sites for social-networky tracking of reading lists, reviews, recommendations, and read books. I'm currently using goodreads for this. My "read" list is far from complete, and as I think of it, I backfill with authors or books, but it's more about current reading. I'd welcome comments here (or on goodreads) on my current to-read list or new suggestions for additions.

2008 Games Summary

Well, after several years of gradual decline in game playing, the trend has finally reversed and I again played over 1 game/day average. Additionally, it was a good year for gaming: we moved and I now have a dedicated game room (pictured), my kids continue to want to play lots of games, and my daughter is now able to play some "real" games, and my colleagues have been totally bitten by the Race for the Galaxy bug. Overall, it's been a good gaming year.

Games Room
We said when we moved we'd make sure the new house had a room that could be the games room. Well, we moved, and we did (see photo) and I played a lot of games there. In the end I played 112 games at home, about 96 of them in the new house. Even better, the games room is front and center in the house, and I look forward to hosting more gaing sessions in 2009.

Gaming with Kids
My kids continue to turn in to excellent gamers. My 4-year-old is now playing games with meaningful choices and my 2-year-old's favorite way to delay bedtime is to ask "One more game?". Further, my 4-year-old was the person I played the most games with in 2008. In 2008, I played 63 games with her.

Gaming at Work
At the end of 2007 I started organizing gaming at lunch at work on Fridays. By mid-2008 we were regularly having two tables of gaming for the hour, and a waiting list for Race. We got a second copy of Race and expanded to Monday lunchtime gaming as well. In the end I played 59 games at lunch and another 29 at other work related events (retreats, after work, etc.) The majority of the work gaming has been Race, but with a solid sprinkling of Dominion, 6 nimmt!, For Sale, Pandemic, and the like.

Regular Gaming Group
I made 17 out of 24(?) sessions this year, and this remains my main source of variety in games and remains a great group to game with.

Overall Stats

• 370 Games Played
• 27% at home (family & guests)
  
• 24% at work
• 19% regular gaming group
• 18% "events" (eg, Unity Games)
  
• 12% other friends' homes
  
• 168 different Titles
• 140 days with at least one game played
  
• 66 new-to-me games played
• Gamed with 159 different people
• 26 new games or expansions acquired
  

Top Games

• 80 games of Race for the Galaxy
  
• 22 games of Dominion
• 11 games of Electronic Catchphrase
• 8 games of Monkey Madness
• 8 games of Pandemic
• 6 games of 6 nimmt!
• 6 games of Arbos
• 5 games of Walk the Dogs
• 5 games of Go Away Monster!
• 5 games of Zack & Pack

Lanna Thai Diner Review

When we moved up to Reading, we made an effort to look around for appealing local restaurants. I can't say we've been that successful, but we did find one unambiguous hit: Lanna Thai Diner in Woburn.

This restaurant is an old 50's style diner; all chrome and vinyl, outside and in, but turned into a Thai restaurant. The owner, Max, is friendly and fun. The resulting ambiance is outstanding. Those things alone would make it worth visiting once, but what has made it our regular "date night", when we have grandparents around to do babysitting is the food.

Everything we've had there is very good, from a variety of curries to the Pad Thai, but the Karie Curry special is the one that we always make sure at least one of us gets. It's spicy, but not overwhelmingly so. It has a sweet, almost cinnamon-like flavor to it that is just amazing. The portions are generous but not overwhelming and everything has been delicious, including the appetizers (though the satay was sort of unremarkable). Additionally, while they always seem to have customers, and they clearly do a decent take-out business, we've never had trouble getting a table, despite being a quite small restaurant. Service is really quick, but I've never felt rushed.

All in all, I highly recommended Lanna Thai Diner (website, map, streetview)

Atom feed of your recently played games

Scott and his team are a bit overworked so I wrote a simple scraper that turns your "games played" page on BGG into an Atom feed. If/when Scott gets around to officially supporting this, I'll make my feed generator redirect to the "real" one. (Atom is essentially the same as RSS and most services that support RSS support Atom)

Games Played Atom Feed Generator

For example, my games played.

I'm planning to also add feeds for plays of a particular game, or even a particular game by a particular person.

In praise of short games

I like short games. I often find myself defending short games, and the preference for short games to other gamers, so I thought I'd write my thoughts down in a little detail.

First, definitions: A very short game is one that plays in 15 minutes or less. A short game is one that plays in 15-45 minutes. A medium game plays in 45-100 minutes. A long game plays in 100-180 minutes. A very long game plays in over 3 hours. That's how I use those words.

Part of my preference for short games comes from my opportunities to play. I don't get to game as much as I used to, and some of that limited time includes lunchtime gaming at work. But, my inclination toward short games extends back to before I had any such time constraints. The constraints have simply amplified the preference.

The standard argument is that longer games have more strategic opportunities, the ability to more thoroughly explore a game system, and overall to be "deeper". For the most part, I agree. The argument goes further something like, "If you're enjoying it, what's the difference between one two hour game and 4 half-hour games?". To me, a lot. This is the key, and why I prefer short games.

To get to the root of the issue requires deconstructing how I get enjoyment out of games. I enjoy the social aspect, it's part of why I don't do "online boardgaming" much if at all. I enjoy the decision making, but that's actually a small part. I enjoy the process: Going from a start condition where everyone is equal; adapting to random factors and other people's play; developing a strategy or strategies; picking tactics to execute that strategy; executing those tactics; resolving the endgame. The whole is far greater than the sum of the parts. The positive experience of a complete game, the beginning, middle and end, is greater than the micro-activity of "what do I do next". Each of those parts of the process has value in and of itself, not just "enjoyment per unit time". It's similar to why people presumably enjoy whole movies or TV shows rather than a simple collection of entertaining scenes. So, if the enjoyment of playing a good game yields 1 EP ("enjoyment point") per minute, I'd say the process of developing a strategy is worth 5-20 EP, on top of the time it takes. For each of the 5 or so "aspects" of gameplay, that often ends up being the dominant part of the fun. Plus, as mentioned, I like novelty/variety so I'd say most of the time I'd give a game a 1-10 EP bonus depending how long it's been since I'd played it.

Now, most very short games, and many short games have the problem that they don't have all that process in one game. They tend to be just tactical games (no strategy), or dominated by random factors, or insufficient variety to make the adaptation part meaningful, or trivial enough to make the execution automatic. But, say a good 30 minute game has obvious tactics given a strategy and a mundane endgame. This leaves it with 30 minutes of gametime, the adaptation aspect, the strategy development, and the execution. If each of these parts yields only 5 EP because, say they're more shallow than they might be in a longer game, then the game yields 45 EP, or 1.5EP/minute. Then, say I have a two hour game that has all 5 of the above aspects, but at a higher level of depth, so they're worth 10 EP each. This means that game is 170 EP, or 1.4EP/minute. So, I'd have gotten more EP playing 4 half hour games, even if I play the same one over 4 times. If I play different ones to get the "variety bonus", the gap widens. The specifics of this deconstruction aren't actually that important. It may be there's 10 aspects, not 5, etc. etc. The core is that there's something to "playing a complete game" which is more than just an activity.

Fortunately, there are some short and medium games that not only have all 5 (or whatever) of the above "aspects", but provide them with comparable quality to longer games. Race for the Galaxy is certainly one such game for me. It looks like Dominion may be too. But, on the flip side, longer games often suffer from a reverse problem; I used 1EP/minute as a baseline up there, but if the core activity of the game is more or less fun, that could be different. In a lot of longer games, I actually find the core activity is substantially less enjoyable. In some cases, this is why they're longer; they have 60 minutes of interesting stuff to do and 40 minutes of mundane manipulation or bookkeepping. This lowers their baseline rate and makes them less fun, for me. But, most shorter games don't quite hit enough of the aspects, while a 45-60 minute game usually does. So, in the end your 60 minute game which hits all 5 aspects at 5-10 points each, is just as good, or better, than the above-described hypothetical 30 minute game. This means I end up playing a lot of "medium" length games.

Finally, there's the issue of winning and losing. For the most part, my winning or losing doesn't have a huge effect on my enjoyment, but usually it is more fun to win than to lose, even if it's just a marginal effect. Other people's enjoyment also matters. In any game, one turn of bad luck, one error, or the confluence of small errors or luck can turn a game against someone. In a short game, it's over soon and the feeling of being stuck in a bad place resets. In a long game, you can get stuck there for a while. So, both because I would rather not be stuck with my errors or bad luck and because I don't want others' to feel stuck with errors or bad luck, it further pushes me away from long games.

But, there's an exception. There's a small number of very long games I like a lot. For example, Descent and Battlestations. These jump out of the above logic by being a lot of fun in the steady state, as an activity, unlike many long games, as well as usually having all the "aspects" in spades.

So, that's why I have a preference for short games, and obviously other poeple's structure/model for enjoying games may be radically different, but it's why I'll keep seeking out the Races of the world; short but complete and well-rounded games.

1000 different games

I love variety in games; I don't like playing the same game a lot. It's not unheard of for someone to suggest a game and have my reaction be "Oh, I just played that a few weeks ago.", or for games that are less favored, "Enh, I've already played that this year." The most favorite and filler-y of the fillers rarely receive more than 10 or 20 plays in a year, and for a while, I was playing a lot of games each year (~800). Recently, Race for the Galaxy, has been the exception, but the general rule still holds.

So, it was with pleasure and a certain degree of pride that I recently passed the 1000 different games played mark. Of course, I haven't been keeping track of games played my whole life, but I probably played a total of 15-50 different games before I have any chance of remembering details. Further, before 2000, I didn't keep very precise records. For example, I know I've played Wiz-War, but not since 2000, and I don't remember exactly when. So, the 1000 threshold is a little fuzzy, but I can now say I've played:

• > 1030 different games in my life
• 1018 different games that I have some record of
• 1001 different games since 2000, for which I have specific logs

Of course, it also depends how you count "different" games. I count, somewhat arbitrarily, by the distinct packaged games, where plays of a game with an expansion counts as the base game. So, Ticket To Ride and Ticket To Ride: Europe are two different games, but Ticket To Ride and TTR with the 1910 deck are the same game. Further, all the games in New Games in Old Rome, count as one game. Further, if BGG considers two games to be the same game, such as Europa Tour and 10 Days in Europe (which are not-quite-identical, but close enough) I count them as the same, even though they are technically distinct packaged games.

Some stats on the 1001 games:

• I own 52% of them.
• The average (mean) number of plays is 4.
• The median number of plays is 2.
• 47% have been played once.
• 22% have been played 5 or more times.
• 9% have been played 10 or more times.
• 1% have been played 38 or more times.
• I play at least one new game in 42% of gaming sessions.
  
• The average number of new games per session is 0.9
• Unsuprisingly, the rate at which I play new-to-me games has been almost monotonically decreasing over time, looking at a window of 100 new games. There was a brief upward blip the year when I first attended Gathering, when I played 54 new games in under a month, leading to 100 new games in 6 months. Lately, it takes me over a year to play 100 new games.
  
• Since 2000, there have only ever been 4 calendar months where I haven't played a new game. (August 2004, May & September 2005, Jun 2007)
• I've never played more than 10 new-to-me games on a single day, but I have played 10 on four separate occasions.
• I've played multiple games which begin with each letter of the alphabet, except X. I should clearly correct this, since I own at least 3.
• I still want to play more new games.

Simple election Monte Carlo toy

I wanted to be able to fiddle with my own model of the electoral college. At the moment, despite the general media's efforts, the race isn't that close, but I wanted a modeling framework in which I could interpret "dramatic" changes that might occur.

So, I made a Personalized Election Modeler which lets you specify the chances you think Obama will win each state, and runs the simulation for you.

Request for online photo hosting/sharing suggestions and opinions

I'm looking for a good online photo sharing/backup solution. The three options I'm considering at the moment are SmugMug, Flickr and Picasaweb. None seem to be quite up to my requirements, so I'm hoping someone can suggest something better or how to use one of these services better. I'm leaning toward SmugMug, but I'm not entirely happy with it.

I want:

• Reasonably priced for a large amount of storage. I have over 40G of photos and don't want to pay >$100 a year. This seems to eliminate Picasaweb.
• Access control. I actually like SmugMug's access model, but would prefer finer grained (per photo control) rather than having to move photos around between albums to change access. The three tiers of public/unlisted/password-protected work well for me.
• Automatic/easy upload from iPhoto, iPhone and Android. Most seem to do fine in this regard, but opinions are welcome.
  
• Decent file management. I want to be able to search browse and share all my photos at once, not have to go fiddle in dozens or hundreds of albums. I don't want to have to keep track of what I've uploaded and what I haven't. I don't want it to end up with bunch of duplicate uploads. All of them, including SmugMug seem to have issues here.
• Reliable. I'm want this in substantial part as a backup solution.
• Good temporal organization/photo finding. I think of my photos as organized by time, not by "album" and none of the sites seem to do well with this model.
  

In all, the main reason I'm not so sure about SmugMug is that it doesn't deal well with things like duplicates, it seems very locked in to the whole "album" metaphor (as do the others), and their site is a bit slow.

Thoughts, experiences, suggestions, ideas?

Update: I think I'm going to go with flickr. Easy browsing by time, good price, and it's fast.