And while I’m posting about engineering culture, I have to post the DevOps equivalent of Douglas Engelbart’s “Mother of All Demos” — “10+ Deploys a Day: Dev and Ops Cooperation” by my former Flickr colleagues Allspaw and Hammond.

Adam Mosseri’s “Data Informed, Not Data Driven” is a great talk about product and engineering collaboration.  But what I love most is casualness about what working at scale looks like, throw away phrases like, “Yeah, I’m pretty sure that’s rolled out to everyone.” It’s not a magical unfathomable plane, you project through what works about what you’re currently doing to get there.  Everything else is failure.

Greg Wilson’s “What We Actually Know About Software Development, and Why We Believe It’s True” is one of the best talks I’ve ever seen for equipping people to think critically about software development. Critical building block in figuring out how to scale your highly productive engineering culture. (assuming you have one)

“Get the $9 volt regulator”, and other deep insights from James Hamilton on the real business of running a data center — the messy bits that you ignore at your peril.  

It would be hard to overstate the impact of Brad’s early architecture work at Live Journal, both in popularizing the architectural patterns that form the basis of the social web (Flickr, Facebook, Etsy, etc) and in releasing the software that powers much of the Web (memcache, gearman, etc)

This <a href=”http://danga.com/words/2004_oscon/oscon2004.pdf”>2004 deck, “Inside LiveJournal’s Backend</a> is the earliest I can find, but there were drafts of this floating around earlier, and we all read them.

Video is a version updated for 2007.

I remember reading Jon Bentley’s “Ternary Search Trees” article in Dr. Dobbs in 1998. I also remember crashing his lecture version at Amherst College, I assume that same year. I don’t think I’ve ever met anyone before or since who enjoyed data structures that much, and it’s a pure joy to watch him in action.

“Three Beautiful Quicksorts” is another classic in that same vein. If you haven’t seen it, well you probably don’t understand quicksort.

“computer programmers are optimists.”

“I think Einstein said it, because I heard him say it.”

This is a tumblelog I’ve been meaning to start for years.  As such I’ve forgotten nearly every video I ever threatened to publish here.  So please make suggestions, new interesting talks or old classics are both appreciated.

Update of Jeff Dean’s famous 2009 talk, “Designs, Lessons and Advice from Building Large Distributed Systems”, the source of the much quoted “numbers every engineer should know”:

• L1 cache reference: 0.5 ns
• Branch mispredict: 5 ns
• L2 cache reference: 7 ns
• Mutex lock/unlock: 100 ns
• Main memory reference: 100 ns
• Compress 1K bytes with Zippy: 10,000 ns
• Send 2K bytes over 1 Gbps network: 20,000 ns
• Read 1 MB sequentially from memory: 250,000 ns
• Round trip within same datacenter: 500,000 ns
• Disk seek: 10,000,000 ns
• Read 1 MB sequentially from network: 10,000,000 ns
• Read 1 MB sequentially from disk: 30,000,000 ns
• Send packet CA->Netherlands->CA: 150,000,000 ns

“Horse graves are very deep and will sever cables.”

 (I couldn’t find a video of the original)

Availability and Consistency

Classic Werner Vogels talk from 2007 on developing a working definition of scalability, and a working understanding on CAP.   Additionally interesting thinking about the parallels with engineering organizations.

Historical context: this was 2 months before the Dynamo paper was published.

(ps. InfoQ kind of sucks, and doesn’t allow embedding)

“If your system has to reach agreement to get work done, you might as well give up now.”

Arguably the best part of Facebook is their fast, auto-completing, context sensitive search across their heterogeneous documents. And this video is on how they build it.

Also interesting mental model balancing “Memory Consumption <=> Cache Bandwidth <=> Compute”.

Also bloom filters!

(via Videos Posted by Facebook Engineering: Typeahead Search Tech Talk (6/15/2010) [HD])