Meta tags:
description=Sam Lambert joined GitHub in 2013 as the company’s first database administrator, and is now the company’s director of technology. In this interview, he discusses how the service — which now boasts…;
author=Derrick Harris;

Headings (most frequently used words):

the, in, to, and, on, ruby, medium, you, computing, your, day, cloud, more, from, job, thinking, what, next, told, by, people, behind, 642, of, is, with, up, that, here, building, derrick, harris, can, free, unbundle, learn, share, claps, founder, editor, writer, architecht, at, pivotal, might, know, me, gigaom, way, back, now, software, any, topic, github, scaling, nomadic, tech, team, keeping, all, git, no, global, engineering, network, hubot, rescue, bandwidth, ai, native, silicon, valley, developer, adventures, elixir, background, processing, using, ractor, static, typing, speed, rails, docker, mac, future, stem, macros, explained, pattern, matching, first, impressions, make, yours, responses, an, open, platform, where, 170, million, readers, come, find, insightful, dynamic, expert, undiscovered, voices, alike, dive, into, heart, bring, new, ideas, surface, follow, writers, publications, topics, matter, ll, see, them, homepage, inbox, explore, if, have, story, tell, knowledge, or, perspective, offer, welcome, home, it, easy, post, write, abouthelplegal, about, help, legal, get, app,

Text of the page (most frequently used words):
the (98), and (95), that (74), you (46), can (30), our (30), just (28), they (28), what (27), for (26), have (24), more (21), github (21), from (18), with (18), #people (17), are (17), really (16), there (14), work (14), but (13), infrastructure (13), about (12), your (12), ruby (12), don (12), all (12), company (12), scaling (12), keep (12), team (11), hubot (11), how (11), like (11), something (10), need (10), git (10), follow (9), see (9), new (9), working (9), lot (9), stack (9), way (8), use (8), this (8), world (8), around (8), technology (8), get (7), medium (7), them (7), worked (7), often (7), things (7), make (6), using (6), back (6), think (6), chat (6), problems (6), got (6), when (6), try (6), own (6), long (6), was (6), not (6), also (6), tech (6), some (6), site (6), any (5), write (5), rails (5), derrick (5), harris (5), cloud (5), now (5), engineering (5), companies (5), basically (5), together (5), then (5), other (5), time (5), well (5), different (5), will (5), works (5), actually (5), quite (5), interesting (5), terms (5), app (4), come (4), into (4), computing (4), day (4), know (4), trying (4), through (4), amazing (4), going (4), example (4), already (4), large (4), their (4), than (4), distributed (4), culture (4), city (4), employees (4), application (4), fast (4), provision (4), physical (4), look (4), host (4), projects (4), data (4), scale (4), user (4), very (4), core (4), share (3), offer (3), home (3), thinking (3), where (3), million (3), find (3), here (3), first (3), editor (3), job (3), architecht (3), next (3), told (3), behind (3), software (3), might (3), written (3), 642 (3), could (3), being (3), colleagues (3), these (3), context (3), database (3), start (3), happen (3), able (3), stuff (3), everyone (3), joining (3), were (3), doing (3), joined (3), everything (3), probably (3), provisioning (3), shipped (3), essentially (3), project (3), wherever (3), unique (3), those (3), percent (3), growth (3), certainly (3), always (3), usage (3), capacity (3), amount (3), build (3), com (3), until (3), web (3), which (3), users (3), obviously (3), interview (3), pragmatic (3), take (3), simple (3), nomadic (3), story (2), tell (2), free (2), topic (2), dive (2), bring (2), learn (2), pattern (2), mathworks (2), building (2), startup (2), unbundle (2), founder (2), writer (2), pivotal (2), gigaom (2), claps (2), old (2), used (2), much (2), many (2), fully (2), massive (2), cases (2), watch (2), via (2), out (2), page (2), gigantic (2), alert (2), been (2), teams (2), would (2), terminal (2), looking (2), graphs (2), each (2), boarding (2), chatroom (2), watching (2), because (2), physically (2), located (2), anywhere (2), comes (2), idled (2), watched (2), did (2), learned (2), automation (2), source (2), ever (2), code (2), run (2), mysql (2), drop (2), tables (2), specific (2), one (2), offices (2), sounds (2), key (2), requirement (2), order (2), backend (2), monolithic (2), had (2), patterns (2), face (2), requests (2), him (2), office (2), permanent (2), location (2), fly (2), nomads (2), year (2), pretty (2), continuing (2), embrace (2), nature (2), moment (2), every (2), say (2), night (2), issues (2), corner (2), slowly (2), gradually (2), appear (2), quickly (2), develop (2), understand (2), over (2), fairly (2), deploy (2), continue (2), controlled (2), hardware (2), machines (2), class (2), databases (2), chatbot (2), someone (2), phenomenal (2), small (2), services (2), does (2), metric (2), storing (2), let (2), want (2), sure (2), continually (2), quick (2), motto (2), most (2), domains (2), custom (2), bespoke (2), sense (2), proud (2), technical (2), functionality (2), sorts (2), base (2), less (2), bits (2), hack (2), why (2), maybe (2), little (2), while (2), choices (2), shell (2), sam (2), lambert (2), aug (2), 2015 (2), dec (2), legal, help, knowledge, perspective, welcome, easy, post, writers, publications, topics, matter, homepage, inbox, explore, yours, open, platform, 170, readers, insightful, dynamic, expert, undiscovered, voices, alike, heart, ideas, surface, brandon, weaver, matching, impressions, codeburst, ellis, andrews, macros, explained, future, stem, level, coding, anton, chuchkalov, speed, docker, mac, better, programming, frank, gerold, static, typing, andré, guimarães, sakata, background, processing, ractor, mhz, developer, adventures, elixir, bandwidth, native, silicon, valley, response, anymore, among, collaborating, whole, adds, solves, traditional, haven, solved, collaborative, experience, taking, hear, integrating, fantastic, tweet, status, updates, double, check, shared, console, failure, couple, jump, diagnosed, duplication, effort, getting, directly, incidents, touch, wood, collaboratively, reflect, previous, debug, dashboard, awkwardly, paste, outputs, issue, coming, pulling, room, yeah, engine, humorous, schematic, possibly, imagine, zero, requirements, interface, commands, failovers, backup, clone, migrations, mitigation, attacks, ask, member, staff, show, floor, mentioned, earlier, tool, function, such, workforce, rescue, again, productive, myself, colleague, refactor, transforming, environment, decisions, factoring, entire, never, conversation, issuing, pull, met, end, months, social, space, areas, meet, enjoy, necessity, baked, last, five, six, cities, laptop, decided, month, ago, cabin, woods, wisconsin, seems, branch, concerned, remote, currently, england, traveled, places, completely, possible, based, organization, bigger, engineers, line, hiring, challenge, has, good, talented, backgrounds, same, values, similar, defending, wouldn, keeps, term, concerns, mind, global, network, handling, though, tackle, times, strange, reveal, slight, domain, fixes, trending, increasing, rate, realizing, server, footprint, regular, basis, point, slack, empty, ready, provisioned, pool, puppet, roles, classifications, nodes, themselves, tag, avoiding, laborious, consuming, procurement, step, associated, gear, chassis, minutes, incredibly, flat, flexible, who, consumes, brilliant, datacenters, amazingly, dedicated, providing, underlying, local, servers, eye, grow, spikes, starting, pressure, kind, obsessed, performance, performant, drives, reinvent, wheel, writing, frameworks, usual, website, hosting, unusual, applications, apps, absolutely, known, only, elements, architecture, stores, aspect, operations, hard, extra, means, balance, public, versus, private, repositories, authentications, permissions, correct, access, anyone, else, achieve, frontier, itself, fascinating, volume, growing, seem, manage, ways, api, obscure, manner, standard, framework, extremes, case, uses, optimal, hit, rewrite, certain, challenges, busy, keeping, irony, half, tend, stick, conservative, cto, jokes, interviewed, join, dba, said, surprised, sitting, assumed, sort, hip, datastore, process, went, along, revealed, set, hackers, spend, stable, rather, chasing, after, latest, shiny, technologies, freedom, experiment, solve, approaches, extend, put, shelf, matures, somewhat, ironic, considering, experiments, hosted, tempted, change, committers, allows, view, smaller, nothing, chosen, game, moving, applying, varied, techniques, overcomplex, shop, intend, languages, store, powers, metadata, redis, bit, non, persistent, caching, memcached, seven, created, founders, started, ton, daemons, proxy, aggregation, result, strung, scripts, scripting, surprisingly, effective, still, unix, philosophy, internally, simplicity, shy, away, complexity, usually, provider, unfair, walk, philosophies, underpin, collaborate, 2013, administrator, director, discusses, service, boasts, relatively, talks, largely, officeless, workplace, its, remotely, powerful, homemade, called, min, read, sign, wayback, machine, http, archive, org, 20201203050717, https, 4db562b96dcd, timestamps, capture, fail, success, 2021, 2020, 2019, jan, nov, 2025, captures,

Text of the page (random words):
that underpin github sam lambert we take a very unix philosophy to how we develop software and services internally we like to be continually proud of the simplicity of a lot of our infrastructure we really do try and shy away from complexity and over engineering we like to make more pragmatic choices about how we work and what we work on for a long time very key bits of our infrastructure were strung together with shell scripts and simple scripting and it s surprisingly effective and still works really very well for us what does that result in in terms of your technology stack the core of what you see and use as a github user is a ruby on rails application it s seven year old app now created by founders when they started the company that s the core of the application but obviously there s a ton of git in the stack we have custom c daemons that do things like proxy git requests and data aggregation mysql is our core data store that we used for storing all data that powers the site as well as the metadata around the users we also use redis a little bit for some non persistent caching and things like memcached c shell ruby quite a simple monolithic stack we re really not an overcomplex shop we don t intend to try and drop new languages for every small project we ve got core ruby committers that work for us and that allows us to scale what we have and keep a pragmatic view on all our technology choices and try to keep our stack smaller really there s nothing much you can t do with the stack that we ve already chosen to keep at this game and keep it moving we just have to keep applying varied techniques to what we ve got that s somewhat ironic considering all the projects and experiments that hosted on github do you ever see new things and get tempted to change things up we certainly take a look at new technologies our employees have a large amount of freedom in what they do and people will try all sorts of stuff and experiment often it s just to know why you re not using technology you could look at something understand why it s interesting what the problems are that you re trying to solve maybe then take some of the approaches to extend what you re already doing or maybe put it on the shelf for a little while while it matures but there is an interesting irony in that half the new projects in the world happen on github and we tend to stick with a fairly conservative stack our cto often jokes about when i was interviewed by him to join the company as the first dba at github i actually said in my interview i m really surprised to be sitting here i assumed github is using some sort of new hip datastore then as the interview process went along it was more revealed to me that this is actually a really pragmatic set of hackers that just hack on ruby hack on c and spend their time working on more interesting things using a more stable stack rather than chasing after the latest and shiny tech keeping up with all that git what are the challenges that keep your team busy a lot of it is volume obviously our user base is growing we also have a very technical user base and they seem to manage to find ways to use the api in an obscure manner using a standard framework there s a lot of stuff that you don t get to see the extremes of until it s a large use case there s a lot of patterns that rails uses that are less optimal at a large scale we might hit issues like that and have to rewrite certain bits of functionality obviously we also have a massive amount of git scaling something like git in a backend infrastructure is quite different it s not something that anyone else is trying to achieve we re actually on the frontier when it comes to scaling git the application itself at our scale which is fascinating we have an amazing team that works on that and works really hard to build in the extra functionality we re like a git host in someone s infrastructure which means all sorts of work to balance public versus private repositories and to make sure authentications and permissions are correct when users try to access code for a rails app github com is a really really quick site and we have a motto that it s not shipped until it s fast so git is the most unique aspect of your technical operations absolutely we don t want to be unique in any other sense than what we re known for that s also something we re really proud of i often say to people let s only write bespoke elements of our architecture that make sense for a company that stores git data we don t need to reinvent the wheel we don t need to write our own databases we don t need to start writing our own frameworks because they re all in domains that are usual it s a website it s web hosting in the domains that are unusual we fully embrace the need to write custom applications or build bespoke apps for that what s the metric that drives engineering most in terms of what your team works on we re quite obsessed with performance we want to make sure the site is always performant and continually fast for a rails app github com is a really really quick site and we have a motto that it s not shipped until it s fast in terms of a specific metric that we keep our eye on it s capacity for storing git that s something that we have to continue to grow as our usage spikes more and more we re starting to see more pressure on that kind of infrastructure we have some really interesting projects being worked on at the moment that will let us keep scaling quite often with scaling problems they just come around the corner they don t just slowly gradually appear no cloud computing here what does the underlying infrastructure for github look like are you in the cloud or on local servers we host in our own datacenters we actually have an amazing provisioning story we basically can provision hardware like it was the cloud we have a really small but amazingly dedicated physical infrastructure team and they do phenomenal work in providing us these amazing services that we can use if i need a new host i can basically tell our chatbot hubot that i need x amount of host of this class on these chassis and it will just build them and deploy back in minutes we have this incredibly flat flexible but physical infrastructure as someone who consumes that infrastructure it s phenomenal and to watch it working is brilliant it sounds like you re avoiding the laborious and time consuming procurement step often associated with physical gear we have some slack capacity for hardware essentially the physical infrastructure team will provision machines that are empty and ready to be provisioned by the teams that are going to use them for example the database infrastructure team can look at how many machines are in the pool of the class they need for databases and essentially they ve written their own puppet roles and classifications about how those nodes work then they can just provision them themselves or just tag them so that it s capacity for that team are you scaling the server footprint on a regular basis or is it a pretty controlled growth pattern at this point we keep it controlled in terms of how we order and how we provision but in terms of usage of the site is trending up at an increasing rate more and more companies in the world are realizing that they re tech companies so the usage growth of github is just going up and up and up we re handling it well though quite often with scaling problems they just come around the corner they don t just slowly gradually appear they come quickly and we tackle them as they happen and we have some interesting use cases at times people bring strange things to the site and they ll reveal slight scaling problems but we have an application we can develop on quickly and people that understand the domain well we get over those problems fairly fast and deploy fixes and continue going i ve got colleagues that don t have a permanent location they just fly from city to city and work from wherever they re just nomads and they re all around the world building a global engineering network are there issues that keep you up at night or long term concerns always in the back of your mind defending our infrastructure is certainly something that we always think about i wouldn t say it keeps you up at night but it s something we certainly think about and scaling our organization the bigger we get the more engineers we need but we need to keep that growth in line with our culture i think hiring is a challenge that every tech company has continuing to get good talented employees from different backgrounds from around the world but you ve got to find people that have your same engineering values and that like to work on things similar to what you have and what you can offer i m also concerned with continuing to embrace the distributed nature of the company the company is 60 percent remote currently i m in england at the moment i ve traveled around the world working from different places that s something that is completely possible based on our culture and distributed nature that seems pretty unique are those 60 percent of employees working from home or from branch offices they re working from home you can work from anywhere last year i worked probably in five or six different cities around the world just working from my laptop wherever we decided to go a month ago i was working from a cabin in the woods in wisconsin i ve got colleagues that don t have a permanent location they just fly from city to city and work from wherever they re just nomads and they re all around the world that s something we can offer to people that s baked into our culture there s no requirement to work in any office our office is actually more of a social space we have areas for people to meet and enjoy being together but there s no necessity to be together myself and a colleague shipped a gigantic refactor of our backend essentially transforming it from a monolithic environment to a distributed one we had a lot of decisions to make a lot of re factoring and new patterns in that entire project we never had a face to face conversation we just worked together through chat and issuing pull requests and we then met each other at the end of that project about 6 months into him joining github again that s just the way we work that s the way our culture works there s no requirement to be physically located in order to be productive and do what we do for a long long time your on boarding was joining our chatroom watching what other people were doing i joined the company and i just idled in chat and just watched how people worked and what they did and i just learned that way hubot to the rescue you mentioned hubot earlier as the provisioning tool but is there more to it it sounds key to how the company is able to function with such a distributed workforce hubot can do basically everything in github you can ask hubot where a specific member of staff is and it will show you where they are in the world or what floor they re on in one of our offices for example there s probably about 40 different provisioning commands you can do a mysql stack you can do failovers you can drop tables you can backup tables you can clone you can run migrations you can do everything you can do mitigation of attacks basically everything you could ever possibly imagine to do in our infrastructure you can do via hubot there are zero requirements to interface with any code you can run it all through hubot a humorous schematic for hubot source github so it s a lot more than an automation engine yeah it s automation but it s a lot more it s the context as well for a long long time your on boarding was joining our chatroom watching what other people were doing because we re not physically located anywhere when an issue comes up you see the alert coming to chat you can start pulling up graphs that everyone in the room can see and everyone can see what you re looking at i joined the company and i just idled in chat and just watched how people worked and what they did and i just learned that way i try and reflect back on how previous teams i ve worked on would debug stuff everyone would be in their own terminal or on their own dashboard looking at graphs and then trying to awkwardly share them with each other or paste terminal outputs for example with chat you just dive in and the context is all there you ve got this basically gigantic shared console for our company for example if you get an alert about a database failure and a couple people jump in you can see that it s already been diagnosed and already been worked on there s no duplication of effort and the people that need to know start getting context directly when we go into large incidents touch wood they don t happen often we re able to really collaboratively work together if we tweet via hubot out to our status page or out to the updates page people can double check what you re going to write it s just a fully collaborative experience and it s something that more and more companies are taking up you hear of massive companies integrating hubot to do these fantastic use cases it s just amazing to watch really the old way of working i just don t think i could go back to anymore i m so used to being among all my colleagues and my team collaborating on what we re trying to do through chat it s a whole new way of working that adds so much and solves so many problems that i think a lot of traditional companies haven t solved s c a l e what s next in computing told by the people behind the follow 642 4 startup engineering github 642 claps 642 claps 4 response s written by derrick harris follow founder editor writer of architecht day job is at pivotal you might know me from gigaom way back in the day now follow s c a l e follow what s next in computing told by the people behind the software follow written by derrick harris follow founder editor writer of architecht day job is at pivotal you might know me from gigaom way back in the day now s c a l e follow what s next in computing told by the people behind the software more from medium can free bandwidth unbundle the cloud can ai and cloud native unbundle silicon valley derrick harris in architecht a ruby developer s adventures in elixir mhz in the startup background job processing using ractor ruby 3 andré guimarães sakata static typing in ruby 3 0 frank gerold in better programming speed up rails on docker on mac anton chuchkalov in level up coding building a future in stem mathworks editor in mathworks ruby macros explained ellis andrews in codeburst ruby 2 7 pattern matching first impressions brandon weaver learn more medium is an open platform where 170 million readers come to find insightful and dynamic thinking here expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface learn more make medium yours follow the writers publications and topics that matter to you and you ll see them on your homepage and in your inb...