Productivity Engineering - Ballmer Peak

Published February 14, 2021

How do developer teams stay productive? In this episode, we are talking with Kathryn Koehler and Andy Glover from the Netflix Productivity Engineering team to learn more about what a Productivity Engineering team does to help improve productivity.

Guests

Picks

Panel

Episode transcript

Edit transcript

Ryan Burgess
Welcome to a new episode of the front end happier podcasts. A lot of our listeners are aware that I recently moved into a new team to lead a team and productivity engineering at Netflix. And I thought it'd be really great to really talk about what is productivity engineering do and really needing a lot of help in that area. Because I'm still learning. I've leaned on two experts Kathryn Koehler and Andy Glover. We're both on productivity engineering at Netflix, Andy and Catherine, can you give brief introductions of who you are? What you do and what your favorite Happy Hour beverages?

Andy Glover
So my name is Andy Glover, I run productivity engineering and Netflix. I've been here for coming up on seven and a half years. What else besides favorite drink?

Ryan Burgess
Who you are what you do? I think you've covered and then what's your favorite Happy Hour beverage?

Andy Glover
My favorite Happy Hour beverage is a red wine from California.

Ryan Burgess
Is there any like specific like a Pinot Noir? Any?

Andy Glover
Yeah, so I like cabs in surace.

Kathryn Koehler
I'm Kathryn Koehler, I have been at Netflix, almost five months now I report to Andy, I know, I get to work with such a great group of people there. And I am running the not yet to be renamed develop org that handles all sorts of local development and in bootstrapping, adding security, adding real ease of use around getting up and going at Netflix so that our customers can focus on business logic. And what I am tucking into right now is called juneshine, which is a hard kombucha. And this is blood orange mint. So I had a box of this delivered to my house a couple of days ago, no signature required. Oh, signature required for those of you who are under 21. But yeah, that's that's what I'm doing today.

Ryan Burgess
Right on, I feel like anytime I've had a call delivered, the signature is required, like they have like, left my house without dropping it off. If I'm not there to sign for it. So you're you're lucky in that sense. Let's also give introductions of today's panelists. Stacy want to start

Stacy London
Stacy London. I'm a senior front end engineer at Trello Jem Young, Senior Software

Jem Young
Engineer at Netflix.

Ryan Burgess
And I'm Ryan Burgess. I'm a software engineering manager at Netflix. In each episode of the front end happier podcasts. We like to choose a keyword that if it's mentioned at all, in the episode, we will all take a drink. And what did we decide today's keyword is automation, automation, automation, which is likely going to come up maybe a couple times, and we will all have our drinks. Alright, I figured a good way to start the episode is really just hearing from each of you. Maybe it's not even in capacity of work, but probably is is like what makes each of you productive? What are things that help you be productive?

Stacy London
Well, I guess as an engineer, I could say for me, it's anything that's like come to say right away automates away things.

Ryan Burgess
years, manually,

Stacy London
things like I don't know making, you know, build processes really seamless and easy or code quality checks are things that anything that doesn't have to inquire, you know, stop with the human, another human has to intervene and do something. So that to me, it really, I think, helps get your code from being written out to production as fast as possible.

Kathryn Koehler
Moving into manager land, it's really about optimizing your time and no longer automating anything that you're working on.

Stacy London
Cheers.

Kathryn Koehler
So I've started using clockwise, which is a great tool for moving my calendar around increasing consolidated blocks of focus time. And if I have that focus time, I'm a procrastinator. So I like to be data driven, and have time to just get really deep into what I'm doing, and not pop my head up. And then I also put things in front of me like carrots, I will not do this until I get through this work. And so that keeps me productive. And it keeps me focused, because otherwise I tend to be all over the place. And you can get so randomized with time as a manager as somebody who's juggling married things like household home, kids work, etc. I really have to focus during that time. And that's what makes me productive.

Andy Glover
Yeah, similarly, I It's blocks of time and being really disciplined about, you know, an hour here two hours here. I used to feel guilty about blocking off time where I wouldn't go to a meeting or accept meeting invites. And now I realized that that is my productivity game. I think another thing is exercise from the standpoint of like some sort of release so that you can feel Guess when you do have those blocks of time I know if I have an exercise, you've done anything physical in a while, like I think I just go crazy specially and it's I think heightened and COVID. So yeah, exercise and then blocks of time where I can focus which is no different by the way than when I was a programmer, or, you know, previous to people management I was developing, it's like, I always wanted those blocks of time where I can be heads down and not be interrupted. And it's same for, you know, I guess any profession,

Ryan Burgess
I think I'm gonna echo a lot is just having that dedicated time, but also somewhat time to step away to write, there's time when you just need to think and so it might be going for a walk, I'm not great at doing that, or making sure that I actually find the time to do it. But when I have done it, I've almost felt like I've been in productive mode, because like, maybe I'm thinking through a problem ahead of time, which is kind of feels counterintuitive, it's like, no, I need that time to actually solve the problem. But sometimes, sometimes stepping away from the keyboard has been really good. I also love anything that can just automate my life in general cheers. And that can be from tools that do it for the computer, but even just things just even like I love, like smart home things like just little optimizations that I just don't have to think about anymore. I'm always a big fan of like offloading not. So those are things that I do also time of day, I find them very productive in the morning. And so that's also not an optimal time that people like to start working. So I can find I get like a lot done then versus by the end of the day. Yeah, I'm checked out, I'm done. So that's even finding the right time of day can be really helpful.

Kathryn Koehler
Do you know what people say that being multi if you multitask, you're more productive, but it's not actually true. Because you're context switching all the time, I find that I can do two things at once better. So if I'm running or biking safely, I'm listening to an audio book at the same time, I can get through some of the sort of professional reading that I'm trying to do. Or if I'm knitting, rage knitting, which I like to do, I pick that up during the pandemic. I also listen to like leadership books and things like that at the same time, which make me twice as productive. And I guess I could turn it up to one and a half or 2x. But I can't listen to people talk like chipmunks. It's just too distracting.

Andy Glover
So can you ride a bike with a listening to something,

Kathryn Koehler
if I have one ear bud in, I'm fine. I will not go to earbuds because that's just dangerous. And I need to make sure that I have here through on and I'm not anywhere super super populated. Do not do this at home.

Ryan Burgess
That's fair. And I will I like to listen to podcasts or books while I'm walking or doing that type of stuff to is that I feel like is the healthy multitasking. When I'm in meetings and seeing slack messages and Twitter and all that that is not multitasking that is literally like distraction city. And I'm not paying attention. And that's frustrating. I think more so in the pandemic, because when I went to meetings in person, but I would take like a notebook just so that I could pay attention and not have the computer in front of me. So that's been something I've tried very hard to like, do not disturb. And that's that's helped a little bit, too. Absolutely. So what does it mean to have a company that has a dedicated productivity engineering team? And I know, CZ, I think Atlassian even has some form of this as well.

Stacy London
Yeah, I was I was gonna ask like, if we consider because they don't have, there's no team or like group called that in Atlassian. But there's definitely like, like platform teams or I was previously previous to Trello, I was working on the front end platform team. So we were working on tooling that helps build things that helps help developers be more efficient and get their jobs done easier. So it's I don't know if it's the same that it sounds like it could be

Kathryn Koehler
it is definitely the same thing. You just mean better branding. Yeah, I guess, a little productivity in front of that. And you're good to go.

Andy Glover
Oh, and and Netflix productivity engineering is inside of platform. And what's interesting is, you know, given our previous conversation about, you know, productivity tips. It's funny when I was in a situation at Netflix, in a meeting where I mentioned that I was with productivity engineering, and I was with some some business folks. And they were like, Oh, that's really interesting. We want to learn more about what you do and how you can help us and and I had to explain to them that we were, you know, our target audience, our developers not, you know, directors, seeking to make cut, you know, contracts with, let's say, studios, and they were just yearning for like, but are you know, I need more protein, I need to be more productive. It's just kind of amusing that I think all of us yearn for help there. But in our case, we're squarely focused on developers.

Kathryn Koehler
It's true. Yeah. And what does it mean for Netflix to have a productivity team? It means that we help our technical people get to their jobs. faster and do their jobs more efficiently and effectively. So they can spend all of their time thinking about business logic, great. Whereas we're consolidating all of the bootstrapping, we're packaging up great in demand experiences, so that, you know, with a couple flips, flicks, a switch, people can get up and running. And that's the ideal state gets a little more complicated in practice, we do actually support some non technical folks within Netflix, because we also have with Ryan's team source control, management, but also Confluence documentation, things like that. And so we do have a large group of non technical folks that we also support with our offerings. I

Ryan Burgess
like that you said the target audience, though, is developers, because I think that that's like, it can start to get muddy, if you start to try and make everyone productive. I mean, the goal is to make people more productive, but I think the, the focus is really helpful.

Andy Glover
Yeah, and Katherine hit the nail on the head with, you know, business logic in that, you know, every corporation at the end of the day, wants, you know, to produce value to whatever end users in our case, it's our members, you know, remember joy, ideally as quickly as possible. And so from a productivity engineering standpoint, our goal is to remove a lot of those obstacles so that developers can be productive, you know, on day zero, when that product, you know, that productivity while I'm putting air quotes around, it could be, you know, getting a feature out to, you know, production, ideally on day zero. And that feature would then, you know, obviously, make our members even more happy with the experience of working, you know, either watching Netflix or working with Netflix, from a studio standpoint,

Ryan Burgess
how would you best describe a productivity engineering team being successful? Like, obviously making engineers productive is a good thing. But like, I'm curious, when? How should we be thinking about that as like, what's what's the goal, or how do we know that we're being successful,

Kathryn Koehler
things should just work, right. And it shouldn't require a lot of hand holding. Things should be discoverable, they should make sense, they should get you set up and where you want to be with all of the necessary accouter Bas, like security scale testability built in. And then once they're up and running, anytime we make a change to that underlying system, they shouldn't have to go through an enormous amount of pain to update, right. So we should be holding their hands or making migrations more seamless, impossible for folks. I liken it to being a waiter at a really fancy restaurant, the meal just happens. It's not about like, oh, man, this is cold, this is terrible. They're interrupting me all the time, what the heck is going on, you leave feeling good about the experience. And you kind of have some impression that people were helping you in the background, but you could still focus on your conversations with the people you're hanging out with. And it's just an overall great experience, where we're that support staff, right. We're the folks that are sort of building that great experience for people.

Ryan Burgess
I really like that analogy, because it's really like great services, when you don't really think about it, it's when it's bad. You're like, Oh, that was bad service. So it's like, I like that that's really good.

Andy Glover
Either way, Catherine is the queen of analogy, I suspect that should have been like the drinking thing is like, that's gonna be the first of a great many through this

Kathryn Koehler
wave of analogies, mixed metaphors, oh, we're in trouble.

Andy Glover
You know, I think it's also worth pointing out that productivity engineering and Netflix is somewhat of a new development in that. So we've always been or I should say, there's always been a platform team. But productivity engineering, as a formalized organization was only created, I think, I want to say in the last three years, the developer experience at Netflix, I think in the beginning was was the amazing restaurant where like he got like, you know, a five course meal that was just absolutely delicious. But as the company has grown in terms of scale, complexity people members, that five course meal is still I guess it's still fine. I'm gonna kill this analogy. But like the break in between, like you're hungry in between the wait, you know, the waiter bringing that meal. And so you're experiencing end of the day isn't all that great, because you're like, I'm hungry, and I had to wait forever to get that other thing. So in the last three years, we recognize maybe we should formalize a group around productivity because there's some low hanging fruit there and that our developers at the end of the day aren't as productive as they could. So when we think about the experiences, it's, it's, you know, how can we connect our products and make the, let's say, the, you know, the transition through the SDLC a lot easier. Katherine, you know, introduced herself as running, you know, develop. org. We think of the SDLC is three pieces. There's like local development. That's everything happens. And then there's delivery, like how do you get that thing whatever you're working on into an environment And then there's like observability. After that, like, how do you know what's running? And what do you do when it actually breaks? Those domains, we have just deep, deep, deep, you know, experience and expertise there with amazing tools like Newt or Spinnaker Atlas, but we never thought about, like connecting them and these experiences as she mentioned. And so there's the low hanging fruit is like making that that process through those domains easier. And there's a lot of work to be done. And I'm sure we'll hit on it. But the the evolution to get to productive engineering, I think, is instructive.

Ryan Burgess
Right? And you said a couple things there that you're going to have to define like, what is Spinnaker? What does the Atlas what is Newt?

Andy Glover
So I'll start with Newt. So Newt is a is for Netflix.

Ryan Burgess
I, you know, I'm so guilty is like, I know what something is like, you know, once you figure out that acronym, and you're like, I can explain what that tool is. But you're like, I can't remember what all the words are in the acronym. That's just,

Andy Glover
yeah, Netflix workflow, toolkit or tool. But it's a command line tool that essentially attempts to bootstrap kind of your local development experience. Spinnaker is our CI CD platform. So how do you build something and ultimately put it somewhere, whether it be you know, any artifact that you produce, you know, via nude, and then Atlas is our telemetry class. So you know, you want to get metrics from your application, your library, Atlas has got it

Ryan Burgess
for you. Which Speaking of which, I've used Atlas many times to debug things, or just like, especially when you have a production issue, that's the first place you're going is to try and get that those metrics. Stacy, I'm curious, too, is like, there's similar tools within Atlassian, too. But like, there's a lot of this when you were on the platform developer team was a lot of similarities between that

Stacy London
well, I've seen builds a lot of like, productivity tools. Oh, weird.

Ryan Burgess
What are they? What are they built? Yeah, I've never heard of them.

Stacy London
Yeah, we have, we have some of those. Yeah, we got like CI CD, you know, products and stuff. So like pipelines, you know, doing the getting code out to production. There's also a lot of tooling that, you know, we don't necessarily build in house, there's observability tools or, you know, things that help you monitor logs easily. We're using, you know, industry tools for that, like Splunk. And, and century for errors. And those kinds of tools, or some of those are, you know, bought, but there are like inbuilt, you know, in house build stuff to like we're doing is an example, on the front end platform. Team, there's the site concept of like a mono repo. So a lot of a lot of code in a single repository, a lot of, you know, multi, multiple apps, multiple projects. And so there's like custom tooling around that to make that work well and be efficient. Make it so that you can work efficiently as a as an engineer in this large in this large repo. And I think that's the case and some other bigger companies that have these this concept of a mana repo, you generally have to do a lot of custom tooling around that. To be productive. I feel

Kathryn Koehler
like you're almost the most legitimate person among us when it comes to productivity because you work on a productivity team for productivity company, where, whereas Netflix is output sort of drives down productivity, right? Like we are the procrastinator tool, so I'm just I'm just giving you some productivity first, is I think about it a little bit. Yeah,

Stacy London
I can only be productive if I take breaks. And when I take breaks, I watch Netflix, so they feel good.

Ryan Burgess
Oh, just just a nice little ecosystem there. I love it.

Jem Young
I think the whole productivity, the idea of it is fascinating. Because especially as we all mature, as engineers, we understand engineering isn't just code, you know, your graduate school, you get up there and you're like, you see coders in the movies, you're like, what is it? It's like, oh, this is multiple screens, and the two keyboards, maybe three keyboards, who knows, and you know, just cranking out code and all this stuff. But we get older and we mature, our organization mature, and we realize actually, there's two parts engineering, there's coding. And then there's operations. And operations is oftentimes the dirty word. Like, people don't want to do that. They look at documentation or tests or making the developer experience better as part of that someone else's job. Like I'm here to code. But increasingly, especially at Netflix, we're seeing like, actually, now there's a lot of value in making 10 engineers more efficient, rather than you just cranking out one line, a few lines of code. The whole area is just fascinating. And as an industry, we probably need to focus a bit more on it. Because we spend so much time in every talk I've ever seen is on, you know, what's the latest design system? What's the latest? What's the latest front end tool and what are the browser's doing, but no one says like, hey, here are the steps we implemented and our team is now three times more productive than they were last year. Because I don't know we started documenting our processes or we opened internal StackOverflow Things like that you don't really hear that much about it, because it's, it's not sexy. It's not directly you shipping code, it's about making others more efficient. And that's just not conversations we're having. So I'm really, I think it's a really good topic.

Ryan Burgess
I love what you're saying there, Jem. And I think about it is it's interesting too, is like we're talking about companies, Netflix, Atlassian, fairly large companies. And they have actually now have teams dedicated to this. I've been in companies where it's very small, and you always have maybe like one or two engineers that are just passionate, and they just take that on and that, like, they're amazing for doing that. Like, they're just like, they're taking that on as an extra thing. But it's really cool when companies start to say, like, no, like, this is a thing. And we need to have more dedication. And we are getting value out of this for like we're able to scale. And that's essentially what's happening, which is really cool, too.

Jem Young
So hard question for the group. At what stage should a company think about implementing some sort of productivity organization? Because I would say a 10 person company, you probably need to make sure the lights stay on 100 100 people, maybe 500? Where's that ladder? Is there even a clear line?

Andy Glover
I wonder? Is it is it related to the product kind of maturity or trajectory, only from the standpoint of because I was thinking about what Ryan said, you know, when you're a small company, there's always someone really passionate about that. And previous to Netflix, I was at a startup. And I think maybe I was that passionate person that was like building that productivity stuff. But it was like it was very narrow minded and kind of focused on like, let's say certain aspects of what I found painful, once a product kind of gets into people's hands. And you've got the demands of like, you know, it's no longer beta. It's actually like, let's say it's legitimately being used, and it's scaling from that standpoint, I think we started to hit some kind of walls. And I wonder, you know, productivity went down, right? Because you had to be aware of all these things, so to speak. And we should come back to what is productivity mean, in the context of development, but I suspect even that startup, and that was fewer than 20 people could have used, maybe not a formal organization, your question, but at least someone like formally looking at like, how can we ensure that we're still acting like a startup and moving fast but aware of, as you pointed out, operations start to make things a lot more difficult. You know, once you get to that stage,

Kathryn Koehler
I agree with that. And I think that at that 20 person, and this is 20% tech or not 20 person company, right, like 20, person tech org, you start to buy, you don't build, you start to buy, you start to take a look around and see where where the scale is and where that infrastructure is that you can take off the shelf and maybe modify. But I think that if you don't do that, you have all of these folks within the individual teams doing this glue work, right thinking about how does it all go together? How do we do automation? How do we, you know, make sure that our systems are here, sorry, how to do it? How do we make sure that all the if you don't want to do that in a silo, because then you have a bunch of different frameworks that are half baked coming out of these groups, and your overall efficiency just takes a nosedive? Right? And so people have to be really judicious about, you know, what is that last responsible moment before they start investing in this more central team. But I think I love what you said, Jem, about ops and development and that mashup if people start thinking more proactively about their own productivity and how they can force multiply, like that is a serious leg up on on the usual route, the usual route to development.

Ryan Burgess
Maybe it's not like the number of people like even you know, like, I like Catherine, you said, like, 20 people, that seems like a real, maybe someone more dedicated to it. But I think it's also to his win, like I've in the past, have created something where it's like, it's helped me in some way, like it took a pain point off of me. And then like, others start using it. And that's when it starts being like, do I have to support this now? And now that's a tax like, it's like, I think when it starts to be that when something's been created, and someone's having to invest in the support and iterating on it, that's when it's like, maybe that needs to be a bit of a role. Because I know, there's times where I built something that's like, janky. It does what I need to be done, but then others start using it. I'm like, Oh, that wasn't exactly how I intended it to work. And I don't have the bandwidth to build it out to scale. Yeah.

Stacy London
Or maybe like you get, you have like two teams, you're big enough that you have like two engineering teams working on something. And there's like, the passionate person on each team that wants to do productivity stuff. And like, two people start to like, try and automate something at the same time that they're both trying to solve the same problem. Like at that point, it's like, oh, maybe we should have like years, two years here. And maybe then it makes sense to like, oh, maybe we should have you know, one person focus on this more and try and solve it for both.

Kathryn Koehler
Yeah, but the off the shelf solution could just be your JIRA cloud, right. Like you don't have to have this head be lifted. So Stacy, what size teams? Do you see people adopting a Trello or a JIRA cloud instance or, you know, these, these productivity tools that are often overlooked because it's not something that people roll themselves. It's,

Stacy London
it's so hard to say like, even like, when I was I was working in a smaller, well, not that small, the startup that I was at, before it came to Atlassian was about 80, I think 100 people, but like, the engineering group was like 2030. And even then we were using, like, the full stack of Atlassian server products to, because it was just, you know, helped connect everything and make everyone more productive. So even at that small of a group of engineers like that stuff is still helpful. Trello is interesting. That's like, productivity for any like one person to big companies that are using it. So that's kind of an interesting, it's more, it's a little more unique in that in that respect.

Ryan Burgess
Yeah, I think like, I mean, I've definitely even seen Mars isn't on today, but I know that she has like some of the coolest Trello setups for like her own personal life. It's really cool. And I think it's, I was always gonna say like, Trello makes sense when there's like two people, right? Like, the minute you add a second person that it's that makes a lot of sense as a tool. But she proved me wrong is like, no, there's like a lot of cool things that just as organizing your life can can be in a tool like that. So that is funny.

Jem Young
And I think it's important, especially I we have a lot of listeners or friend and happier regulators who are earlier in their career just getting started. It's important when you're looking at a company to evaluate the leadership in this lens, there's many ways to evaluate a leader and where you want to go, but one of them is do they respect the productivity? Do they understand that mythical, you know, the mythical man month, which is butts and seats equal more code, which is that not at all true? I've seen a lot of that and some some symptoms of a unhealthy culture is that belief that if we just hire more engineers, more stuff will get done, which, of course, is not how it works at all. What I look for, personally, is people that do respect that in terms of, okay, my engineers, aren't, we're not, we're not getting to the point where we want to be we're not building features in this instance, as fast as we want to, what are the blockers on that? And historically, what I've seen is people are like, Oh, the engineers aren't working long enough, they just need to work a few more hours every now and again. But this is this the antithesis of productivity, versus, actually, it turns out, you know, they need their own space, that's really what they need. They're right next to sales and sales on the phone all day. And the engineers are getting distracted or something like that, I I really think it's a valuable look at an organization, if you think about that lens about the leaders job is to help you be more productive, they can't do the code for you. But what they can do is clear away all that all the extra process, and tooling that doesn't work and cruft that gets in the way of you and doing your job effectively.

Kathryn Koehler
Yeah, I had a general manager asked me to cut my team's estimates in half, so we could get twice as much work done.

Ryan Burgess
And I worked right, clearly was a great idea. I was like,

Kathryn Koehler
Oh, my God, thank you so much for making that recommendation. Wow, I can see why you got this job. You're amazing.

Andy Glover
So that also goes to like, how you what you measure, right. And oftentimes, I think productivity, the people are looking for the outcome, right? Like, I to Jem's point, like, whatever we want to ship more features faster, whatever it is. And while I think that might be the most likely is the end goal, but like thinking about if you start measuring towards that it has awful consequences, as Catherine just pointed out. And so the real, something we I wouldn't say we struggle with it. But early on, we had a lot of debates on like, what are what are the productivity metrics we want to track? Like, how do we actually go and say, Hey, this is a great idea. Look, we've changed this metric. For what it's worth, I said that the organization has been around for three years, we still haven't actually settled on a series of metrics that we think are super important, which I think goes to the heart of the matter of like, at the end of the day, the company wants developers focus on, you know, producing value to our members. And there's a suspicion that they could be doing it faster, better, cheaper, whatever. We actually don't know what that baseline is. It's just anecdotal that we think they're less than less than productive. And so we got to figure out what the what the baseline is, and then show over time that we've, you know, made that metric better. And that's yeah, that's kind of the open question. It's definitely not lines of code. And other Thank you

Ryan Burgess
for calling that out. That is not one or like, admits, like, I have heard of some companies measuring commits to like developer productivity. And I'm like, we're rewarding the wrong thing. Like I'll just start, you know, every single line I write is a commit commit fit, like look how productive I am. I am curious, though, when you started talking about those, Andy, like, what were metrics that come to mind that could be useful to monitor for productivity? Well, and

Andy Glover
so, you know, my background previous to productivity engineering was in the delivery domain. And so one metric we were, you know, obviously, that was key to us in the beginning was deployments, like we're deploying more per day. So therefore, engineers at Netflix must be highly productive. And we could actually show it over time, you know, we started at 2000. And within time, we were at 4000, then we actually just stopped counting it. But then if you dig in deeper, it's like, well, are those 4000 deployments to production? Maybe it could be other environments? How many rollbacks? Like do you want for maintenance? 1900, Romax? You know, so it turned out, you know, maybe deployments isn't the best metric for productivity. And I think what we've, I think, you know, Katherine, we've talked about this, I think what we largely believe it is, it doesn't have a good name, actually, for a long time, we thought it was velocity, but turns out, yeah, velocity is problematic in that, like, well, what if, you know, I'm part of a team, maybe I'm in like a, you know, a financial application that's dealing with PII or whatever, you know, basically people's money, I have to run very long test to ensure that I don't mess that up. Because the consequence of messing it up is, you know, detrimental to the business. So I may, you know, be able to produce a feature like integrate a new billing, you know, partner, but then to test it, it's going to take hours or days or months. And so as my velocity, then like, a month to get that into, you know, in front of users, as opposed to maybe down the UI team, they can put it out in a day. And so we we actually started talking, and then there's lead time, right? So the business comes to, you know, an engineering group and says, We want you know, the green button on the top left, and at the end of the day, the business just wants to like they want that in production tomorrow, or today. Right? So it's that, but there's that lead time for everything. Jem alluded to, like, before coding, there's like a, you know, a bunch of conversations and like, you know, maybe some design, and then eventually it hits a, you know, a developer who actually is going to put that green button. And then there's everything after that we just talked about. So lead time is factor in there. But velocity is likely not it. And so, at this point, the working, you know, thing that Catherine, I've been talking about is and Brian, is essentially, if we could get a baseline for how long it takes a developer when they show up at Netflix, to push to production, or put a feature in front of an end user, how's that? Let's get away from the delivery part. If we could, if we knew what that was, then there's something we could put in that ignores lead time. But then we could measure it and then say, okay, like right now across the company, let's say it's two days. And our goal is to get it to like, you know, a first day or something like that. But that's a working theory, and I think will likely either get proven wrong or learn a bunch of lessons as we start to manage. Measure that more concretely.

Kathryn Koehler
Yeah. Yeah, that decrease in cycle time. And that, that, also customer satisfaction, right? You have a couple of really nicely worded questions that you ask people consistently through time. And you sort of, you know, correct for good days and bad days, but really pull that information in and figure out like, are we reducing your toil? When you think about using our tooling? Does that put a happy face on you? Or are you like, oh, here we go again,

Andy Glover
that goes back to like, yeah, you mentioned experience. And that's exactly it. It's yeah, I couldn't agree more.

Ryan Burgess
I also like the onboarding factor of like an engineer jumping in is like, there's so many factors, which it's hard, you're, you've got this flood of information coming at you. And I even think back to all these gems first day or a couple first days at Netflix, getting set up on the environment. He ran into this like, weird issue that was like, so bizarre, but it costs like I remember helping with that it was like half a day, just to track down this one like, really weird issue. And funny enough, is I if I remember correctly, Alex Liu, who is now in the productivity engineering group, which is in Catherine's team, who ended up helping us solve that, but like, those types of things that delayed his productivity by hours are almost a full day just because this one little issue. And so if you can make that to get like that much easier. That's I think that's a really good way to measure early onboarding,

Kathryn Koehler
and taking a look at our support volume too, right? Like how many requests are coming in for people and what are the nature of those requests? Is a table stakes or is it like really tricky, you know, like 1% questions, you can tell how well you're doing as a productivity or simply through that window

Stacy London
time it takes to get set up with your development environment. And then once it is running, how long does it take for the built? Let's say you make a change for, you know, whatever tooling you're using. So, you know, let's say it's doing a web pack build or something, how long does that take to happen? So how long are developers just sitting there waiting? While that happens, and then multiply that out times, you know, their whole day times X number of engineers, you can start to get like, some numbers, they're like, Oh, if we made, you know, the build time, much more instantaneous, we're saving, you know, 1000s of developer hours, you know, over times, those kind of metrics seem healthy, because it's like, there's nothing bad with that being faster.

Ryan Burgess
And that's something you can easily track to like when a bill kicks, build kicks off to when it's done. And then you can start to really pick out like, why is this slow, and you can start to figure out why it is maybe causing that and fix it, and then start to see that come down, which is that that's a valid point to

Kathryn Koehler
write. And then also really having a healthy culture on your team. So that people are not trying to gamify these metrics, these metrics are a tool to help you help the customer and to understand how effective we are being at our job, right, don't over index on this one thing, and then just go super hard at it and then think, like, I'm done. It's, it is just a tool, just like everything else.

Andy Glover
One thing, you know, with build times, and you know, let's say, Bill or deployment frequency, is we categorize those as proxy proxy metrics, that gave a signal of like productivity in a certain area. But again, the bigger picture was cycle time. You know, if your build time is you speed that up, I like whatever, you know, it used to be five minutes, and now it's two seconds, that's a win in that domain. But then it turns out, but it actually it's deployments. That's the big, it takes two hours to get the thing into production. So like a lot of you know, let's let's look there as well.

Jem Young
And Katherine Edie, you both mentioned this, and it's underrated. So I'll I'll mention it again, you've asked developers their sentiment on their own productivity, which is totally missed by so many people, because you're like, oh, Stacy, we built this new tool that interacts with this whiz bang tool and you deploy in seconds. Don't you feel great about it and facing and say, No, I didn't know about it, I don't know how to use it, the interface is confusing. And that happens so often, because we build all these great tooling. And we have this process, you're like, it's beautiful, we're done, we can go on vacation now. And it turns out, developers aren't any more productive, they're actually more frustrated, because now there's six tools to do something. And if you had just gone out and asked them to start with, how do you feel, what's your sentiment about developing within my organization? And that like right there, we'll cut to the heart of it. Because engineers, when it comes to coding will be very honest. And if something's blocking them, they will tell you very quickly, I think a lot of larger companies don't like to do that. Because it's more work. It's much easier to measure commits, or deploy time or pull requests, or Yeah, lines of code written, because those are discrete metrics. And you're like, Oh, we're down 33%, we must be doing our job or something like that, when it turns out, actually, people are just, you know, writing less code, or they're shipping more often, even though it's more fun. It's things like that. So I think that's really important is to remember the human side, the operational side in in all of these when you when it comes to metrics,

Stacy London
the coolest productivity engineering group is that your customers are not captive. I don't know what the right word is. That sounds

Kathryn Koehler
bad. But like fish in a barrel? Yeah, you have easy

Stacy London
access to them the like, they're the engineers working at your same company. So like, yeah, so what a cool opportunity to be able to like go do usability studies on them, or watch them or ask them and get, you know, feedback instead of like, customers, you know, that are out in the wild, and you don't have that kind of access to so it's a cool, it's kind of a different way of working, versus like working on product and customer face.

Ryan Burgess
Totally. Yeah, no, I think like that was absolutely what I want. And say, Stacy, I love that you said that. Because it's like you have this unique perspective to actually go talk to your customer versus when you're building like a large product that's being shipped to I mean, you can try and talk to your customers, but it is not it's an it's a fool's errand. You can't do that. And so but like, you literally can go to this person who's in your company and ask them how things are going. I think that is definitely a metric to use.

Kathryn Koehler
We're undertaking customer surveys of 200 of Netflix developers, and that's 10% of our technical workforce right there. Right, and we're in the middle of it right now. And we're encouraging all of our developers to sit in on these interviews, and hear from the horse's mouth, fish in a barrel horse, whatever, whatever. Zoom in on a barrel. Barrel. Thank you for like, their eye opening, right? Like overall. Yeah, I have a really good experience. Do you use any of our stuff? No. was too hard to use. So I stopped using it. So we rolled our own. And that's one example. Right. And that's really hard to hear. But so valuable for us to figure out what to prioritize on to make, you know, these these folks lives easier.

Andy Glover
But I think there's a lesson learned here that I think would be great to share. It highlights, you know, again, the investment in productivity engineering is relatively new at Netflix. And it also speaks to, like every company has, like, you know, the person or people that are highly interested in like, making developers lives easier. I, every company I've ever been at, there's always someone that's looking to, like, modify this to make it, you know, more easier and whatnot. And Netflix has been doing that from day one. We have, like, amazing developers that have built like just massively cool systems that ultimately, you know, facilitate, you know, making Netflix what it is today. And one cultural premise. And Netflix is this idea of decentralized or like, you know, loosely, coupled, highly aligned. And so what happened over the years is, you have these various products, and going back to like, the three domains I talked about with productivity, at least we look at the world from develop, deliver, operate, or excuse me, observe, you had these engineering teams in those domains, building, you know, and productivity tools and things for engineers. And they would actually go to the engineers and say, Hey, is this thing helpful? Is it useful? And they would say, Yeah, this is this is good, thank you. And then we'd like keep doing that. And the kind of aha moment occurred, sadly, in the near past recent past, I should say, where we, rather than asking from a siloed standpoint, like does this thing do your thing for you? Is we and Catherine hit on it, it's what's the experience, like when you come to Netflix, and your whole day spent, you know, working on features? What how is that working for you, and it was this aha moment will realize has a huge room for improvement, in that they, you know, a developer would be in this thing. And they'd get their job done. And maybe they were happy, maybe they work. And if they weren't, you know, they let the that team known. So that team optimize that thing. And then the developer would would then jump over to this thing, and it was the same process repeated, you know, and time, but no one was looking at the overall experience of like, from, you know, whatever, when you show up at work, and then when you finally decide to go home, how productive Are you? And there was a massive gap there. And so, you know, back to the gems earlier question about when should accompany, you know, look at formalizing productivity, maybe another point to make is that, when you think about productivity, look at it from the standpoint of like, a developer's day or their life cycle, not in these silos. And sadly, that's what we missed as, as as as an engineering organization and platform is that we were so hyperfocus individual silos, we never asked the like, the basic question of like the entire thing to her. So that's what we're highly focused on. And that's what we've zeroed in on the experience, because we know the experience is choppy, and we can make it better if we, you know, if we start asking at that level,

Ryan Burgess
what are some of the challenges of being a productivity based engineering team? What are some of the challenges? Well, this

Kathryn Koehler
factory is tough. I mean, walking around with that much swagger. really humble? Very humble. Oh, wait, was this a real? Okay? Yes,

Ryan Burgess
I actually like that response. It's great.

Kathryn Koehler
The scale of the thing, it's, it's tough. How do you run a lean team and meet the needs of all of your customers. So you have to be pretty ruthless in your participation. And you have to think really strategically about what you want to go after what you invest in, and we want to help everybody, but sometimes we can't, right, because some things are very far off the paved path. And we'll just end up thrashing the team if we if we keep supporting. So I would say that that push poll of what do we support? What do we not support? How do we go after the right thing? How do we place bets in the right places that will pay off down the road? How do we build a nimble company that can adopt and address things like moving from streaming to studio? Right? How do we not just pigeonhole ourselves? So I think that's really challenging, but I think that's also where all the fun and the opportunity is, as well. And the leisure suits are pretty cool. So

Andy Glover
to build on that, I think, yeah, the challenges are on metrics. I mean, everything we've talked about, I think one thing also that we are leaning heavily into and learning how to embrace is like Product Management. Again, P want to work on productivity engineering teams are naturally like the person that the startup That was building in any ways they want to help other people, they're highly empathetic. And, and that's where we ended up where we are today and that we have all these things that help people in the slivers, but never looking at the big picture. And a big challenge is, is, there's this mindset of we're just building for developers, and they're right there, we can go talk to him. So we like we don't have to build it. Like we would build a product like the Netflix product itself, where like Jem and Ryan, you were working, like there's like product management that thinks about, like, it's very much prioritize, none of that happens. You form teams, it's like, we're just like building stuff for developers. Like, of course, we know what's important, because I'm a developer, too. But like really thinking about, like, effective privatizations Catherine talked about, and unfortunately, having to say no, because like this in may help you, Jem. But like, the bigger impact is, other thing over here that's going to help all of let's say, like, studio get this done. And, and so our challenge going forward, by the way, is, is it hurts to say no, because again, we want to help you is can we do it in such a fashion that it's a platform that you could then extend, so like we didn't leave you like a hanging? It's, we can't help you we're gonna prioritize this big thing. Maybe it's utilitarian doesn't. But like, you can extend our, our experiences so that you can still be productive, but like, you're not waiting on us, because again, we're scaling. You know, we don't have unlimited resources. So it's, it's just like building product. And I think sometimes people forget that. So

Ryan Burgess
like, one thing to add on to that too, Andy, is you get these requests, because yeah, Jem is an engineer that might come talk to you and say, I need this. But it's also our job to understand his need, or what the request is not just to say, like, hey, I need this button in the right hand corner, like that's going to solve my world and life, and I'm going to be the most productive ever. It's like, what do you need that for? Like, what's that solving for you? And then also understanding, maybe there's five other requests that are very similar? And so how do you think about that more holistically, too, and then there's your impact, too, it's like, you're not just solving one off like every, you know, just being a team where you're like, cool, got a request, I'll do it. Another request comes in and do it. It's more holistic, and thinking strategically around that, which which can be challenging, I think, because you're having that customer who literally just asked you for, whereas on the product of like Netflix or Atlassian, you're not necessarily always hearing those like daily hallway conversation of like, Hey, Ryan, go fix this for me, or it's just, that doesn't happen as much. So I think that that adds to the challenges too.

Kathryn Koehler
Yeah, we want to build this really nicely paved banked on ramp unto a beautiful highway that a lot of other people are on, we did not want to build speed bumps walls, we are that nicely banged around, right? Where you can come up to speed and you can get you can get where you need to go

Ryan Burgess
the analogies Catherine, I love it.

Jem Young
You said this, and I like I want to highlight it again. One of the problems I imagine you having is, if you if you Andy it right, if you do your jobs correctly, people won't be sure you've done anything at all. Yes. And I imagine, at a certain level, you have to justify your value to the company. Because you say like, oh, people are so much more productive, blah, blah, blah. But you talked earlier about the difficulty of measuring that. And then but let's say you're somehow you get to 100% efficiency, you are crushing it, me as an engineer, I'm gonna be like, What do I even do over there? Like, I don't have any problems being productive. And it's because you did all this work. So I mentioned that's probably a product issue with any productivity engineering organization is people don't understand what you did. And only only when things are going wrong, then they start blaming you like, what are they doing over there?

Kathryn Koehler
Yeah, we have to be confident people.

Jem Young
Like, sorry, I have one last question. Because it's like such a fascinating topic. How do you balance that you want to build tooling and establish processes and make people more productive? But how do you stay adaptable for the future as well? Because the tools you build at say, 2000 engineers probably won't work at 20,000. Engineers, and how does the organization do you stay on top of that, and realize you're not being you give people flexibility, but you want some standard and rigidity? In the things you implement? For us.

Kathryn Koehler
It's really late, like, up leveling the layers of abstraction. So when those changes do come, we can make them seamlessly, right? You're you're interfacing with this, this, this, this level, and we're being intentional, and it's configurable, and it's extensible, and it's composable. Right. So that we can sort of change the the the landing equipment while you're flying the plane.

Andy Glover
In fact, we've already learned that lesson. You know, you mentioned when you go for 2000 to 40,000 I think you said so Netflix, I guess when I got to Netflix, there were fewer than 2000 employees. So I don't know how many of that was engineering we'll say was less than 1000 I think Engineering now is more than 5000 or 4000 5000. Ish?

Ryan Burgess
I think it's not. Yeah,

Andy Glover
so we went from 1000 to 5000. And when when we had, let's say, in the, in the productivity engineering kind of things, or the productivity suite that we built, exposes still some of its around today, all the low level, like primitives, infrastructure primitives, and it made that let's say, generation of the company highly productive, because they could, like, you know, turn the knob and switch the switch, or whatever the flip the switch, but it, it has created a challenge for us in that, like, we want to evolve the technology stack. And that requires a massive migration like either hands on or like, you know, getting in people's, you know, basically queue priority queue and being like, we need you to prioritize, like getting off of this old thing. This quarter, because we want to kill it. That's, that's really hard. And so to Catherine's point spot on is we want to raise the level of attraction, so that we can, you know, change the engine on the plane when it's flying, rather than asking him to land it, you know, de plane, change it around and get back on because that just doesn't happen, you know, with with businesses, right? I mean, Atlassian Netflix, we're all moving at the speed of light, so to speak, or speed of innovation. Oh, yeah,

Stacy London
there we go.

Jem Young
Silicon Valley. I love it.

Andy Glover
So yeah, it's just like, yeah, so yeah, we want to we want to raise a level of abstraction. And that's going to take time. And it's going to be it's going to be painful, but we are often it will pay dividends you know, on the on the backside of that.

Kathryn Koehler
Yeah, it's gonna be good, painful. It's gonna be like Jem pain. You know, you go work out, you're like, Yeah, I did something. We're moving mountains here.

Ryan Burgess
Great segue into our pics. In each episode, the front end happier podcast, we'd love to share things that we've just found interesting. Want to share with all of you. Alright, um, you want to kick us off with your pics? Oh, sure.

Jem Young
Today, I have two picks. The first one is a TV show. Not a new one. It is a Netflix Original outside the United States. It is Star Trek Discovery. I had started it when it first came out because I am a Star Trek fan. And I was turned off if that's like, this isn't Star Trek. This isn't this isn't the card and like, these noble ideals he's espousing this is totally different. This is more modern. And I recently came back to it with my wife because we ran our TV shows a watch we finished it screen and finish temps of unions we yeah, we ran out of like, things that draw us in or entertain us. So we started started Star Trek Discovery. And it's so good. It's so good. Like, you have to really get into it. And except that it's not the next generation or any of the Star Trek that came before it. It's a different universe almost. But they really build a compelling story and compelling characters out of based on the Star Trek universe. And it's, it's been really good so far. So I highly recommend it if you can watch it. My next pick is my valley silicon pick, that is the part of the show where I pick things that are ridiculous or too expensive and only exist because we here in Silicon Valley get paid too much. So for this pick, this one is called the air biotech 360. And as usual, I will share it share the link with everybody and the link will be in the show notes. But I'll just read off the description of what this product does. Featuring an all natural probiotic plan, this device will systematically diffuse millions of these bacteria fighting probiotics into the air landing on the surface bacteria reside on Yeah, yeah. Wow. So for this one, I'm not sure there's any science behind that. Like, I don't think putting bacteria in the air cancels out the other bacteria. I don't. I don't think bacteria work that way. I think you're actually just putting more bacteria in the air, which seems counterintuitive. But hey, if if this is something you want and believe is necessary, you can get the air biotech 360 for the low price of $999 We'll just call it 1000. So $4,000 You can have a device that sprays bacteria in the air that kills other bacteria, but the court of course, because it's Silicon Valley, there's a subscription service because you know, you can't sell something once you have to sell it repeatedly. And of course a refill on the probiotics are of course $900 So $1,000

Kathryn Koehler
This is your cero for bacteria.

Ryan Burgess
That's exactly what it is.

Kathryn Koehler
I mean, I've got hard kombucha here all I have to do is stick my nose in the in the hole and just take a sniff is that you actually

Jem Young
want to spread it around because you know you want to kill out the other bacteria.

Ryan Burgess
So Katherine yank it spit it everywhere. That's exactly how you do it.

Jem Young
I just like there, there's no white paper, there's no science. As far as I can tell behind this. It just seems like an aerosol, this person system, and then they put a fancy label on it and charge people a lot of money. Yeah. So you know, that's why it's my valley silicon pic. Because I don't know why these things exist. Or tweets about it does whatever, they'll need

Ryan Burgess
a second version of it gem that is Wi Fi enabled so that you can hold it from anywhere, that will be your premium one, that's probably 1500. Maybe 2k. Because that that Wi Fi chip is a little more expensive

Kathryn Koehler
with my K 95. filter out these probiotics to double mask.

Jem Young
You're gonna have to double mask, but I think they probably also sell a mask for you for about $200 It will let in the good bacteria, but not the bad bacteria.

Kathryn Koehler
It's Wi Fi enabled.

Jem Young
It's Wi Fi enabled Bluetooth, so you know when you're wearing it properly or not. But those are my picks.

Ryan Burgess
All right, Stacy, what do you have for us?

Stacy London
Yeah, follow that.

Ryan Burgess
I know.

Stacy London
I got two music pics. First one is a song called colors. To two flavors of it. It's by tiggo. One is color. So dub and one is color so bright. It's just like remixes of the same thing. But you say? Yeah,

Andy Glover
I'm so excited to meet someone else who knows Tikka. Oh,

Stacy London
you're not? Yes. Yes. I don't know. Much like I just heard these songs recently. So I'm like digging into the library.

Andy Glover
Oh, gosh, this is excited. We got to talk more about this offense. Awesome.

Stacy London
Yeah, the these particular songs, I guess he must have like maybe lost someone that was inspirational to him. So they're kind of inspired by that. And then yeah, just really dancey Fun, fun songs. And then the second is 6000 feet by bonobo and Totally Enormous Extinct Dinosaurs, which is one of my favorite named bands of or groups. And it's just a kind of a nice chill track at the real euphoric melodies, just kind of good a good coding and coding song,

Ryan Burgess
right on Catherine already have for us.

Kathryn Koehler
I'm reading a book called Accelerate. And it's not just because I'm sucking up to my boss. It is it is legitimately about building and scaling high performing technology organizations that I picked this up a couple months ago. And it's absolutely timed for the conversation today. And for anybody who's doing productivity. And this is it's been a good read so far. And it's all backed in data and actual science, which I like. So that's I have two pics, actually. Because this one's fun, but not fun. And the other one, I just downloaded Lolita by Nabokov, on my AUDIO BOOK player, I'm trying to make my way through the classics. And I know this one is a little bit controversial. But I listened to it while I read it. And it is recorded by Jeremy Irons. And so it is phenomenal. It is phenomenal.

Ryan Burgess
Alright, Andy, what do you have for us.

Andy Glover
So I'm getting through, I'm almost done with a book called Team of Rivals. And it's probably one of the best books I've read in a long time. And it's about Abraham Lincoln's cabinet, and how he went about picking the Cabinet members. And it's great. You know, again, I should say, again, but I am a history buff. And growing up, you always learn kind of one side of history. But as you know, on my journey, as an adult, I'm learning more and more about history and seeing different sides of it. But this is a really interesting book in terms of how, how actually a good person, when it came to leadership, Abraham Lincoln was in that he, his cabinet was full of people that actively did not like him. And that's how we staffed his team and Netflix we talk about like building dream teams and how like additivity what we're looking for in a team, and there's a lot of lessons from how he looked at his team, and was willing to bring people on that openly. didn't like him. And he was able to, through his, you know, relationship building, get all of them to value him as a leader and ultimately became friends or, you know, respected colleagues and all these people very interesting book and then one plug for a Netflix show that we just got through two seasons. It's called undercover. And it's a it's a produced in Belgium, so I had to watch it. It's it's smoking and Flemish, I don't speak Flemish so I had to watch it with subtitles. Really, really good story on it's the first seasons based on a true story of like an undercover operation to bus, a drug dealer based out of Belgium. Really good story. And so it's two seasons really enjoyed it and it's kind of got me on a kick. Watching a lot of foreign shows because it turns out they're really, really good.

Ryan Burgess
All right, well, and I have two picks as well, I feel like this one is a nod for Stacey a bit. And also for productivity in general. Gemini the other day made a Netflix engineering Spotify playlist. And I mean, I think some of the songs on there probably inspired in some way by Stacey anyways. So I feel like you've inspired Gemini that way. So we quickly just threw this together the other day, and we tweeted it out and shared that so I'll put that in the show notes. And then another one actually, that I'm kind of playing off of a pick from gems. While ago gem the device you bought like the air device that checks your air quality you're aware aware, thank you, I bought one as well. And then so got me down this whole kick of monitoring my air in like the office specially with the door closed and seen the more I talk in meetings all day, the air quality drops and plummets with like co2, obviously in the air. And so I went down this like rabbit hole of researching plants and things to improve the air quality. And I came across a plant which is very good for air quality is and the name is amazing. It's the mother in law tongue plant. It's just a really cool plant. It doesn't take a lot of work to manage it either. Like I think you have to water it every two to three weeks. And like yeah, I'm like my mother in law. Exactly. Hopefully she doesn't listen to this podcast. But yes, it's very cool plant looks good too, but it is supposed to help with your air quality. So I thought that was a good add on to if you got the aware that gem suggested a few episodes ago.

Kathryn Koehler
That's also the safe plant.

Ryan Burgess
It is also called that I went with the like there's the two names, but I was like kind of like mother in law tongue. It sounds pretty interesting. So

Jem Young
I really think people should get air quality monitors like I know, you know, I've anti smart home all around. But for instance, now my windows closed and my doors closed because we're doing a podcast, the co2 levels my higher like at my office are extremely high right now. And I can like a really bad right now. Yeah, yeah. If you've been talking for an hour, you're just putting out a ton of co2. It's it messes with your cognitive load. So speaking of productivity, this well,

Ryan Burgess
yeah, there you go. That's, that's a good added on plants, air quality, it will make you more productive. All right. Well, before we end the episode, I want to thank Andy and Catherine, thanks so much for joining us and sharing us full of knowledge. It was a pleasure having you both where can people get in touch with you?

Andy Glover
Thank you. You can get in touch with me on Twitter. I'm a Glover.

Kathryn Koehler
Oh, yeah. Thank you so much. This has been awesome. I would like to say that I'm on Twitter, but I'm not really on Twitter. I mean, I have a Twitter account, but I have three posts since I set up so it might not get a lot of traction, but you can try. Katie J Kohler or at Katie J Kohler is my handle

Ryan Burgess
and I'm sure they can find you on LinkedIn. Well yes things Yeah,

Kathryn Koehler
old people Facebook. Feel free to find me on

Andy Glover
Facebook.

Ryan Burgess
Facebook. Yes. Thank you all for listening. Today's episode, you can find front end happy hour at front end happier.com Subscribe to us on whatever you really like to listen to podcasts on. And you can follow us on Twitter at @frontendhh any last words?

Stacy London
automate, automate, automate,

Ryan Burgess
automate all the things Cheers.