The Past, Present, And Future Of The FLUFL: Barry Warsaw Shares His History With Python - Episode 271

Summary

Barry Warsaw has been a member of the Python community since the very beginning. His contributions to the growth of the language and its ecosystem are innumerable and diverse, earning him the title of Friendly Language Uncle For Life. In this episode he reminisces on his experiences as a core developer, a member of the Python Steering Committee, and his roles at Canonical and LinkedIn supporting the use of Python at those companies. In order to know where you are going it is always important to understand where you have been and this was a great conversation to get a sense of the history of how Python has gotten to where it is today.

Do you want to try out some of the tools and applications that you heard about on Podcast.__init__? Do you have a side project that you want to share with the world? With Linode’s managed Kubernetes platform it’s now even easier to get started with the latest in cloud technologies. With the combined power of the leading container orchestrator and the speed and reliability of Linode’s object storage, node balancers, block storage, and dedicated CPU or GPU instances, you’ve got everything you need to scale up. Go to pythonpodcast.com/linode today and get a $60 credit to launch a new cluster, run a server, upload some data, or… And don’t forget to thank them for being a long time supporter of Podcast.__init__!


Datadog is a powerful, easy to use service for gaining comprehensive visibility into the state of your applications.

The easy to install Python agent lets you collect system metrics and log data, supports integrations with all of your services, and distributed tracing.

Their customizable dashboards and interactive graphs make finding and fixing performance issues fast and easy, and their machine-learning driven alerting ensures that you always know what is happening in your systems.

If you need even more detail about how your application is functioning you can track custom metrics, and their Application Performance Monitoring (APM) tools let you track the flow of requests through your stack.

Start tracking the performance of your apps with a free trial at pythonpodcast.com/datadog. If you sign up for a trial and install the agent, Datadog will send you a free t-shirt.

 



Announcements

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • When you’re ready to launch your next app or want to try a project you hear about on the show, you’ll need somewhere to deploy it, so take a look at our friends over at Linode. With the launch of their managed Kubernetes platform it’s easy to get started with the next generation of deployment and scaling, powered by the battle tested Linode platform, including simple pricing, node balancers, 40Gbit networking, dedicated CPU and GPU instances, and worldwide data centers. Go to pythonpodcast.com/linode and get a $60 credit to try out a Kubernetes cluster of your own. And don’t forget to thank them for their continued support of this show!
  • This episode of Python Podcast is brought to you by Datadog. Do you have an app in production that is slower than you like? Is its performance all over the place (sometimes fast, sometimes slow)? Do you know why? With Datadog, you will. You can troubleshoot your app’s performance with Datadog’s end-to-end tracing and in one click correlate those Python traces with related logs and metrics. Use their detailed flame graphs to identify bottlenecks and latency in that app of yours. Start tracking the performance of your apps with a free trial at datadog.com/pythonpodcast. If you sign up for a trial and install the agent, Datadog will send you a free t-shirt.
  • You listen to this show to learn and stay up to date with the ways that Python is being used, including the latest in machine learning and data analysis. For more opportunities to stay up to date, gain new skills, and learn from your peers there are a growing number of virtual events that you can attend from the comfort and safety of your home. Go to pythonpodcast.com/conferences to check out the upcoming events being offered by our partners and get registered today!
  • Your host as usual is Tobias Macey and today I’m interviewing Barry Warsaw about his role in the Python community, past, present, and future.

Interview

  • Introductions
  • How did you get introduced to Python?
  • For anyone who isn’t familiar with you, how would you characterize your role in the Python language and community?
  • What have been your main areas of focus in your role as a core developer?
    • What are some of the other forms that your contributions to the language and community have taken?
  • What are the contributions to Python that you are most proud of?
  • Looking back at the past 25 years of Python, what do you find most interesting/surprising/exciting?
  • How has the focus of the community changed or evolved since you first began using it?
  • What are you currently focused on in your role in the steering council?
  • What are the aspects of the language and community that you think need greater attention?
  • What are the core strengths of the language and community that you believe will carry it through the next 25 years?
  • In your current and previous roles you acted as a guiding force for Python. What are the main use cases for Python at LinkedIn?
    • What kinds of projects are you involved with to support the other engineers in their use of Python?
  • How much of an impact has the invisible hand of the PSU had on the overall trajectory of Python?
  • Outside of Python, what are the programming languages or communities that you look to for inspiration?
  • What are your personal goals for the future of Python?

Keep In Touch

Picks

Closing Announcements

  • Thank you for listening! Don’t forget to check out our other show, the Data Engineering Podcast for the latest on modern data management.
  • Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
  • If you’ve learned something or tried out a project from the show then tell us about it! Email hosts@podcastinit.com) with your story.
  • To help other people find the show please leave a review on iTunes and tell your friends and co-workers
  • Join the community in the new Zulip chat workspace at pythonpodcast.com/chat

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Click here to read the raw transcript...
Tobias Macey
0:00:13
Hello, and welcome to podcast dotnet, the podcast about Python and the people who make it great. When you're ready to launch your next app or want to try out a project to hear about on the show, you'll need somewhere to deploy it. So take a look at our friends over at linode. With the launch of their managed Kubernetes platform, it's easy to get started with a next generation of deployment and scaling powered by the battle tested linode platform including simple pricing node balancers, 40 gigabit networking dedicated CPU and GPU instances and worldwide data centers could a Python podcast.com slash linode today that's Li n od E and get a $60 credit to try out a Kubernetes cluster of your own. And don't forget to thank them for their continued support of this show. You listen to this show to learn and stay up to date with the ways that Python is being used including the latest in machine learning and data analysis. For more opportunities to stay up to date, gain new skills and learn from your peers. There are a growing number of virtual events that you can attend from the comfort and safety of your own home. Go to Python podcast comm slash conferences to check out the upcoming events being offered by our partners and get registered today. Your host, as usual is Tobias Macey, and today I'm interviewing the fluffle Barry Warsaw about his role in the Python community past, present and future. So Barry, can you start by introducing yourself? Hello, everybody.
Barry Warsaw
0:01:26
I'm I'm Barry Warsaw. I've been a core developer since 1994. So I've been involved in Python for quite a long time currently serve on the steering Python steering Council, my second term, and I work at LinkedIn
Tobias Macey
0:01:42
looking at the acronym for steering council where one letter off from the Python secret underground which emphatically does not exist.
Barry Warsaw
0:01:50
Exactly.
0:01:52
I'll just point out that, you know, Python, of course, is named after Monty Python. So humor has always been, you know, an integral part of the Python community.
Tobias Macey
0:02:01
Absolutely. So you shared the story a little bit in the interview that you were on a few years ago now talking about the overall Python enhancement proposal process. But for anybody who hasn't listened to that, can you just give a bit of the background of how you got introduced to Python?
Barry Warsaw
0:02:16
Sure. So actually, it was interesting because I don't know how many people remember Usenet. But Usenet was a big thing was like a kind of a bulletin board forum system that people used. And comp, like Python showed up one time, and I was kind of into programming languages. So I checked it out. And it was nothing but money Python jokes, so I thought, well, this is great. But like, what is it doing in comp Lang? Right, like, what does it have to do with, you know, programming languages, so I kind of set it aside. And then a few months later, some friends of mine at the National Institute of Standards and Technology which I had worked at, you know, up until about 1990, sent me an email and said, hey, we've got this guy coming over and he's going to give a three day workshop. shop and would you like to attend and it was, you know, local to where I was living. So I had just started at the corporation for national research initiatives. And so myself and one of my colleagues went up and we attended the first Python workshop in November of 1994. And I got to meet Guido and I got to see how the language was being developed. And I just kind of fell in love with both I mean, Python, the language I had sort of dabbled in it before the workshop and and gotten some initial impressions and really loved the language. And once I met Guido, you know, he's still a very close friend of mine. I think he's just a fantastic person and technologist and, and I think he's been an amazing leader for for the language. So I just fell in love with it. We ended up using it at cnri. And we, you know, over the next few months, Guido, we hired Guido and he moved to the states and I worked with him, you know, within throwing distance, you know, in the next cube for about eight years before he moved Down to California. And then three years ago, I followed him out here, so.
Tobias Macey
0:04:05
And as somebody who has been involved with the language for so long, what are your senses of how much of the core elements have stayed true all the way throughout? And how much is it really changed in terms of its character or capacity in that time?
Barry Warsaw
0:04:20
Yeah, that's a really interesting question. Um, last year, I gave a talk at the local Bay piggies. So it's, you know, the, you know, Silicon Valley Bay Area, Python interest group, and I went through and pulled down the tar balls for all the Python versions going back to like 093 grabbed all those looked at the change logs, I couldn't compile any of them on any modern system. without significant amount of work. None of them built. None of the early Python zeros or Python ones built but I went through and looked at the change log and kind of got a sense of what the you know, remembering what the language was like in the early days. And a modern Python developer would look at python 1.1. And say it looks very familiar, you know, it has not radically changed. In some ways, it has exactly the same feel that, you know, I fell in love with 25 years ago, but in a lot of ways, it's changed immensely. For example, you know, there was one release early on where Guido just, you know, within one release, change the equality operator, you know, it used to be that assignment and equality were both a single equal sign. And in one release, Guido just said, I'm changing it to double equals, and there was no consideration of backwards compatibility. There was no problem with like people just having to change their code. So like a lot of these little things got introduced early on, and the language was changing very quickly, and now, it still evolves but much more slowly. And you see things crop up like keyword arguments. Everybody sort of takes you know, thinks keyword arguments. are a standard part of Python. But there was a time before keyword arguments were part of the language. And you can see the sort of revolution in the way people think about it. I remember one of the biggest impetus for that was tickle TK, you know, tried to do graphics with Python and the to the tickle bindings, you had to specify every single argument and it just got completely unwieldy. And then once keyword arguments were added, you know, all the calls got much simpler. And you can just see how like that one little feature has changed the flavor the feel of Python immensely, and it's something that somebody coming into the language today wouldn't even think about.
Tobias Macey
0:06:41
Yeah, I can definitely see how that would be a pretty revolutionary change, where all of a sudden, you can have sane defaults for all of your functions, while still allowing for all of the expressivity that you really want, which is one of the more useful aspects of the keyword arguments but not having them and having to specify every single argument every single time would lead to a lot of boilerplate to be Able to wrap the function calls in the same defaults so that you have some way of not having to rewrite them every single time.
Barry Warsaw
0:07:06
Exactly right, exactly right. And there's just so many of those little things and you know, looking back at Python now you can see how Guido's sense of aesthetics for programming has really served, you know, both him and the language and the community really well. He just seems to get it right at the heart of the matter. And a great example, in modern days I think is pepp 572, which is the walrus operator. And when the first draft of that pet came out, I was really against it. I was like, this is this is wrong for the language, you know, we don't need it and the semantics are tricky. But you know, I had I really had faith in in Guido because he really listened carefully to a lot of the comments and feedback that was coming back some of it not entirely kind Which is very unfortunate, but what ended up coming out, I actually turn the page and I'm a big 572 fan. I'm a big walrus operator fan now. So you know, you see this, this sense of trying to do the right thing and find the right balance between power expressive sensitivity, and pythonic ness, which is a very sort of amorphous quality to wrap your hands around, but it's still happening today. And so, you know, I have a lot of faith that the language is still you know, still a coherent, friendly, fun language to use.
Tobias Macey
0:08:36
Yeah, it's definitely interesting seeing the evolutions I've only been using it since around the 2.4 time range, which has still been enough of a change in terms of the capabilities particularly with Python three, adding in a bunch of new library capabilities and adding in a lot of the default numeric operations that people are using it for these days and I am on board with you in terms of the walrus operator. If you And some of my code recently and definitely appreciate being able to have that assignment expression in line with the conditional. So it's interesting seeing some of the battles that are won and lost in the peps and sort of how that plays out.
Barry Warsaw
0:09:14
Yeah, at one point I had the I was, I was kind of proud of this. I had the most rejected peps of anybody. So you know, I think you got to just you know, you put these ideas out and sometimes and in fact, I've had peps that were have been retro actively accepted, believe it or not.
Tobias Macey
0:09:30
So yeah, well, definitely lessons to be learned and failure for sure for sure unlined with the work that you've done with the peps for people who aren't familiar with you and your overall involvement in the Python language and community. Can you just give a bit of a characterization of your role in how you have worked with the language and for the language and the overall community, the ecosystem that has grown up around it?
Barry Warsaw
0:09:54
Yeah, that's it's hard to sort of succinctly summarize 25 years but You know, I did early stuff with, I'm sure nobody remembers anymore something called the grand renaming. But anybody who's played with the C API for Python, you know, it looks very nice. It's it's all logical for the most part, a couple of words here and there. But the naming scheme makes sense. It's easy to find all the documentation. It wasn't the case early on. And one of the very first projects that I worked on when Guido had moved to the States was what we call the grand renaming. And that was taking all the C API stuff and putting all the pie, you know, string, this underscore this and that's in PI XC, this and that's in front of all the public names, because at the time, you couldn't embed Python in other c programs, and you couldn't really extend it very well because there were lots of name clashes that were just sort of global c namespace stuff and we made all that better, you know, with a lot of direction from pipe from Guido Of course, but myself and a few of the other early Sr. Ayers did that. So I've dabbled in the C code. I've dabbled in the library. I've dabbled in the build system. I've been a release manager for for several releases. I, I kind of came up with the whole idea of peps. And I named the peps and I really feel like the pet processes. One is something that I'm particularly proud of, because I think it's really given us a way to evolve the language. And we could talk about that, but I've really dabbled in a lot of things. I would say that these days I don't do as much programming on Python, as I had in the past. I just sort of see my role is kind of changing. And of course, my interests are changing as well. But I've really dabbled in quite a bit of the the code.
Tobias Macey
0:11:43
Yeah, definitely second year notion that the peps have been integral to the overall evolution of the language and it's something that a lot of different languages and development communities and library communities are leaning on this notion of a public sort of RFC process. And a lot of the acronyms mirror the peps as far as the overall idea of enhancement proposals. And so I think that that's definitely been a great reference point for other languages and communities who want to be very deliberate about the way that they work on the capabilities and functionalities and the ways that you use the technologies that they're building. I think you're absolutely right.
Barry Warsaw
0:12:22
And and, you know, the peps kind of came out of our experience at cnri, because they were running the IETF meetings. And so we saw firsthand the, the RFC process for internet standards, but we wanted something that was much more lightweight, and it was just enough bureaucracy to capture what we needed to capture. And so that was really the origin of the Pepsi although the acronym Pep was definitely a backronym. You know, I just thought, you know, it was kind of a funny epi word, you know, a happy word and man I filled in the building the backronym afterwards but yeah, it's it's that's been wildly successful. I've been, at one time I was heavily involved in the Debian community and they use depths Debian enhancement proposals. So it's been very cool to see that happen.
Tobias Macey
0:13:13
Yeah. And as you mentioned, the Python community because of its origin in humor, he has a lot of inside jokes and Easter eggs and the peps are no exception there where you have Pep 401 where you were declared the fluffle accidentally when you were trying to nominate Tim Peters for the rule. Yeah, yeah, exactly. And Pep 801, which was reserved for some enigmatic reason that nobody quite understands.
Barry Warsaw
0:13:37
Yes, excellent. And there's also you know, we have Pep 666 I think and, and what there's another funny one, I think, for of course, for a one, but yeah, that that was kind of, you know, we were I was one of the pep editors. So that one point, before we move to GitHub, we had to sort of dole out the numbers and I was able to sort of reserve few friends, you know, fun numbers. So
Tobias Macey
0:14:03
do you have any good plans in place for number 801?
Barry Warsaw
0:14:06
I don't but I, you know, there was a band in the 70s, called 801. And that's kind of why I'm reserving it just because I, you know, maybe something fun will come up at some point,
Tobias Macey
0:14:17
maybe you can auction it off at one of the PI ladies events ago. It's a great idea. And so in terms of outside of the language and your engagement with the community, and some of the ways that you have worked to help promote the language and promote its use, what are some of the other ways that you've contributed in that regard and some of the contributions that you're most proud of either technical or otherwise?
Barry Warsaw
0:14:41
Well, that's Yeah, that's interesting. You know, I've said this before this is this is a story I've told quite often, which is that at that first workshop, you know, there were 20 of us in a little room in a government, you know, research facility in Gaithersburg, Maryland. And a number of the people who were there came out and said, you know, you can cannot tell anybody that I'm here because Python is a competitive advantage. And we don't want our competitors to know that we use Python, you know, fast forward now. And Python is everywhere. And it was something where I really believed in Python quite strongly right from the get go. You know, I mean, I've been a fan of programming languages. I've, you know, I've used dozens and dozens and dozens of languages in my career, and I still continue to learn new languages. And I love that. But there's something special about Python. And I really noticed that right on and I really tried in every job that I was at, or every side hacking project to promote Python and really say, you know, Python is the right solution for this. And I think a number of us in those early days really believed very strongly in Python, and just pushed it in into every nook and cranny that we could get away with. And you see that now Python is arguably within the time Top three, if not the second most popular language, the domains in which Python is relevant are so vast and broad that I think that's a function of us really pushing Python, not just me, of course, I mean, everybody who, who is a fan of Python, you know, pushing Python into new and better and more interesting and challenging domains. So you see, like Python growing up into the number one language and data sciences, you know, which nobody could have even thought about back then. So, yeah, pythons not appropriate for everything. But it is appropriate for so much that just being an advocate for it and pushing for it and proving that's the other thing is really building systems. And proving that Python is can build small scripts, can build libraries can build applications, can build websites can build Instagram, for example, you know, like Python is so versatile. So easy to get into. And I think that that is just something that we've always been promoting and pushing and that that I'm particularly happy about. So yeah, one of my favorite ways to encapsulate that is I've heard a few times that Python is not the best language, anything, but it's always the second best language in everything. Yeah. Although argue it's the best language for a lot of stuff.
Tobias Macey
0:17:22
And so looking back at the past 25 years of your involvement with Python, what are some of the things that in retrospect, have been the most interesting or surprising or exciting and some of the turning points in both the success of the language and in your own career and involvement with it?
Barry Warsaw
0:17:39
I think I think the thing that never ceases to surprise me is being up on stage. Now hopefully we can do this again after you know, after we get a handle on the whole Coronavirus thing, but being up on stage and looking out at 3000 Python fans and those are only the people you know, like like sellout crowd at pi con. And that just blows me away every single time. I look out into that to that crowd, and it's not just you know, sort of grizzled old veterans like myself, a lot of young people are coming into the language and very diverse, you know, communities of Python fans, that just, you know, I always believed in the language, but I guess in your heart of hearts, you sort of think there's lots of competition in programming languages, why should Python win, you know, and yet, in a large way, it sort of has, you know, it is so pervasive in industry and in education. And that, to me, has just been completely surprising, I guess, and very exciting. And I think pythons got a very bright future. It's not going anywhere. And I think as as more young developers come in, and I've really been encouraged to see you know, a lot of younger developers come in and and also fall in love with Python. And then take an active role in evolving the language and the implementation and that to me, that's a sign of hell.
Tobias Macey
0:19:08
Yeah, I definitely think that its utility is a learning language is really helping and bringing in that new generation of programmers and I'm trying to do my part with teaching my children as well. But it's interesting to to see how the overall focus of the community has changed and grown since the early days where as you said, it started with a roomful of 20 people in a government building. And now it's, you know, multiple conferences, worldwide, meetup, groups, online forums, everything that you can possibly imagine in terms of communication and community. So what are some of the main areas of focus that you have seen in the community as it grows? And what are some of the areas that you think need greater attention or support?
Barry Warsaw
0:19:53
I guess there's there's two ways to look at it sort of like from the technology side and then from the community side, from a technology side One of the things I think Python has done really well is it's sort of been a kind of a chameleon language. So what I mean by that is, you know, in the early days, you know, people were writing scripts to do system administration or little, little applications and tools and things like that to solve their, their problems of the day. And as the web got very big, you saw things like Python, adapt and grow into various web technologies, both crawling the web and providing, you know, back in the day, CGI bin scripts and building sort of the back into the web and adapting to, to the, to the web is that as that's come up, and you know, now it's its role is is hugely important in the data sciences. So I think as technology changes, Python is able in large part to adapt to that and who knows what the next big thing is going to be. But I suspect that Python will be one of the first languages that embraces whatever that next big thing is. I do think Python has some technological challenges that I, you know, as part of the steering Council, I hope that we can find ways to address for example, Python, I think needs to be much better at multi core. And I would love to see Python in the browser, like support, you know, a web assembly back end or something like that. So I think there are tech technology, technological challenges for Python as computing changes. I mean, you know, the state of computers today is so different from 25 years ago, and you can't even imagine what it's going to be like in the next 25 years or the next five years. But I have faith that Python will Python the language as opposed to necessarily c Python, the the reference implementation, or the most popular infant implementation will adapt to those changing computing environments and new domains. So that's sort of the technology side of things and I and there are people working on all that stuff. So I have, I have a lot of confidence that those Important domains Python will, will adapt better to, on the community side, I think we have a lot of we have a lot of good structures in place a lot of good institutions in place. I think the PSF, for example, has been a fantastic development for Python. And you know, the sense that the community embraces everybody, right? Like, it doesn't matter. You don't even have to be a C programmer, for example, to become a core developer, you don't even necessarily have to work on the standard library to become a core developer like we like there is so much to do that people can contribute and make Python better and documentation, build systems, you know, ci CD, there's just so many things, education, building the community, bringing in more widening the diversity circle and bringing in more people from underrepresented communities into Python. Those are all like really important things and i and i Know that, you know, I I certainly support all those things. And, you know, everybody that I know of in leadership positions in the Python community support that as well. I mean, that's where we, you know, I'm a big believer in diversity is our strength, you know, and that's along so many different different dimensions. But that's what you know, those new experiences and that enthusiasm is what really keeps Python going. So, you know, we have challenges in the community, and we're trying to address them. And I think anytime you have a programming language or or community that's as large and as diverse as we are, you're going to have challenges to, you know, in any large Society of human beings. That's a natural, natural effect. But you know, we really want to make this a welcoming, friendly, happy programming language and development community. And to your point about the size, when do you think Python is going to become a sovereign nation? Hmm. Well, I'm hoping the PSF buys Sealand and then we can just own There.
Tobias Macey
0:24:03
And so you alluded to your membership in the Python steering Council. And that is definitely one of the biggest shifts in recent memory for the language with Guido stepping down as the benevolent dictator and deferring to this new community structure that there was some period where there was no firm ideas to what that meant for the future of the language. Thankfully, it has become settled. And we've come to what I think is a very good solution to that problem. And so in your role on the steering Council, what are some of the main areas that you're personally engaged with and focusing on?
Barry Warsaw
0:24:39
I think, you know, I took very seriously my first term, which I think the main thing that we had to do, you know, even though we had like, had this community vote, and we came up with the current governance, governance system, there was a lot of things that were sort of still kind of left up to the steering council to decide, you know, what the process was for various things and those things still coming. We still are sort of like, we're not quite sure what the right process is for X, Y, or Z. But I felt it was a responsibility of mine as it was in the early days of Python to get the processes and mechanisms for functional governance in place, you know, that could sustain. I mean, I'm not going to be on the steering council forever. And you know, there's going to be lots and lots of people serving on the steering Council and I and I feel like we've gotten to a really good place where it's, it's a important, stable, functioning institution for deciding how, you know, the language is going to evolve. And I took that responsibility pretty seriously in my first term. And we also had a lot of just housekeeping to do because as after Guido had stepped down, there was a long period of time where lots of peps needed to be decided on and we just had to slow sort of slog through those and figure out you know, are we going to delegate responsibility for actions thing this one or are we going to keep this one in our in our wheelhouse and we just had to get through all those peps in this second term, I think we've you know, we're in a pretty good steady state with things. And so now we're really talking about things like what's the future of Python like if we, for example, if we had
Tobias Macey
0:26:19
a pod portion of podcast, data dog what would happen productivity slower than you like two more minutes all over the place? Time slow. Moving with data dog, you will troubleshoot your dog tracing, correlate those pipe on traces with related logs, or support detailed flame graphs to identify bottlenecks and latencies. performance of your apps the free trial is on podcasts
0:26:49
comm slash data dog.
0:26:50
To sign up for a trial and install the agent data dog. We'll send you a free t shirt to keep you coming around while you keep track of your app or
Barry Warsaw
0:26:56
big ticket items in Python and And how we're going to fund them. You know what the structure for that is going to be? How would you still value and balance the contributions of volunteers, understanding that volunteers have a very difficult time implementing and bring to fruition some of these really big changes, right? Like, those are some of the issues that we're dealing with as well.
Tobias Macey
0:27:20
And for some of those far reaching and impactful areas of improvement for the language, Russell, Keith McGee did an excellent keynote at the last icon, which I'll link to in the show notes, and we did a follow up conversation on that. And to your point of funding, I know that that's a conversation that is ongoing, but at the last Python, I saw a few different ideas in terms of how to pursue that and some of the existing challenges in terms of how the organizational structure of the PSF limits in some ways, corporate donations, and just some of the overall ideas about how to bring in more money to allow for those more far reaching and forward looking products. And I'm wondering what your overall thoughts are on that or any of the current developments or current efforts that are going on to help ensure that those projects are able to happen and be sustainable.
Barry Warsaw
0:28:12
One thing that I'm especially happy about is the way the steering council as the technical leaders and the PSF as the community leaders work together. So, you know, I feel very strongly that those divisions of expertise and responsibility are important for Python because the community nurturing the community is not a technical, not necessarily a technical side of things. And while there's lots of community in the steering council that we think about, we're really sort of focused on the technical side of pythons evolution, but the two organizations work really well together. So we talk to the PSF on a regular basis. In fact, Eva attends our steering council meetings and that's been fantastic. I can't I cannot speak highly enough of Eva He's absolutely fantastic. So, you know, they look to us to sort of what are the technical directions that we want to pursue? And then the PSF kind of thinks about, you know, how can we build the structure and the organization to make that happen. And so those are ongoing talks that we we have all the time about how to how to do the things that we want to do that we think that it's difficult for the volunteer community to do but not devaluing, or taking away any of the fun and responsibility and focus that the that the volunteers bring to the Python, it's, it's, it's, it's ongoing discussions about how all that stuff works, you know, and nothing that we're doing, you know, we try to communicate to the community when we have something that's a value to communicate, and we and, you know, the the community's input is is absolutely important. You know, it's it's vital. You know, Python has always evolved from a sort of more grassroots movement. You know, if you like Guido has said this before, For there are a few initiatives where he said we should do it this way or this way. It's all it's a lot of it has been things that percolate up from the community. And then you know, Quito finds the right way to, to make that happen, you know, to put that together. So I don't feel like the steering councils role is to come up with great ideas. It's more to look at the ideas that are coming up from the community and from the grassroots and that people are challenged with or thinking about and trying to decide what the focus is, you know, what, what the right focuses are.
Tobias Macey
0:30:38
This portion of podcast dotnet is brought to you by data dog. Do you have an app in production that is slower than you like? is its performance all over the place? Sometimes fast and sometimes slow? Do you know why? with data dog you will, you can troubleshoot your apps performance with data dogs end to enter tracing and in one click correlate those Python traces with related logs and metrics use their data Tails, flame graphs to identify bottlenecks and latency and that app of yours starts tracking the performance of your app. So the free trial at Python podcast.com slash data dog, if you sign up for a trial and install the agent, data dog, we'll send you a free t shirt to keep you comfortable while you keep track of your apps. And so moving now into your professional engagement with Python, as you mentioned, you worked very closely with the W community for a number of years and were with canonical for quite a while currently your work at LinkedIn as part of their Python tools team and helping to facilitate the use of Python at that company. And so I'm wondering what you have seen as being the main use cases for Python, both at LinkedIn and canonical and just some of the challenges that crop up at some of these larger organizations or more distributed communities.
Barry Warsaw
0:31:52
Yeah, so that's really interesting. So I think, you know, death within Debian and a boon to Python is used all the time. The place you know, launch pad was our big code hosting and Ubuntu development portal, that's all Python, or at least it was until, you know, while I was there, at LinkedIn, we use a ton of Python, mostly in the back end in the tools and foundation areas. So if you walk up to linkedin.com on your browser, or you know, the iOS devices, or the Android devices, you know, a lot of that's going to be, you know, mobile technologies, you're going to be talking to Java, back ends, but tons and tons of stuff on the internal operational side of LinkedIn is written in Python. And it's just a great language. I mean, you know, you look at look at like Instagram or so many other big companies, you know, they the one of the reasons why Python is so pervasive, I think, is because it's very easy to bring a diverse experience of developers. So almost any team is going to have, you know, senior people and more junior people. You're going to have luck. churn, right? So you're gonna have new people coming into a code base and you want to get them ramped up to speed quickly. And Python just like ticks, all those boxes, you know, it just really makes it easy to maintain relatively. And so I think that's why you've seen, you know, certainly why LinkedIn, for example, has adopted Python. So pervasively, we have some fairly big systems on the internal side that that are built in Python. And I think we're pretty happy about that. Java, of course, is the other big language that we use lots of command line scripts, and libraries and tools are written in Python. So Python is a pretty important language at LinkedIn. And, and I think that's true in a lot of big organizations. Surprisingly, one of the developments that I think makes that feasible is type annotations. And and I've been really that's another feature that I was like, really against when it got added because I didn't want Python to turn into Java and I there was a sprint no Instagram, I think or Facebook, I don't remember exactly the details a couple of years ago. And, you know, the my PI team had come to us and said, the reason why we're investing so heavily in my PI and type annotations is because we can bring in a new Junior programmer into a team. And those type hints make it so that they can be productive much more quickly. And so, in my own code, I started to say, well, let's adopt type hinting and see what it does to the code base. And it just, again, it's just the right mix of not too much bureaucracy, but just enough to make it so much more productive. It's the right balance, you know, and so like now, even in my own code, I'm going through some of my open source libraries, and I'm type annotating everything and I'm, you know, I'm feeling great about it, you know, adding my PI checkers and it just gives you a better sense of confidence and, and self documentation. So that's another feature that I was against, and now I'm totally on board with you know, so those cut, those kinds of things really make it much more palatable in a corporate environment.
Tobias Macey
0:35:07
Yeah, I've had the opportunity recently to start a couple of Greenfield projects in Python and have been using the type hints and annotations, which I hadn't previously really had much experience with. And I'm completely on board with you and the rest of the community that they make everything much easier to reason about where you can be working in one module and have intelligent auto completion as to what it is that you need to have in there and some error checks to keep yourself from doing something stupid without having to flip back and forth between 15 different files to make sure that you have all of the arguments correct.
Barry Warsaw
0:35:38
Yeah, exactly. Exactly. It's, it's, it's, it's a great feature, you know.
Tobias Macey
0:35:44
Yeah. And in terms of some of the Python projects coming out of LinkedIn, there are a couple I saw a while ago that where it's being used for managing your on call rotations and alerting, which is absolutely mission critical. And so the fact that LinkedIn is trusting Python with that definitely is A huge vote of confidence.
Barry Warsaw
0:36:01
Yeah, yeah. And one of our engineers, you know, we at LinkedIn, all of our command line tools are many of our command line tools are written in Python. And so we wanted to have sort of a single file executable format. And we had adopted pecs. I don't know if people are familiar with pecs, but it basically is a way to build a zip file, an executable zip file, but we found that there was a lot of problems with the pecs format, because mostly because it had been around a long time. And it had a lot of backwards compatibility hacks that affected performance and things like that. And so one of our engineers built and from the start open sourced a tool called shiv, which is just all modernized. It's it's the same basic idea but much more modernized. It adopts only Python three, you know, it has all the modern conveniences of Python three, and it's just it's a great tool so that and so we've completely shifted to using that but that was one of the things that we were like, We should open source this and and get you know, give it to the community and engage with them. And that's been super successful. So LinkedIn is, is, is pretty open source friendly.
Tobias Macey
0:37:05
Yeah, ship is another tool that's on my list of things to try out for these projects that I'm working on. Now to simplify the distribution of the projects. Another interesting one that I'm keeping an eye on is pi oxidizer for being able to build standalone binaries.
Barry Warsaw
0:37:18
I'm a big fan of pi oxidizer. I ran an experiment last year to try to convert to using pi oxidizer and I ran into some problems. Some of them were related to LinkedIn, my own, you know weirdnesses because every every corporation has its own weaknesses. And some of them were things that I would have wanted to adopt or change in, in PI oxidizer but but it's a really cool project and I really like rust, you know, and I like that melding of rust and Python quite a bit. So I'm hoping that pi oxidizer, you know, is something that, you know evolves and, and, and can, you know, solve LinkedIn use cases for example,
Tobias Macey
0:37:56
and what are some of the other projects or categories So of work that you're doing to help support the other engineers at LinkedIn and their use of Python and just some of the interesting challenges that you're tackling there.
Barry Warsaw
0:38:09
Ah, so one of the things that I'm my latest kick really is on things like getting widespread adoption of like QA tools is what is sort of how I'm putting putting that together. So these are things like flake eight for doing static analysis. And and this specific set of flake eight rules that we want to adopt to us adopting things like I sort and my pie, you know, pervasively within the within our code bases, I would say black is kind of a controversial tool, which we haven't quite adopted yet. I won't go into a lot of detail about that. But you know, there we have, we have some discussions about about black but that's, that's along the same lines. So I think a lot of these tools are really important and then on top of that, adopting Standard documentation tools. So like, for example, you know, sinks for, you know, and and and we have an internal read the docs implementation. So we want to be able to have all of our documentation easily available, both API documentation and then you know, pros, usage examples, tutorials and things like that. I've recently started playing with Sibyl, which is a an outgrowth of man. Well, and it's the doc test. I'm a big fan of doc tests. And I'm, I've been adopting that I've been using doc test for a long time but like pot pie test in my own open source stuff I have been using knows to but I think pi test, you know, clearly has won the day and it's a great system. So finding ways to integrate document doc tests into the testing stuff.
Tobias Macey
0:39:47
So that's that's really where I'm, I'm sort of focusing my attentions on inside LinkedIn these days is what's the right set of tools that we can deploy and get adopted in a way Widespread way so that we just have greater confidence and discoverability and, and documentation for all of our all of our internal Python code bases. Yeah, those are definitely important endeavors. And it's remarkable the amount of impact that they can have on the overall productivity of a group of developers by just adopting those standards and saying, Here's your boilerplate. Here's all the configuration here the tools that are going to provide guardrails to you so that you can just focus on writing the things that we actually care about, and everything else is taken care of.
Barry Warsaw
0:40:31
Exactly. That's exactly right. It just gives everybody greater confidence about all the code, you know, like coverage is a great example, right? The tools that I am responsible for within LinkedIn, I try to get them to 100% code coverage. And I try to fail the builds if if they don't, you know, if they fall below that
Tobias Macey
0:40:49
now, 100% code coverage, doesn't mean you're not going to have any bugs, but it just increases the confidence level that you have in that code base. So that's another important tool. In the in the tool toolbox, I guess, yeah, it helps prevent projects from descending into legacy where it's the big ball of mud paradigm where you have to keep it running because it's critical to so many things. But nobody knows how it works. And nobody wants to touch it, because they're afraid that it's going to fall over and break. Exactly,
Barry Warsaw
0:41:17
exactly. And it's especially, you know, for us, we got rid of Python to August of last year. So we took about a year to convert all of our Python two stuff to Python three and stop allowing Python to to build. So it took us about a year, maybe six months of planning and six months of execution, which really wasn't that bad. But that kind of like code coverage. Those numbers when you're converting to Python three are critical, right? Because if you don't know what you're actually testing, you don't know if that change is gonna break something or not, especially in a big migration like that. So I'm a big proponent of boosting your your your coverage and just being able to know what your coverage is. Right, like if I walk up to a random codebase, and I say, how much of your code is, is covered by your tests a lot, you know, a lot of code bases don't even know because they don't collect that information. And I think you have to at least know, so that you can then ratchet that up and put and invest in your own. You basically, you're investing in your own future, right? So absolutely.
Tobias Macey
0:42:23
What's the adage that write your code as if somebody in the future is going to read it? Who is a sociopath?
Barry Warsaw
0:42:29
Right, I always say I always say be kind to your future self, you know, exactly. I looked at enough of my code that I've written, you know, years prior in whatever language and I've said, Oh, this is terrible. You know, what was I thinking back then? So, yeah, so be kind to your future self.
Tobias Macey
0:42:47
Absolutely. Yeah. I think we've all had that experience of looking back at their projects and saying, I don't understand why I did it that way. Because that makes absolutely no sense. Well, or how did this ever work?
Barry Warsaw
0:42:58
You're You're so right. And that is another thing that I try to promote is like, what how do you comment code, right? Like, I don't need you to say, oh, on adding one, you know, to this value over here, when I'm reading code, I want to know, why did you pick this algorithm? As opposed to that algorithm? Right? I'll go through that code. And I'll say, Well, I thought about doing it this way. And I tried to do it that way. And that that was worst performance and blah, blah, blah. And I'll put that in a comment so that when somebody comes by in a couple of years, whether that's me or somebody else, and they'll say no, why, why did they do it that way? And not A, B, or C, that comment will explain, you know, the thought process to come up with that particular approach or that particular algorithm and and that's that, again, that's an educational that's an educational role, the proper way to do comments, the proper way to do documentation
Tobias Macey
0:43:49
is all important things for software. It's not just about making the bits fly about, it's making sure that the humans who read it are able to figure out why they're being lazy. Right. Exactly. Outside of Python, what are some of the other languages or communities that you look to for inspiration in your own work and for potential future directions for the language or new capabilities?
Barry Warsaw
0:44:11
I like to think that I'm sort of very on the in the background learning rust, you know, I'm a big fan of rust. I think rust is a very interesting language. You know, CNC plus plus have been around forever. I think my own feeling is, you know, I'm very comfortable in C, I've always loved C and Objective C and c++ and those kind of those kind of languages, but I don't think that they're sustainable enough today. And I think rust can serve the role that CNC plus plus serves, but in a much more structured and safe way. And I love the fact for example, that rust can produce c API ABI compatible binaries, so you can build extension modules in rust and Python doesn't care, you know, right. So that's that's one particular and I think they forgot A sample like cargo and the whole dependency management system of rust is something that we can learn a lot from pythons. dependency management is problematic and in some some cases, but a lot of that is because Python was the Trailblazer, you know, in a lot of ways. And so, you know, we can learn from these newer systems that take a fresh look at some of these problems. So rust is I guess, one community and language that I'm I'm following somewhat, I have a pretty diverse set of interests, like totally outside of software engineering and things like that. So finding time for all the things I want to do what's really difficult.
Tobias Macey
0:45:38
Absolutely, yeah, I know that you are outside of your accomplishments in the software realm. You are also somewhat accomplished bass player. So what are some of the other areas outside of programming that you lean on to help maintain your sanity and sort of free yourself from the drudgery of technology?
Barry Warsaw
0:45:56
Well, music is is the big one. I you know, I'm in a couple of bands. I do have my own home studio which I like to do you know, occasionally come in for me playing music and playing bass is is like you know Chicken Soup for the Soul right like that's for me that's that's what lifts my spirits and and, and that's been difficult during Coronavirus because like all of that public gigs and things like that have just you know dried up overnight so that's that's been tough but I'm doing music as much as I can these days and then the other thing that keeps me sane is I've been practicing Tai Chi for for 20 years and that is a hugely important part of my specially Coronavirus sanity.
Tobias Macey
0:46:45
Yeah, I can see how that would definitely be beneficial. Yeah. And how much of the overall success of Python is due to the Invisible Hand of the Python secret underground, which
0:46:54
emphatically does not exist.
Barry Warsaw
0:46:59
Yeah, that's fun. That's a really funny one.
Tobias Macey
0:47:02
Who's responsible for that meme? Because that's been kicking around for quite a long time. I
Barry Warsaw
0:47:06
think it was me.
0:47:09
blame for that one, too.
Tobias Macey
0:47:10
Fair enough. So are there any other aspects of your work in Python or in technology as a whole or your engagement with the Python community or its impact on your life and career that we didn't discuss the you'd like to cover before we close out the show?
Barry Warsaw
0:47:25
No, I think we've covered a lot. I mean, I just, I'm still as big a fan of Python today as I was 25 years ago, you know, and I just encourage, like, I mentor, a couple of people inside LinkedIn. And I really see my role these days, both within Python and as a experienced, you know, senior developer engineer in much more of that mentoring and educational role than in just like hacking code, right. And I really want to encourage people, especially young people who look at Python, and say that This is a cool language. This is a cool community. And I want to get involved too. please reach out to the steering council to there's a mentorship list. Don't feel like you can't be a core developer. If you love Python, you can be a core developer don't feel like you have to be a C programmer, to be a core developer don't feel like you have to write tons and tons of Python to be a core developer. There's so many opportunities to engage with the community side through the PSF or the technical side. And I just really want to encourage people to think broadly and have faith and confidence in themselves and that there are people in the more senior positions and leadership positions who will help you in your journey to shape pythons next 25 years
Tobias Macey
0:48:48
and also if you're considering other communities, how many of them have their own friendly language? I'll gladly
Barry Warsaw
0:48:53
right.
Tobias Macey
0:48:57
All right. Well, for anybody who wants to get in touch with And follow along with the work that you're doing. I'll have you add your preferred contact information to the show notes. And so with that, I'll move this into the pics and this week I'm going to choose the series Hannah, which I just watched the second season of and it's just a really well put together show great storyline really good acting, highly recommend that for somebody who's looking for something to watch and be distracted for a few hours. And so with that, I'll pass it to you, Barry, do you have any pics this week?
Barry Warsaw
0:49:22
I have two actually, that I'm really into these days. One is a show on Netflix called midnight Gospels. And it is it hits all my buttons. You know it's trippy and weird and animated and just bizarre but but underneath all of the weird animation, it just gets into some really deep philosophical topics. So I've been loving that one. I've been savoring those episodes. And the other thing I absolutely obsessed with these days I love the show called the expanse which I think was on amazon prime and I think the fifth season is coming out at some point. Probably next year but my brother turned me on to the audible books you know, the the books on they're not on tape anymore but you know the the through audible you know, I listen, I put them on my phone, I listen to them in the car and it's just absolutely fantastic. I'm, I'm like looking for excuses to drive places now so that I can listen to it. Listen to those books. It's really good.
Tobias Macey
0:50:22
Yeah, I've been enjoying the expanse series. I haven't taken the time yet to read or listen to the books. But I'll put that back on my list. So definitely appreciate you taking the time today to join me and share your experiences with the community and with the language. It's always great to talk to you. So thank you again for all of the time and energy that you've put in to help make Python become what it is today. And I hope you enjoy the rest of your day.
Barry Warsaw
0:50:45
Thank you, Tobias. I've really enjoyed this as always, and I hope I get it. We get a chance to see each other again in person someday soon.
Tobias Macey
0:50:53
Absolutely. All right. Thank you.
0:50:58
Thank you for listening. Don't forget Check out our other show the data engineering podcast at data engineering podcast comm for the latest on modern data management, and visit the site at Python podcast calm to subscribe to the show, sign up for the mailing list and read the show notes. And if you've learned something or try it out a project from the show then tell us about it. Email hosts@podcasting.com with your story. To help other people find the show. Please leave a review on iTunes and tell your friends and coworkers
Liked it? Take a second to support Podcast.__init__ on Patreon!