Riding The Rising Tides Of Python - Episode 242

Summary

The past two decades have seen massive growth in the language, community, and ecosystem of Python. The career of Pete Fein has occurred during that same period and his use of the language has paralleled some of the major shifts in focus that have occurred. In this episode he shares his experiences moving from a trader writing scripts, through the rise of the web, to the current renaissance in data. He also discusses how his engagement with the community has evolved, why he hasn’t needed to use any other languages in his career, and what he is keeping an eye on for the future.

Do you want to try out some of the tools and applications that you heard about on Podcast.__init__? Do you have a side project that you want to share with the world? With Linode’s managed Kubernetes platform it’s now even easier to get started with the latest in cloud technologies. With the combined power of the leading container orchestrator and the speed and reliability of Linode’s object storage, node balancers, block storage, and dedicated CPU or GPU instances, you’ve got everything you need to scale up. Go to pythonpodcast.com/linode today and get a $60 credit to launch a new cluster, run a server, upload some data, or… And don’t forget to thank them for being a long time supporter of Podcast.__init__!



Announcements

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • When you’re ready to launch your next app or want to try a project you hear about on the show, you’ll need somewhere to deploy it, so take a look at our friends over at Linode. With 200 Gbit/s private networking, scalable shared block storage, node balancers, and a 40 Gbit/s public network, all controlled by a brand new API you’ve got everything you need to scale up. And for your tasks that need fast computation, such as training machine learning models, they just launched dedicated CPU instances. Go to pythonpodcast.com/linode to get a $20 credit and launch a new server in under a minute. And don’t forget to thank them for their continued support of this show!
  • You listen to this show to learn and stay up to date with the ways that Python is being used, including the latest in machine learning and data analysis. For even more opportunities to meet, listen, and learn from your peers you don’t want to miss out on this year’s conference season. We have partnered with organizations such as O’Reilly Media, Dataversity, Corinium Global Intelligence, Alluxio, and Data Council. Upcoming events include the combined events of the Data Architecture Summit and Graphorum, the Data Orchestration Summit, and Data Council in NYC. Go to pythonpodcast.com/conferences to learn more about these and other events, and take advantage of our partner discounts to save money when you register today.
  • Your host as usual is Tobias Macey and today I’m interviewing Pete Fein about his voyage on the rising tide of Python

Interview

  • Introductions
  • How did you get introduced to Python?
  • I understand that you have used Python exclusively in your professional life. What other languages have you been exposed to and taken inspiration from?
  • What are some of the projects that you have been involved with which you are most proud of?
  • How has the community and your involvement with it changed over the years?
    • In your experience, how has the growth in the size and breadth of the community impacted its accessibility to newcomers?
  • You have been using Python and participating in the community for quite some time now, and there have been significant changes in both within that period. What are some of the most significant technological shifts that you have noticed and been a part of?
    • How have those shifts influenced the direction of your career?
  • As you have moved through the different phases of your career with different areas of focus, what are some of the aspects of the work which have remained constant?
    • What have been the biggest differences across the different problem domains?
  • What are some of the aspects of the language or its ecosystem which you feel are lacking or don’t get enough attention?
  • What are some of the industry trends which you are keeping a close eye on and how do you anticipate them influencing the direction of the community and your career in the upcoming years?

Keep In Touch

Picks

Closing Announcements

  • Thank you for listening! Don’t forget to check out our other show, the Data Engineering Podcast for the latest on modern data management.
  • Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
  • If you’ve learned something or tried out a project from the show then tell us about it! Email [email protected]) with your story.
  • To help other people find the show please leave a review on iTunes and tell your friends and co-workers
  • Join the community in the new Zulip chat workspace at pythonpodcast.com/chat

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Click here to read the raw transcript...
Tobias Macey
0:00:12
Hello, and welcome to podcast, the podcast about Python and the people who make it great. When you're ready to launch your next app or want to try a project you hear about on the show, you'll need somewhere to deploy it. So take a look at our friends over at linode. With 200 gigabit private networking, scalable shared block storage, node balancers, and a 40 gigabit public network all controlled by a brand new API, you've got everything you need to scale up. And fear tasks that need fast computation such as training machine learning models, they just launched dedicated CPU instances. They also have a new object storage service to make storing data for your apps even easier. Go to Python podcast.com slash linode. That's l i n o d today to get a $20 credit and launch a new server and under a minute, and don't forget to thank them for their content. supportive this show. You listen to this show to learn and stay up to date with the ways that Python is being used, including the latest and machine learning and data analysis. For even more opportunities to meet, listen and learn from your peers you don't want to miss out on this year's conference season. We have partnered with organizations such as O'Reilly Media chronium Global intelligence, od sc and data Council. Upcoming events include the software architecture conference, the strata data conference, and pi con us. Go to Python podcasts comm slash conferences to learn more about these and other events and take advantage of our partner discounts to save money when you register today, your host as usual as Tobias Macey, and today I'm interviewing Pete Fein about his voyage on the rising tide of Python. So Pete, can you start by introducing yourself?
Pete Fein
0:01:47
Sure. Thank you for having me. My name is Pete Fein. I'm a software engineer and consultant work exclusively in Python for almost
Tobias Macey
0:01:55
close to 20 years. And do you remember how you first got introduced to Python?
Pete Fein
0:01:59
Yeah, I I first got introduced to Python 2.1. Around 2001, I was working in the trading industry in Chicago and a co worker put me on to it for doing some interest rate modeling. And shortly thereafter, I started a startup as a CTO as a job search engine, to kind of really, really cut my teeth on language at that
Tobias Macey
0:02:19
point. And my understanding is that you have used Python exclusively in your professional life. And I'm curious if there are any other programming languages that you have been exposed to or experimented with and taken any inspiration from as along your journey of working in Python or just some of your views on the rest of the programming community as somebody who has been using Python professionally for so long?
Pete Fein
0:02:43
Sure. I'm somewhat unusual and being this experience and only working in one language and I mean, that literally like Python is the only programming language I know how to code in and I can obviously read other things. But the advantage of being a language monologue is it's allowed me to be an architectural polyglot I've worked on a whole range of different different back end systems front end systems. And by focusing just in Python Islamic kind of to expand in other areas. In terms of other influences. I studied computer science as an undergraduate and the I did a little bit in high school, but the first class I did in the fall of 1996, was on scheme list dialect using the structure interpretation of computer programs, that kind of famous book called The Wizard book. And despite having done basically no Lisp, since then, that has more influence on how I think about programming and how I structure my code. And I should should amend that a little bit to say that a year ago, I retook this class with David Beasley who's been a friend, I've done a better half dozen classes with him in Chicago, where what I had done in, you know, 13 or 14 weeks semester when I was 18 years old and kicked my butt we did in Fort Hoboken for days. And was certainly certainly experience and kind of really, really interesting to go back to it. But yeah, and then I, you know, I've drawn on sort of other language paradigms. As you know, things have gotten popular early on was kind of intellectual for a little bit and thinking about some things and you know, async JavaScript, but really, it's just been it's been Python via Dave easily and wizard book.
Tobias Macey
0:04:22
And so you mentioned that at the beginning of your career in Python, you were working in training, you worked as a CTO, working in the web space. And my understanding is that now a lot of your consultancy is focused on the data engineering and data science area. So I'm wondering if you can just sort of give a brief overview of sort of the shape of your career and some of the main sort of verticals and problem domains that you've worked in?
Pete Fein
0:04:46
Oh my gosh, sure. So like I said, I started out in trading, doing interest rate modeling, like trying to figure out you know, how to price bonds, which is where, where I learned Python, and then spent about six years as the CTO of a job search. Vertical, Vertical search startup kind of similar to indeed or simply hired people know those. And I feel like in some ways I have been repeating the lessons I learned during that time, over and over again, in whole or in part, both in terms of web development, which I did for a number of years, Django flask, a bunch of other frameworks. And then over the last five or six years moving into more intensively into data engineering, really big data, supporting data scientists, data analysts, and sort of bringing a software engineering approach to data.
Tobias Macey
0:05:33
And along that journey, what are some of the projects that you have been involved with what you're particularly proud of or that you think were sort of milestones along your career?
Pete Fein
0:05:42
Sure. Certainly that startup and the fact that you know, I got really burned out towards the end and kind of exercise my options I got all my life and I'm still friends with the founder, which feels like as much as accomplishment as anything since then I on the open source side, I've done a number of projects the kind of most successful was a library logging library called Twiggy, which has kind of an interesting backstory. I was at pi con in 2009 and was hanging out on the the chat group and complaining about standard live blogging is a lot of people do. And Jesse know where the conference chair was like, you should write a new one. And I'm like, okay, I just, here's this guy. He's giving all of his time and energy to make this conference happen. And he's telling me that the most useful thing I can do right now is to write a new logging library. So maybe I should listen to him. And once I got into it, it turned out to actually be a really interesting problem domain I really enjoy kind of like system level utilities. I joke that the kind of like code I like to write is, you know, you say if name equals main, and then import Pete's code, kind of things that cut across a whole range of different different problems faces. And so it was fun to take a look at that problem space afresh. I mean, everything up to that point had been based on lock for j, the Java logging library, and we had learned a lot about how to structure applications The intervening 15 or 16 years since blog project came out, and yeah, that library got some use it sort of an interesting an interesting experience for me as an open source maintainer, I really, I really try to write code that works perfectly. You know, it does, it does, it does what it says on the box, it has like 95% test coverage, and you know, just does it does it totally perfectly, because they're just both as an open source maintainer. And then in my consulting business, there's no, there's like no profit and support. It's not fun. It's not a good way to make money. And so the problem with that libraries is I didn't know anybody was using it, it turned out there were like three or four third party extensions people had written and put up on pi pi. And nobody, nobody got in touch with me. It got packaged by Debian and Red Hat. And I didn't know until I was like, randomly googling the project and found it in their repos. And I think I got maybe six bug reports in 10 years. And I know maybe I should have left some more bugs in because if I had had no people were weren't using it actively. I would have probably worked kind of more, more than I did. And so yes, so out there. So people, some people are still using it. All right talk to the author of struct log, which is a add on for for standard of blogging and said he said he had kind of been influenced by the work I had done on that. And that felt like as much as an accomplishment as anything. I mean, that's that's success in my terms is I was really just hoping to reopen, reopen the space and kind of try some new ideas out. So I'm glad some people, people were on with them.
Tobias Macey
0:08:23
Yeah. And there have been a number of different points where people have tried to reinvent logging for Python. And there's some interesting to see the different ways that it gets approached. So there's the work you've done there struct log, as you mentioned, a while ago, I had it Mr. 200 trailering. On to talk about Elliot, which is a different approach. There's the log book project for Environment rocker. So there are a lot of different ways that people have tried to tackle the same problem. But one of the limiting things is that for the most part, you do at some point need to be able to be at least one way compatible with the built in logging library because you need to be able to consume both, you know, consume propagate the log messages from your third party dependencies as well.
Pete Fein
0:09:03
Yeah. And I think that's exactly the challenge is just one of these system wide cross cutting concerns that everybody has to use the same thing. It's like you can't have your own import system. Right? It's just every, you have to support basically, the whole universe of, you know, third party and extension packages and, and have some kind of compatibility or monkey patching. I think that that has really kind of kept us locked in on standard, but blogging, which for its time was kind of quite powerful. But it's sort of, I think, starting to show its age, and the performance is not very good. And it's, it's in some ways, I think we're at the point where we've had a lot of ideas and just that somebody or a couple people kind of need to kind of get down and just sort of do the hard, long work of getting something that you know, supports a bunch of these different different idioms, supports all these different libraries and then doing the work of getting that mature, getting community uptake and then working it into the sandwich library. It's not unlike the packaging dilemma of the last Wish I could say it's done. But it doesn't seem to be of the last, I don't know, five or 10 years. Is that right? This is another area where everybody's got to use the same thing. Yeah, we've been fortunate that some people from the PI PA, the Python packages already have really just been dedicated enough to, to kind of push that through because it was really critical. And logging is important, but it just, you know, we can continue to write programs that you started the blog, and it's basically fine. And so yeah, I think it's tough to, to be motivated sometimes to do that over the long haul. And so I have a lot of respect for people who who put that work in,
Tobias Macey
0:10:36
I wish I had more time. And another interesting element of your work on twiki is, as you mentioned, there have only been a small handful of bug reports over the number of years that it's been available. And so it's an interesting perspective on the idea of when software is done, because it if you look at some projects on GitHub, for instance, it's easy to see something that hasn't been updated and A year or two and assume, okay, this project is dead because nobody's maintaining it. But on the other hand, it could just be that the project is done. And there's no need to do any new updates or maintenance because it works as intended. And there's nothing that has come along in the intervening years that has caused it to break. So it still functions as it did at the first time that it was written. So I'm curious if you have any particular thoughts on that?
Pete Fein
0:11:20
Yeah, it's an interesting, it's an interesting question. I run the Python meetup here in Pittsburgh. And about six months ago, we did an event called Pete picks a Python package. And so just trying to teach people that when I go out, and I'm trying to find a third party library to use, what am I looking at? And this is a question, you know, the ecosystem is so large and so complex, and this was one of these issues is, you know, you know, I go I look at the GitHub activity, and is it done? Is it just is it dead are people not using it? And so, you know, being distinguishing between, you know, projects that have open bug reports or open pull requests that are just aging and not getting respond to that project might be dead. But something that just doesn't have bug reports and started Yeah, it might still be still be active, you know, other things I look at our, our their documentation or their tests, I like to look at some code and then just get a sense of kind of the feel of like what people are doing. I tend to do this when I'm doing interviews, either, you know, when I've been hiring for a company, or just I do this sometimes, for clients, as I do technical interviewing is like, I like to see some real code. And my favorite thing to actually look at is utils. py, is that you learn a lot about a project or a person's coding style about what they what they think utility functions are. But yeah, I think that's I think it's a far more effective way of evaluating a technical peer or a project or than whiteboarding or kinda any other silly thing. And it's like what we do all day is write code. And so like, let's, let's see. So
Tobias Macey
0:12:48
let's talk about that. Yeah, the utils.pi is an interesting thing as well, because there are some projects where there's a utils.pi that's dedicated to that project, or you might have an entire module that you pip install for utilities that you cargo cult across all of your different projects. And it's interesting how you figure out what are the dividing lines along bad? And how do you package them so that you're not just bringing in a whole bunch of useless functions that have no bearing on the project at hand.
Pete Fein
0:13:12
Right. You know, I've used the bolt ons project, just several play on daltons. But it's kind of a whole I, that doesn't, you know, three or four dozen kind of common recipes at this point. And so yeah, it's just as the as the community in the ecosystem has grown. And the tooling has gotten better for sharing code. It's just the the amount of work that we have to do redundantly, has has reduced so dramatically. And you know, I think I think it's sometimes useful as particularly for people who are like, coming up in their career kind of earlier in their career to go through the exercise of writing music, or certainly reading how these things are written. But it's just the the amount of high quality code out there that you don't have to go figure out and make the same bugs over and over again, compared to when I started my career. It's just amazing to me Just how much more were able to get done. It's just really it's, I mean, this is this was the point of free software like this is the point of free software and open source is that we can make a collective code Commons that everybody benefits from. And it's amazing to see how how that's actually panned out
Tobias Macey
0:14:16
how well is that out. And that's an interesting thing, too, is that early on in the lifecycle of a language and its community and ecosystem, it's very easy to have a significant impact on a broad range of people who are just discovering the language. So for instance, elixir is a fairly recent language that I've heard a lot of people having the same type of experience of, it's a new language, nobody has written the canonical ORM for it. So I can go ahead and be the one that writes that. Whereas if you're coming into Python, now there are dozens of RM so if you write another one, it's just another LRM. So not as many people are going to see it or be as impacted by it. So I'm curious what your experience has been of somebody who joined the community fairly early, but not at the beginning. And having been engaged in helping to support and grow the community through running these meetups, how the sort of accessibility of the community and the ability to have an impact has changed over the years from when you started to where you are now?
Pete Fein
0:15:14
Sure. I think that's a that's a great question. I got first involved in the Python community and open source of the Chicago Python user group. And I say user group these days, and people look at me funny, but it predated meetup. And when it was basically a self hosted mailing list and conference table, or you know, in a coffee shop, and I remember being about 25, and going to a meetup, and there are 12 or 13 people there. And three of them are the course of version developers, which was the version control system everybody used before get and I'm just like, 25, and I'm like, holy crud, I'm, like, I types of version like 100 times a day and I get to hang out with you folks. And that was just a really just a transformative experience for me, that wait like this. These are just people and they're just around and they're just willing to talk. young kid. And so I've really tried to kind of carry that spirit forward. I both you know, and part of giving talks of icon and sort of being helping involve that. And then when I had the opportunity to take up the the Pittsburgh meetup, which had gone dormant, just be able to pay that back in a broader sense of accessibility, it's just the community gotten larger, we've become much more aware of what our community looks like and what we're doing to make an inclusive or not for all sorts of people. So when I first attended pi con in Dallas in 2007, it was 96% men and as a guy, like it was terrible. This is terrible. This wasn't good for anybody. And it's the strides we've made on that front, you know, when I went went last year, and Cleveland just looked around, I was like, you know, it's like 35% coming around. It's just like people was like, just people was like, wait, this just looks like this. This looks like normal people. And I really appreciate the efforts of Python community has made on that front. You know, this, this has been my home and open source for almost 20 years, and it's sort of spoiled me. I go to other conferences, right. Go to other language communities or whatever everybody is striving to be what Python is in terms of a community. And I really appreciate being able to play a small part in that.
Tobias Macey
0:17:09
And what are some of the other ways that the growth of the community and the way that it has evolved has impacted your career trajectory from working in trading to where you are now working in data?
Pete Fein
0:17:21
Sure. I mean, at a at a technical level, the changes I've seen over this time are just amazing to me. So when I started, there was no cloud. I remember as Amazon released this s3 product, which all you could do with static storage, like there was even like hanging on and I was like, Oh, this is interesting. We can do something with this to what's what's available. Now. There was no web story, when I started. And I remember being at the Chicago group in around 2007, or 2008. And Ruby on Rails was eating our lunch. And these guys come up from Kansas, and it was the first public demo of Django and we're all just like, Oh my gosh, we still get to have careers. This is awesome. You know, as then more recently, the the growth of data as a discipline has been certainly interesting for me. So I've sort of been working in that space since since I was doing the search engine in the early 2000s. And it's just seeing that applied and just all these different domains that we can can make concrete decisions and it's really in the last year or two that I feel like data engineering subspecialty has has kind of come into its own that you know, like I'm not a statistician I'm not a data scientist. I like to joke it's been a really long time since anybody taught me calculus. Oh, I like I kind of understand it but just like don't don't let me near machine learning my like it's not you know, I think and so being able to bring a real like hard engineering focus to people coming from those backgrounds is a sport them in the work they're doing has been really rewarding.
Tobias Macey
0:18:50
I think career wise, the big change over this time has been just the specialization across tech that for what was for me was one job and you know, 2003 is literally five or six different tops down from web developer back end developer, Dave, engineer dev ops. And we used to call that sis admins. And you know, it's just you kind of use, like, like one person would do all of these things. And the ability to focus, I think, as the field has grown, has just kind of generally improve the quality of software products were able to produce. Yeah, I remember probably six or so years ago at this point was when everybody was using the term full stack engineer to me and somebody that had to do everything from writing the JavaScript that ran in your web browser to writing the Django code that powered the Web API to managing the database and figuring out how the overall system architecture looked. And as you said, there's been a lot more specialization as front end engineering, as with JavaScript has become its own separate discipline because that ecosystem has evolved and gained a level of sophistication that it lacked early on, and the complexity of the web environment has grown particularly with trends in DevOps in the movement. Things like microservices or using a lot of third party API's to be able to stitch together an application. And then, as you said, the rise of data as its own discipline, and needing to actually put a lot more rigor and engineering around how you manage the lifecycle of the data and the data flows to ensure that it continues to be available and consistent and have the necessary amount of cleanliness and rigor that is necessary to be able to gain useful analytics from it.
Pete Fein
0:20:26
Yeah, and the data side is still evolving, which keeps things interesting to me. You know, you said that, as communities mature, that there's maybe less opportunity to have an influence I think that's true to a degree but just as these engineering disciplines have gotten more sophisticated, there's an opportunity to both for your own you know, personal personal growth and as well as community development work on those, you know, as disciplines and so like yet, we're still in the early stages of what engineering software engineering means for data. And that's been interesting for me and as a consultant, you kind of I mean, I think is true in any part of the field, but like you kind of have to keep moving. So I 2012 I was doing a lot of Django and I can still do that. But just there's a lot more people who do that now. And it's allowed me to kind of continue to move in new directions. And I think one of the things that's so rewarding about being a software engineer is if you like learning new stuff, if you're the kind of person who's curious and self motivated, it's just such a such a wonderful field to work in is that just there's an infinite number of rabbit holes you can go exploring. And so I think, I don't think that's going to change. I mean, I think as we continue to, to help software eat the world, like we're going to find more and more things that like, even if it's all Python, and like, Okay, great, we've like we've solved Python we haven't solved. But, you know, as we apply that to more and more domains, there's going to continue to be opportunity to grow. I mean, I'm like really curious to see what happens with IoT and like physical devices. I've done done some consulting work on, you know, a whole bunch of self driving car and like robotics startups here in Pittsburgh. And it's been a lot of fun for me to do things that aren't the web. I like the web just fine, but just you know, I spent much of my career Building the web or consuming the web or advertising on the web, to go out and be like, wow, I get to work on things that move around in the real world, or, you know what, just a little bit of project for company making heart monitors. It's like, wow, this is real stuff like this is real stuff. And as we start to couple software with physical technology, the opportunities to I think really make a difference, like really make a difference in meaningful ways in people's lives are just increasing.
Tobias Macey
0:22:24
Yeah, and another area of growth is on the higher level piece of things of making programming easier and more accessible to people who don't actually want to do it as a career and just want to be able to have some influence over the technology that has been permeating their lives, whether it's the smartphone or some of the smart home devices or their vehicles and just being able to have some way of modifying the logic and these devices to suit the specific needs that they have and not just have to take what's handed to them as a free package. Good that they're not able to look under the hood for and moving programming and moving computational thinking and computational accessibility further and further up the ladder.
Pete Fein
0:23:09
Yeah, it's just a really just powerful trend generally, one of the meetups I did here last year, there was a woman who presented on, she works on the artificial pancreas project. And so this is an open source project to make a closed loop, pancreas system, so she has juvenile juvenile diabetes, and the automatic insulin pumps that were on the market are just very primitive, like very, very lagging and stuff like that. So people hack the hack the firmware and wrote, you know, monitors that are much more responsive. And I'm like this, like, this is the promise of free software. This is what open software is about is people like building. Chris. You know, it's just amazing to me, and I think the challenge is the community is just how do we welcome these people in is just More and more people get involved in like people who don't think of themselves primarily as coders, you know, there's a pretty large community here in Pittsburgh of people who do computational art. And they are, they're artists first, and programmers Second, if that's a meaningful thing to say, but you know, programming is their paintbrush. And as more and more people get involved This is how do we continue to spread our culture? And I mean that both in a technical and social sense, you know, basically have people don't you know, people coming in don't know, you know, version control, or testing or kind of all these kind of basic practices that we've developed, and how do we spread that but then also just, you know, how do we set our community values, accessibility, you know, free software.
Tobias Macey
0:24:40
And your note on things like version control and basic engineering best practices is something that is worth talking about, as far as as you have moved through these different phases of your career and worked across these different verticals and problem domains. What are some of the constants that have been helpful throughout and some of the areas that you see as being valuable regardless of where you might end up working within the software realm.
Pete Fein
0:25:05
So obviously, for me, Python, Python, Python has been a constant. I keep getting tempted to learn other languages. And I just kinda, there's this joke in the Python community, that's the second best language for everything. And you know that if you need to make a web page, maybe PHP is better. And if you need to crunch some data, maybe R is better. But if you need to crunch some data and put that on a web page, well, now you're doing two things. And Python Python excels at that. And so out in the real world, like we're often doing more than one thing, it's been been really excellent. And so just, that's been just the big tool in my toolbox for nearly 20 years. What else people it's just as, as I get get a little bit older, just the, you know, software, it's fully humans. And whether that's the clients or you know, your employer, your boss, the business people or it's the other developers is just it's, it's, you know, it's it's surprising is that, you know, I think a lot of people get into technology because there may be a little less social, a little outgoing and then you know, you find That's it really be successful that you have to deal with the humans and that it's really rewarding after a time beyond that, I mean, just, you know, the, I mean, just the availability of what's out there in terms of best practices, like logs or Stack Overflow, which this is a relatively new development,
Tobias Macey
0:26:18
relatively in the course of my career, again, I think sort of reflects that community minus is that we're just yeah, we're trying to make something better. And it's just, that's what's so so inspiring to me about working this areas, it just it feels like an area of the world where we can make some progress. We can make things better in in small ways. And on the other side of things as you have moved across these different problem domains, from web to data and working across the different communities that you've been involved with and your different roles, what have been some of the biggest differences and some of the biggest learning curves as you have moved into these different areas.
Pete Fein
0:26:50
Sure. So one nice things I like about being a consultant is I do get a lot work and a lot of different moving verticals and it's it's fun for me to learn a little bit about people businesses or people's particular problems, kind of keeps keeps things interesting, you know, in a, on a technical side moving from the web to data. Oh my gosh, I feel like I mean, it's taken, I don't know, close to 20 years, but the web is largely solved problem. I mean, not entirely, but just, you know, like database back web sites like Wi Fi, it took 20 years, we finally got it. And less so and some of the other areas. And so in terms of this domain, there's a it's interesting, like a lot of web code or even like kind of systems code, like data, you know, DevOps systems, there's both the right answer in terms of that, you know, you can write a function in your web server, and you can test it and you can like, thoroughly test it, say like, this is like, there's an Oracle, it's like, this is what they should do. And that somehow coupled with a bit of a loosey goosey ness about how that actually performs it's like, well, the web server went down for a minute, okay, whatever, we just, we don't care just doesn't matter, you know? Or, hey, we, you know, we dropped a couple of requests or like, it was just whatever, it's fine. And then on the Data side, it's interesting is there's kind of a flip where a lot of these problems that there is no right answer is just, you know, you crunch some data and just there, the results you get is what you get. And you don't know, you know, it's there's not an algorithmic solution. It's not like it's two plus two equals four. It's like, well, the trend line is up into the right. It's like, Okay, how steep has it, we just, you know, it is well, it's just what the date is. And so that that has on that side that, you know, correctness is harder to define. And then sometimes on the engineering side, it's more important. So like I said, I did a small project for a company that makes heart monitors. And so some of the questions I like to ask when I'm doing data projects is like, Are we allowed to drop date on the floor? Sometimes it's just a, an engineering compromise that makes things easier. And so you go to somebody who's doing web analytics, and I'm like, yeah, we missed a few clicks, whatever. And then he talked somebody who's monitoring a patient's heart, they're like, we can lose no data at all. And that contrast is interesting to me between stuff on the web where like, you can in theory, get it right but you don't have to like totally right but you don't have to and then on the day, Decide where there is no, definitely right answer, but it's got to work perfectly. And so how do you, I don't know, we're still figuring it out. And so that's what that's what keeps it interesting for me is like I like, I like being able to go in and help people solve their problem and build them a platform that they can kind of continue to grow their business and iterate on and I kind of go, go do it for somebody else's said, you know, over the course of like, long experience, and I don't know if I'm a better at anybody else than this. I've just been doing it a long time. And I've made made the mistakes, I think, and to kind of spread that knowledge of, you know, around and kind of apply to a bunch of different situations has been interesting for me.
Tobias Macey
0:29:34
And going back to Python itself. What are some of the aspects of the language or its overall ecosystem or the community which you feel are either lacking or they don't receive enough attention and that we should be spending more focus on and more energy on either improving or filling any technical gaps or anything along those lines?
Pete Fein
0:29:56
Sure. So it's interesting to me the growth of the language over the last, let's say, five years, the language itself, I think has become almost non unmanageable. I mean, because clearly the, you know, the core Python team is continue to put out releases. But as a user of the language, I just, I can't keep up anymore. So I knew Python to seven, inside and out, like, inside and out. And with the transition to three, I didn't really start using it on projects till maybe like three, five. And by that point, there was so much new stuff in there that I feel like I had kind of lost touch a little bit with all the new features. And just as the community of committers to Python itself has gotten larger, I feel that like problems accelerated and it's I've talked to other people, you know, who've been around for as long as I haven't, they're kind of in a similar boat is just were like, wait, what's going on? Or like, you know, and as particular as they move to faster release cycles, you know, it's like, wait, there's all this crazy new stuff and typing. And, you know, I just, I was just looking at the release notes for the upcoming three, eight release. And multiprocessing is now going to support shared memory communication between processes in addition to kind of just piping and copying data back forth. This is a game changer, like that's a huge technical innovation. And it's got like, you know, a one sentence bullet point in the change lock. And so how we get better at keeping the user community moving along with advances in the language itself, I think is still a challenge. As as the pace pace of innovation and increases I don't have a good answer for that. The other thing in terms of as a community icon this past year, I heard there was a lot of reading about the challenges of sustainability and open source and just that, you know, maintainers are getting burned out and there's no good funding model and I'm just I'm not entirely sure what problem funding will solve to be honest and I'm kind of usual that that you know, somebody's getting I mean, I guess everybody likes money but I just you know, somebody getting a couple hundred dollars a year is not going to make the difference between are they able to support support doing a project or not so like, what is what is sustainability look like in this space? And honestly, the biggest thing challenge I heard of its that is about project and product management. And so as a community We've gotten really good at onboarding new developers and bringing new people in as contributors. And it's like a lot of best practice that people put together. we've all gotten really good at writing documentation. And I think Eric culture and the folks at read the docs done a great job just showing us important to us and giving us a platform we can do on that. And I think the next big challenge is, is project management is you know, I talked to a bunch of people are maintainers popular projects, and they're like, I spend all my time doing ticket triage, you know, we get we get like 800 bug reports a year I gotta go figure out which of these actually need to be addressed. And then like, manage manage them through a workflow reproducing cases. And that's something I don't think we know how to do ourselves. And we certainly don't know how to teach it. And I think that's kind of coupled with the people want to work on their own projects. I mean, I I'm guilty of this as much as anybody else. And I think this is a broader cultural thing is that you get credit or you get a promotion or like, you know, people want to see their name on the web. And you know, you don't get that for being just having your name in the changes file. You get that for being you know, like the inventor of Whatever, you know, like the venture date, utils, or whatever it is, you know, like these major libraries that people use. And I don't know, I think we, as consequence are reinventing the wheel a lot in the open source space. And then, you know, the people who do sort of accidentally wind up with their hobby project becoming critical infrastructure just overwhelmed. And I don't know how I mean, like I said, I probably could personally do a better job of supporting other people's projects instead of working on mine. And that just I think that's something maybe we all need to
Tobias Macey
0:33:28
think about. And then as somebody who has had these different shifts in your career, going from trading to the web, to data engineering, and being involved in the community and follow along and these different industry trends, what are some of the currently evolving trends or any that you see as being upcoming, which you're keeping a close eye on? And how do you anticipate them influencing the direction that you take in your career and the direction that the broader community might trend towards in the coming years?
Pete Fein
0:33:57
Sure. So definitely, like we talked a little bit about Python for everyone that that more and more people who might have been spreadsheet jockeys are now learning to program or people who did data entry are now learning to automate parts of their job. And so I'm looking forward this year to doing some training some classes here in Pittsburgh. I'm just kind of like not teaching people to program but for people who have a little bit of basic experience maybe did an online class and we're looking to level up their game or for people who know pandas but not Python, I feel like there's a lot of people who are coming from are gone through a data science boot camp can use some of these science libraries. But just you know, in terms of structuring code classes, how the system works under the hood, data cleanup and processing I'm looking to you know, this is as both a career thing and just kind of spread my knowledge to do some teaching on that. I'm also this year going to be writing a book called principles of data engineering with that some Wesley about covering that just not specific technologies, but data software engineering practices, the whole so those I think are going to be big trends. I mean, just the continued growth of data is is astounding to me is that I mean it just every sales organization on earth is going to turn into a data organization. It's just that this is going to allow us to answer questions more specifically, and kind of like, as opposed to just kind of sticking our finger in the wind and being like, ah, I think we should go to New York. It's like, okay, like, Are people buying New York, I think it's going to be a really big trend. And then, like I said, the IoT space like physical computing is going to continue to really develop and that's interesting super interesting to me the one of the handouts that at pi con in the swag bag this past year was a circuit Python, which a microcontroller made by Adafruit that you can now run Python on a everything from like supercomputers to these huge, you know, thousand node clusters down to $5 chip with 256 K of RAM or whatever. And it's the same language is super cool. It's super cool. And just yeah, I you know, this is this saying from you know, the early 2000s that software is going to eat the world. I think that's largely played out is that you know, software has has kind of invaded all of these different spaces. And from where I sit, it feels like pythons about the software is that if you're a college student today that five years from now, if you're going out and starting a project, whether that's a startup or like a hobby project or art project, or whatever odds are, you're probably your first who you're going to reach for is going to be Python. And that's, that's really exciting for me as somebody who's been involved in this community for so long is that it's gonna make a lot of things possible outside of that space. You know, I'm always kind of keeping an eye out for new languages and new technologies. Rust looks really interesting to me as a programming language. I know there's a bunch of kind of experienced Python folks I've been working with are like, really, really getting into it. Yeah.
Tobias Macey
0:36:40
Yeah, we'll see. And since you are living in Pittsburgh and engaged with the community there, and that's where pi con is going to be for the upcoming two events. What are some of the things that people who are attending the conference should know about the area?
Pete Fein
0:36:55
It's great. We just had a meetup this past week, with The event coordinator, the like icon planner from the Python Software Foundation, and Emily Morehouse who's conference chair coming up for the next two years where they were going to be in town to go visit the convention center. And they're like, Hey, we want to meet with the community. When is your meetup and I was like, you just tell me when you're coming. And that's meetup. And so it was great. We had maybe about 30 people come out, maybe we can five a month into a previous event. And so they just people were interested in attending volunteering, and just we're really hoping to be good hosts. And you know, I moved I moved here from Chicago about five years ago. And it's been a wonderful place to live. But it's not obvious in some ways is, you know, like three years ago when it was important, like Portland is just an easy place to go have a conference and it's just there's food trucks, there's breweries, it's just like super even Pittsburgh. Pittsburgh is a wonderful place, but it's a little a little subtle. And so I'm interested in trying to help me people's human needs as conference attendees, just to recognize that we're more than just butts and seats with a laptop and so We are looking to do everything from Yeah, provide guidance on like local restaurants, bars, coffee shops and stuff like that. There's a fantastic fine art scene here, the number of like World Class museums within walking distance of the convention center, including a seven story museum just for Andy Warhol. It's like a five or six minute walk away, kind of giving people advice on transit. I know like when I go to travel conferences, I always try to make sure to get out of the conference hotel or like the conference zone is because like, what else you're going to be Atlanta, what else you're going to be in Dallas or whatever. And yeah, and you know, beyond that, in terms of conference content itself, for people who haven't been it's really a wonderful event, it will be about 3500 people event is capped at that size, and then they could sell more tickets, but they're kind of deliberately keeping it small. There's like two days of tutorials beforehand, three days of conference proper, including, I think it's five or six parallel tracks, hooting with a cold pack on Charlotte. Charlotte says the Spanish language is Spanish for conference talk and so it is a full full track of talks. In Spanish, just like I can't believe it says, it's just amazing to me. And then you know afterwards, there's four or five days of sprints where all these people who are in town for this event and work on open source projects get together and hack on things for a couple days. So if you want to make commit, you know, if you want to sit with the core committers of Django and like work on some Django patches, you want to work on C, Python itself, like it's an opportunity to do that. And then beyond that, there are what are called hatchery events. And so icons like a very progressive conference in terms of just being open to all the kind of weird stuff, interesting stuff people people want to do. And so last year, there was a traditional Performing Arts about the experience of programming. So play spoken word, I had a big part in one of the plays playing the role of a web API, which was hilarious. And so yeah, they've now formalize the process for doing these interesting things. And it's called the hatchery program. And so I know some of the things they're getting proposed is me and some friends are proposing a gallery style installation on computational art. There's talks about a data science specific track, and we'll see what else comes out that those proposals are due in a couple weeks. The CFP for conference talk, like conference talks proper, I think closes in just a few days. So you should get on it, if you haven't, and then if you can't make this conference, there's a whole bunch of regional events. So I attended both pi, Ohio, Columbus and pi Gotham in New York. And they're like little mini icons that are also kind of really great, a great way to proceed. I always like I'm always looking forward to the hallway track. And so that is that is an event in and of itself of just people. There's lots of space to sit and drink coffee and just meet people and kind of find out of an interesting stuff. That's really kind of a core part of the conference for
Tobias Macey
0:40:36
me, are there any other aspects of your experience of working in Python and with the community that we didn't discuss yet that you'd like to cover before we close out the show?
Pete Fein
0:40:45
I think one of the things is you know, people people will come back to me and be like, Why Why just Python like white like, like, Why only language and why just Python and so for me, some of that was just being I think, in the right place at the right time, but it just I love that language for its simplicity. It's just you know, this is runnable pseudocode. If you when you know you and I were to get out a whiteboard and just rough out an algorithm that's going to look like what runnable Python code is pretty close. And so I joked and I'm not smart enough for other things. Like, I'm not smart, I'm not smart enough to do job, I'm not smart enough to do Scala, I'm definitely not smart enough to do Haskell. And that the problems I'm trying to solve are pretty difficult. And having a language that's easy to work with and fits my brain makes doing those harder things possible. And I think that's part of what contributed to it just general growth, particularly on people who are coming into later in their careers who don't have formal training is that it's just it's easy. It's an easy language to get started in. And then the community is great. And I mean that both in terms of the technical ecosystem, like the number and quality of packages is incredible to me. And then and then the people is just I just continue to meet wonderful people and people who just kind of really care about what they do, and that's that's really inspiring to be around
Tobias Macey
0:41:56
for anybody who wants to get in touch with you and follow along with the work. You're doing or hire you from your consultancy, I'll have you add your preferred contact information to the show notes. And with that, I'll move us into the pics and this week I'm going to choose the matumbo analytics platform. It's an open source and self postable alternative to Google Analytics. I've been using it for the podcasts for a while now. And it does a pretty good job of being able to track the data and have some usable reports. So for anybody who's looking for something to be able to track traffic on their applications, it's definitely worth a look. And so with that, I'll pass it to you Pete, what do you have for pics this week?
Pete Fein
0:42:33
my pic this week with the fast API, which I have been using a couple of projects recently. It's a flask style RESTful API and just really, really a pleasure to work with. It's backed by a data layer called identic, which makes modeling requests and responses in different formats and packing it on packing Jason are really trivial and so it's it's easy to say hey, I want to switch this from, you know, passing arguments and query parameters to pass I'm in the body of the function and it has kind of nice integration. So no RRM. But you know, you come back with any day later. It's got like nice integration with SQL alchemy, and it's both very fast has a name, I'd expect worked with async. io, and just really been a pleasure to work with. And it's just kind of a nice to just bang these things out. So I'd recommend people
Tobias Macey
0:43:15
check it out. All right, well, thank you very much for taking the time today and sharing your experiences working in Python and the trajectory that it has taken for your career. So I appreciate all of your time and effort on that front and I hope you enjoy the rest of your day.
Pete Fein
0:43:29
Thank you so much for having me. It's been a pleasure.
Tobias Macey
0:43:34
Thank you for listening. Don't forget to check out our other show the data engineering podcast at data engineering podcast com for the latest on modern data management. And visit the site at Python podcasts. com to subscribe to the show, sign up for the mailing list and read the show notes. And if you've learned something or tried out a project from the show, then tell us about it. Email host at podcast and a.com with your story. To help other people find the show. Please leave a review on iTunes and Tell your friends and coworkers
Liked it? Take a second to support Podcast.__init__ on Patreon!