Learning To Program By Building Tiny Python Projects - Episode 273

Summary

One of the best methods for learning programming is to just build a project and see how things work first-hand. With that in mind, Ken Youens-Clark wrote a whole book of Tiny Python Projects that you can use to get started on your journey. In this episode he shares his inspiration for the book, his thoughts on the benefits of teaching testing principles and the use of linting and formatting tools, as well as the benefits of trying variations on a working program to see how it behaves. This was a great conversation about useful strategies for supporting new programmers in their efforts to learn a valuable skill.

Do you want to try out some of the tools and applications that you heard about on Podcast.__init__? Do you have a side project that you want to share with the world? With Linode’s managed Kubernetes platform it’s now even easier to get started with the latest in cloud technologies. With the combined power of the leading container orchestrator and the speed and reliability of Linode’s object storage, node balancers, block storage, and dedicated CPU or GPU instances, you’ve got everything you need to scale up. Go to pythonpodcast.com/linode today and get a $60 credit to launch a new cluster, run a server, upload some data, or… And don’t forget to thank them for being a long time supporter of Podcast.__init__!


Datadog is a powerful, easy to use service for gaining comprehensive visibility into the state of your applications.

The easy to install Python agent lets you collect system metrics and log data, supports integrations with all of your services, and distributed tracing.

Their customizable dashboards and interactive graphs make finding and fixing performance issues fast and easy, and their machine-learning driven alerting ensures that you always know what is happening in your systems.

If you need even more detail about how your application is functioning you can track custom metrics, and their Application Performance Monitoring (APM) tools let you track the flow of requests through your stack.

Start tracking the performance of your apps with a free trial at pythonpodcast.com/datadog. If you sign up for a trial and install the agent, Datadog will send you a free t-shirt.

 



Announcements

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • When you’re ready to launch your next app or want to try a project you hear about on the show, you’ll need somewhere to deploy it, so take a look at our friends over at Linode. With the launch of their managed Kubernetes platform it’s easy to get started with the next generation of deployment and scaling, powered by the battle tested Linode platform, including simple pricing, node balancers, 40Gbit networking, dedicated CPU and GPU instances, and worldwide data centers. Go to pythonpodcast.com/linode and get a $60 credit to try out a Kubernetes cluster of your own. And don’t forget to thank them for their continued support of this show!
  • This portion of Python Podcast is brought to you by Datadog. Do you have an app in production that is slower than you like? Is its performance all over the place (sometimes fast, sometimes slow)? Do you know why? With Datadog, you will. You can troubleshoot your app’s performance with Datadog’s end-to-end tracing and in one click correlate those Python traces with related logs and metrics. Use their detailed flame graphs to identify bottlenecks and latency in that app of yours. Start tracking the performance of your apps with a free trial at datadog.com/pythonpodcast. If you sign up for a trial and install the agent, Datadog will send you a free t-shirt.
  • You listen to this show to learn and stay up to date with the ways that Python is being used, including the latest in machine learning and data analysis. For more opportunities to stay up to date, gain new skills, and learn from your peers there are a growing number of virtual events that you can attend from the comfort and safety of your home. Go to pythonpodcast.com/conferences to check out the upcoming events being offered by our partners and get registered today!
  • Your host as usual is Tobias Macey and today I’m interviewing Ken Youens-Clark about his book Tiny Python Projects

Interview

  • Introductions
  • How did you get introduced to Python?
  • What is your goal with your book of Tiny Python Projects?
    • What motivated you to start writing it?
  • Who is the target audience that you wrote the book for?
  • One of the notable aspects of the book is the fact that you introduce linting and testing in the first chapter. Why is that a useful subject for the first steps of someone getting started in Python?
    • What are some of the problems that users experience if they are introduced to these tools after they have already established a set of habits?
  • How did you approach the structure of the book to be approachable by newcomers to Python?
  • What was your process for deciding on the scope of the information to include in the book?
  • What are some of the challenges that you faced in identifying self-contained projects that could fit into a single chapter?
  • As a book that is intended to serve as a learning resource, what was your process for soliciting feedback to determine if your tone and structure is effective in teaching the reader?
  • What elements of the Python language and ecosystem did you consciously leave out to avoid overwhelming the readers?
  • What are some of the most interesting, unexpected, or challenging lessons that you learned while working on the book?
  • What are your thoughts on useful resources and next steps for readers who are interested in progressing in their use of Python?

Keep In Touch

Picks

Closing Announcements

  • Thank you for listening! Don’t forget to check out our other show, the Data Engineering Podcast for the latest on modern data management.
  • Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
  • If you’ve learned something or tried out a project from the show then tell us about it! Email hosts@podcastinit.com) with your story.
  • To help other people find the show please leave a review on iTunes and tell your friends and co-workers
  • Join the community in the new Zulip chat workspace at pythonpodcast.com/chat

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Click here to read the raw transcript...
Tobias Macey
0:00:13
Hello, and welcome to podcast dotnet, the podcast about Python and the people who make it great. When you're ready to launch your next app or want to try out a project to hear about on the show, you'll need somewhere to deploy it. So take a look at our friends over at linode. With the launch of their managed Kubernetes platform, it's easy to get started with the next generation of deployment and scaling powered by the battle tested linode platform including simple pricing, load balancers, 40 gigabit networking dedicated CPU and GPU instances and worldwide data centers. Go to Python podcast.com slash linode. Today, that's Li n od E and get a $60 credit to try out a Kubernetes cluster of your own. And don't forget to thank them for their continued support of this show. You listen to this show to learn and stay up to date with the ways that Python is being used including the latest in machine learning and data analysis. For more opportunities to stay up to date, gain new skills and learn from your peers. There are a growing number of virtual events that you can attend from the comfort and safety of your own home. Go to Python podcast.com slash conferences to check out the upcoming events being offered by our partners and get registered today. Your host, as usual is Tobias Macey. And today I'm interviewing Ken Youens-Clark about his book tiny Python projects. So Ken, can you start by introducing yourself?
Ken Youens-Clark
0:01:24
Oh, hi. Thanks for having me on. My name is Ken Youens-Clark and right now I'm working as a senior scientific programmer at the University of Arizona and I work have been working for the last 20 years or so in a field of computer science and biology gets mesh mashed up into something that we call bioinformatics. I think one of the things that maybe sets me apart a little bit is that I have had no formal training in computer science or biology. I studied English Lit and music and in my undergrad so many many years ago and then taught myself programming and then taught myself biology and And that's where I got where I am right now,
Tobias Macey
0:02:02
it's always interesting when people are able to find their way into the field without having any official academic training. And it just goes to show that while those are valuable, they're not necessarily the be all end all or a determining factor in terms of your available skills.
Ken Youens-Clark
0:02:19
Yeah, I think that
0:02:21
what what is definitely necessary in this field and will definitely continue to be is just the ability to learn all the time to teach yourself to find materials that will help you progress because, you know, even though this this books about Python, and Python is extremely popular, you know, in 10 years, 20 years, what's going to have
Tobias Macey
0:02:40
supplanted it. And do you remember how you first got introduced to Python, I started
Ken Youens-Clark
0:02:44
off programming in a number of different languages and then really fell into Perl in the late 90s. I really wanted to get into web development, and then I fell. Perl actually was my gateway drug, if you will into bioinformatics. Especially it In the early in the late 90s, and early 2000s, Perl was just really keen at text processing, which is a lot of what genomics and web processing and web web programming was to. And, and so it was actually a few years ago, in helping my boss, Bonnie hurvitz. At the University of Arizona, we we were teaching courses to teach beginning programming skills to biologists. And we were teaching Perl because that's what we had were really expert in and what had been the predominant language in that field. But it really just was just becoming more and more apparent that Python had really won the the space, especially when it came to a lot of scientific programming. And so I really pushed push Bonnie to to change the course over into Python and I decided I should just become, I should just basically abandon all my Perl programming and just become a Python programmers so that I could really understand the language well enough to teach it. And so I'd say it was about three years ago. was just a conscious decision because of the prevalence of Python in scientific programming to just simply go. And really, I found, you know, there's so many similarities between Perl and Python, I did not find the move to be really at all difficult.
Tobias Macey
0:04:15
And I know that in Python, there are actually a few different libraries that are being used heavily in bioinformatics by a Python being the one that comes to mind first, and then there's also a new language that's inspired by Python in the form of seek that is targeting that same area. I'm curious what your experience has been in terms of working within that ecosystem and your day to day work.
Ken Youens-Clark
0:04:37
I definitely have found that Python, this like you mentioned bio Python, and and so many other modules, really make programming in genomics and bioinformatics really pretty easy. There's few gotchas that you definitely need to be aware of, you know, as with any any language, Python, especially, you know, with its dynamic typing, and Some of its ideas about kind of variable overloading operator overloading, you really need to be super aware of some of the things that you're doing where you can really mess up your data. But I also was able to while I've been at University of Arizona, I was able to complete a master's degree and and did an introduction to machine learning. And so that's another area where Python has just really excelled in the community with delivering just incredibly useful modules. I think that I have come across that seek language. I found it. I think I looked at it briefly and found it somewhat interesting, but I'm not not necessarily i'm not i'm not convinced by it yet.
Tobias Macey
0:05:38
Yeah, it's one of the challenges of any new entry to the either languages or libraries where it takes a little while to get to a point of maturity, where more people are going to adopt it. And there's only a small cadre of people who are going to jump on board early on to test it's metal. Yeah. And so now you have written this book tiny Python. project. So I'm wondering what your motivation was for writing it and your goal for what you're hoping to achieve with the book and the types of lessons that you're trying to provide.
Ken Youens-Clark
0:06:09
Yeah, so um, I would say that the inspiration really will directly came out of the teaching that Bonnie and I were doing at the University of Arizona. So we're trying to take biologists who don't have a formal computer science background, and we're trying to give them in one semester, basic programming skills like I need to, at the end of this course, I need to be able to, you know, process a directory of input files and parse through them and output some result. And so practically speaking, like what can I teach someone in 12 to 14 weeks of class time, at the end of which they'll be a sum, they will have gone from zero to being a somewhat self sufficient programmer, and this was really influenced or buy when I, I got into bioinformatics in 2001 because I got hired by Dr. Lincoln Stein at Cold Spring Harbor laboratory up in Cold Spring Harbor, New York. And he was a big Perl guy. He's got many books to his name. He had modules that he had written that were part of the standard core Perl distribution. I had no idea what bioinformatics was, but I knew that he was a big figure in the in the community. And when I had an opportunity to take a job in his lab, I was like, Sure, I'll do whatever you want to do, you know, because I'm happy to be working in Perl, and happy to be working with this really smart guy. And one of his core programs that he started, he got funding for and he did this for like a dozen years there. But he had started this programming for biologists class where he had people come to campus for like, you know, 14 days. And it was intense. It was like drinking from a firehose, you know, 1214 hours a day, getting these postdocs and graduate students and and sometimes career professionals in a room and just Starting them, here's the command line, here's how you run Perl, here's how you run these basic programs that we use in bioinformatics so that at the end of two weeks, they're kind of sort of on the road to becoming computational biologists and and and so we started off at Arizona, Bonnie and I using those same materials to teach biology. And like I said, after like a year or so of doing that, you're just like, you know, we really should be changing this into Python. And so, so I started having to write my own materials to do that. I think I didn't have to I chose to, I really wanted to, I felt like I had a vision for the way I wanted to teach programming. And over the course of of a few years of working with students, and trying to, you know, Grade A bunch of assignments, I just realized that providing test suites just really made my life so much easier as an instructor. It really helped the students to understand when their programs were working, and then and then for me as the teacher All I had to do was pull their assignments from GitHub and run the test suite. And their grade was simply a percentage of the number of tests they passed. And so I just really started embracing this idea of teaching by using test driven development, there just really wasn't any material out there. And the more I looked, really testing seemed to be this idea that's like, oh, once you get into industry, or once you're a senior level programmer, you'll really understand the the importance of testing your software. And I thought that was just really short sighted, I think we should be teaching testing to the novice to the very beginning programmer. So you know, in my book, in the first chapter, we program hello world, and then we write a test for it. And we say this program, we run it should say hello world, and we test it and you know, and then lets you see if you miss the exclamation point with a comma or you put an extra space in there, you fail the test, you know, it's not good. Close enough is not good enough in programming, it has to be perfect. And especially in bioinformatics, we suffer from
0:10:10
poorly trained people writing, you know, Perl and Python scripts, and barely putting them into some sort of source code repository. And then you go to try to pull their work and reproduce their their work. And it's completely unreal producible because it has all these hard coded paths. It makes all these assumptions about the environment. It hasn't been tested in even the slightest way. And so I think as an industry, we need to rethink where and when we start teaching, testing. And I think it should be taught right at the very front. And so the book is written in a way that there's like 22 chapters, and every single one, I describe a program that you should write, and I say there's a test suite there that you your software should pass and when it passes, you're done. That's it. That's the expectation for that. Hopefully in the you know, as as we go through the material, I start saying, oh, let's look inside this test. Let's see how it works. Let's see what it's doing. And then by the end of the book, I'm asking you to look at the integration test to to write your own unit test. Like if you're going to use this function, here's a unit test. But if you're going to write a different function, make sure you write a unit test. And so I hope by the end of the book, I, you really understand Python deeply because you've used the test, but also I hope that you understand testing itself, which is not limited to Python, would you get that you could take those skills to any and every other language that you work with.
Tobias Macey
0:11:39
And test design is an entire science and art form in and of itself beyond just standard software engineering. And there are a number of software engineering principles that go into it and they also will impact the way that you approach the design of your code in order to make it testable, which I know is one of the main goals of test driven design and cure. As what you have found to be some of the outcomes for the people that you've worked with, with this material of introducing testing as an early concern and some of the ways that that has helped them in their overall growth as software engineers.
Ken Youens-Clark
0:12:15
It's interesting, I think that when you can make something fun, I think it makes it easier to learn. It was interesting to me when I bought a Prius a few years ago. And there's like this dashboard that gives me real time feedback like for how I did on my start, and my coasting in my braking. And every time I do those actions, it gives me immediate feedback, like, Oh, you use too much energy on that start, so that it's basically teaching me how to drive the car, which I thought was really interesting. And it actually becomes a game because it gives you a score at the end. And I'm like, Oh, can I beat my wife score, you know, on how she you know, she got a 97 the last time she went to the to the grocery store. And oddly enough, I think that taking these students and giving them this test Okay, so there's 10 tests that their, their software has to pass. The first of which is did you create a program that was called, you know, food dot p y, whatever it's called. And so you know, the first couple tests are pretty easy to pass, and then it gets a little harder and harder. And and honestly watching them pass one more tests, they get excited, they pass one more test, they get excited, then they finally they pass the whole thing hundred percent. And I literally see, like undergrads and graduate students, like kind of pumped their fists, like Yes, I did. And then it I think, oddly, the test can actually be a really positive reinforcing thing. Instead of, you know, beating you down and say, No, your software doesn't work. It's like, Hey, you just pass it on a test, you're closer to the goal. And so I think, you know, that normally, you know, when you know, as a as a professional, you're not going to have this pre written soft, you know, the software suite that you're going to code against, you're probably going to organically create the code, create the unit test, create is kind of at the same time, I really teach test driven development, though I really think if you're going to write this function, first go write the unit test, like think about what are the data? What's the data you're going to pass in? What do you expect to get back and then go write that function. And but it still it's going to grow over time as you write your program. And I think that, you know, I only have a chance really, to work with kind of near beginners and work with them in their first few months of coding. And so I don't necessarily know if it has this long term effect, that that they will continue to keep writing code. But especially in the sciences, we really prize or we should prize reproducibility. And I think that this really gets at the heart of proving that your software is reproducible. And the other things that I'm teaching there is that the programs in order to be tested, they have to be parameterised. So you know if you're if your program takes an input file it that should not be hard coded, that's a parameter. It's a command line argument so that I can test it with two or three different input files. And, and so and because the way that we use we make our programs parameterised at tiny Python projects is we're using the standard argparse module, then it just automatically generates documentation, and it creates this interface for your program. And I think that people when I've worked with them, I think it seems overwhelming at first like, Oh my god, I just want to print hello world that, you know, I have to add all this other stuff, but it's really not that much to add. And in the first chapter, I try to step you through like, okay, here's how to print HelloWorld. And then but now we want to make it Hello, something, something parameterize something that we could pass in. And here's how we can do that. And I think that you do that the first two or three, maybe four chapters, and it seems kind of challenging. But then after that, I think it starts to Makes sense. And people really come to appreciate that, that this is a pretty sane way to write software. And to make sure that it works.
Tobias Macey
0:16:09
And in terms of the actual test design to I know that there are several different patterns, whether you're using given when then or arrange act assert, and I'm curious what your strategy is in terms of making it easy for your students to be able to understand the test setup and the structuring of the tests when they go to the point where they're writing them on their own, rather than just using the ones that you provided for them.
Ken Youens-Clark
0:16:34
Right. Yeah, this is this is a really great point. And, and so when, you know, I learned how to I really kind of taught myself how to do testing in Perl. And pearl has some really great testing libraries and a testing harness and it works very similar to pi test. And so when I got to Python, and I just looking around you there's unit test and then there's pi test and there's some other frameworks Pi test was the one that was the most immediately recognizable To me, it was very similar to the way perot ran it, pi test, you can just name your your test, like your integration tests. If it's a file that contains tests, you can just name it test, underscore, whatever, and it will find that and then it will go in there and find any function that starts with test underscore and on it for you. And so honestly, I find pi test to be so dead simple and easy to use that I think it takes just a couple of examples of how to run it and how to structure your tests. And then I just inside I just teach using like assert statements, and I really teach a functional programming approach to writing Python. The book does not even attempt to address object oriented programming or testing objects. I stick just with how do you write a function? How do you write a test for that function? And then a little bit later, how do you test it whole program from the outside using an integration test. And that's what I'm providing is I'm providing these integration tests that from the outside, they run the program and see does the program create the expected output. And then about halfway through the book, I start saying, Okay, it's time for you to start writing your own functions. And we need to start writing those tests for those functions. And those tests can live just right there in the source code, or you can put them into a separate like unit file if you want. And so I think that with more than anything, what people need to learn programming is copius. Number six editing samples. And so I try to provide lots and lots of programs with lots and lots of functions and test for those functions. So they can see all the different ways that they could incorporate these ideas.
Tobias Macey
0:18:52
And as far as the experience of the students that you're working with, and any early reviewers of your book, What are some of the types of problems that you see them run into as they're being introduced to these tools for testing and linting, and being able to do things like using my PI for type annotations, and just some of the complexities or difficulties that they reach and your strategies for helping them pass those hurdles?
Ken Youens-Clark
0:19:19
Well, unfortunately, one, one hurdle is just kind of the Unix windows divide. I really, I always get about half my students are on a Windows platform, and sometimes even like on something like a Chromebook, which has presented its own problems, but on Windows, really, to get an environment that really works. I think, effectively, you really need to install Windows subsystem for Linux, and that has its own complexities, but I think it's getting getting a lot lot better. There's a lot of things that without windows subsystem for Linux, Python works really differently from Windows to two I work on a Mac. So I'm really working on FreeBSD. And and most of my Work is especially insipid to scientific programming is on Linux platforms and virtual machines and such. And so that's a challenge is kind of converting the Windows user over to the command line interface and kind of understanding things like your path, you know, in Python path and that that's that's definitely challenging especially for any any beginner really and and then to kind of explain like that there's all these separate tools like you say, like pilot, which I really recommend that you use a code for matter. For instance, I stress in the book that all the examples that I showed them formatted with why a PDF Yeah, that yet another Python for matter. But there's also black and there's, there's other tools that you can use, I try to show what pilot, what that can do for you and why it's why it's really worth your time to run pilot of your code to find errors that would with Python, we just kind of gloss over it at runtime and it would it would just work. And it might blow up or it might not some of the some of the suggestions from pilots or you know just about spacing and stuff, and that and that's fine. So, but we can use these tools like piland, and YPF, to just automatically make our code look much, much prettier, which honestly makes it easier to read and to understand. So it is difficult, like I will see students just have this like jumble of hand formatted code that does work and does pass the test. And I'll be like, just go up here and click on format code. And then you know, Pam, their code is all of a sudden, so much prettier, and they're like, oh, okay, yeah, that's, that's really a lot nicer. So you know, it. It takes time. And I think it takes a lot of examples. I think that the code examples I presented in the book are clean, and I try to keep them very short. Pretty much all the programs are 100 lines or less. Sometimes they're like 40 lines, but they're complete programs. That You can easily read and look at and see the structure. And I think that that hopefully convinces people to, to use these tools. And now one one note about my pie. It's not until the very last chapter that I introduced the idea of type hinting, I tried to introduce it earlier in the book and I just, I felt like maybe it was too much information at that stage, because I am I am trying to target the novice programmer. So I kind of save it to the end. And I introduced the idea of type hinting and why my PI gives you this just a whole other level of information about your program. And I, you know, if I have an opportunity to write a more intermediate book intermediate to advanced, I would start at that point, I would start with every every line of code name with with type annotations, and some and some more advanced tricks that I tend to use in my own programming.
Tobias Macey
0:22:50
And then you mentioned that the programs that you're working on and each of the chapters are 100 lines or less, which is challenging in that It provides this constraint of needing to keep things approachable and conceptually simple, but still interesting and useful enough to be able to encompass the entirety of the concept that you're trying to present. And I'm wondering what types of challenges you faced in trying to identify different projects that can be self contained in such a small number of lines of code, while still being valuable and educational for the reader?
Ken Youens-Clark
0:23:26
Well, you know, just one of the things that I really tried very hard to make the book interesting and light hearted. And so you know, part of part of what I wanted was that the challenges should be on their face in some way amusing, or interesting. Like in my very first example, when I started off, it was just choose the right article. So given a word, like apple, you should put an ad in front of the ATM and given a word like guitar, you should put a and it just my editor was just like Jesus is Boring. Can you spice it up a little bit? And I said, Uh, yeah, actually it is pretty boring. And so I made it into the crow's nest and then you have to say a hallway Captain there's, you know, and octopus off the larger bow. And I don't know, maybe that made it funnier or not. But it was somewhat more interesting. And then I started writing, drawing these cartoons to try to just I don't know, light it up a little bit, make it feel a little bit more human. But there's so much to learn from something as simple as how do you take a positional argument on the command line? How do you determine if the first character that case insensitive is a vowel? What does it mean for something to be about? How can you compare that letter to all the vowels? How can you then use that information to do conditional branching to choose the letter at you know the word a or the or the word and, and it getting so it gets you into string indexing and string methods like upper and lower using say like x in y. So is this character in the list of vowels, you know for the conditional branching allows me to introduce if else statements. But then since this is simply a binary condition, it's either a or n, then I can turn that if statement into an IF expression and show the same concept that took four or five lines of code. Now I can do it in exactly one line of code and to talk about all these things that like, one of the reasons why the programs are short as they are, is because I try to show what is I try to show the most elegant solution I can think of in Python, and actually, I tend to show multiple ways to solve these problems. And sometimes I start off with a very long winded way, or somewhat long winded way of solving a problem. And then I show what here's how we can shorten that, you know, those six lines of code down into Two lines of code. And and it's not just about golfing your code, it's not it's not about, you know, some sort of alpha male like I know how to write really short, brief code. It's really about like, here's a more elegant way to write that same idea, whether it's like a list comprehension, which is, which was something entirely new to me when I came to Python. And it took me a while to embrace it. But then I was like, Oh, yeah, this comprehension, that's actually pretty cool. And then as I go further into the book, I really start drawing on ideas from purely functional programming languages. And I really start trying to introduce you to ideas and like map and filter and reduce really powerful concepts that allow you to start thinking in the terms of functions and how you can fit functions together. So but you know, I start from the beginning just with strings. That's the first chapter. The second chapter is list like what can I do with the list third, chapters, dictionaries, and then I start introducing files and input output handles and then we get into it In a few chapters on regular expressions, which I think is something that a lot, most people I know, especially novices never even heard of that it sounds really, really scary, you know, regular expressions, but, you know, I try to, to come up with like, you know, Mad Libs, you know how I think, I think probably everyone, maybe not everyone, but lots of people have probably tried to create Mad Libs implementation, it's a great thing to do. And so I try to show you how to do Mad Libs, both with regular expressions. And without, as far as, like, what I was trying, I had some ideas of what I felt like needed to be taught. And then I just found examples. And most of these, I just, I've been programming these little, these little example things to teach a class, you know, for years. And I just kind of pick the ones I thought were the most amusing and might translate the best. And like you say, of course, I'm trying to get this into a book. I'm trying to make the code fit on one or two pages with the code annotations. And so, you know, I'm trying to keep these programs variable focused on whatever it is I'm trying to get across in that chapter. And then hopefully they're also a little bit funny.
Tobias Macey
0:28:09
This portion of podcast dotnet is brought to you by data dog. Do you have an app in production that is slower than you like? is its performance all over the place, sometimes fast and sometimes slow? Do you know why? with data dog you will, you can troubleshoot your apps performance with data dogs end to end tracing, and in one click correlate those Python traces with related logs and metrics, use their detailed flame graphs to identify bottlenecks and latency in that app of yours. start tracking the performance of your app. So the free trial at Python podcast.com slash data dog, if you sign up for a trial and install the agent, data dog, we'll send you a free t shirt to keep you comfortable while you keep track of your apps. And then the structure of the book I was noticing in the chapters that you outline the problem but then after you've gone through the solution, you have sections of digging deeper additional steps, or here are some other things that you can try. And I think that that's definitely a very useful exercise for people who are learning programming. It gives them some way to move off the guard rails a little bit and make mistakes and try things out for themselves after they've already gained a little bit of confidence in building the initial implementation and then seeing Well, if I experiment What else can I do? And I'm wondering what you have seen as far as feedback from your students, or from people who have read the book as to the overall structure and how that has helped them in their overall learning journey.
Ken Youens-Clark
0:29:31
Well, it's still really early I haven't seen too many people haven't said like, necessarily where they've gone with this. I also try to a lot of these are very trivial kinds of problems that we're trying to solve. But they really, it's a very short step for these to have extremely real real world implications, things that you can do with with these skills. And so, you know, sometimes I'm trying to hint at that, that this isn't just playing around. I think that one of the things are Really trying to stress is if you're going to that you should expand, extend these programs and add new features and stuff like that. You should also add the tests for them. And I hope that that that makes them say, Oh, well, how would I add a test and they would open up that test dot p y, which is, which is the integration test, it's included in each directory, and I would hope that they would open it up and study how did I test that element of the program. So for instance, there's there's one of the programs it's called the HAUER And it's basically it reads input either from the command line or from a file and then printed out in all uppercase. The inspiration is from Harry Potter, you know, how they're, you know, screams at the at the recipient that blows up. And so I recommend, okay, just change it so that it now does all lowercase and write a test for that. So how would you do that? You know, make it and give the programmer an option. Like it's by default, it's going to do all uppercase before Now you go add this, this Boolean on off condition like, or something like that, to where it's all lowercase, and then go add the test where you have to run the program with that flag, get the output and verify that it is actually lowercase. And I think that exercise in and of itself would be extremely educational because it would it would really force you to kind of interact with, okay, I'm going to make my program do this, how do I ensure that it actually did the thing I expected it to do? Unfortunately, I really, you know, my, I haven't seen as much feedback on that aspect as I would like. But you know,
Tobias Macey
0:31:39
we'll say Yeah, I definitely think that that's useful because with a lot of introductory books, they walk you down a predefined path. And then once you get to the end of it, you have some measure of knowledge, but you don't really know where to go next with it. And by adding these points throughout the book, where there is some level of indeterminacy as to where you can go with it, or what's going to happen when you try this thing, it encourages experimentation throughout rather than just guiding them along that path and then leaving them at the end of it without any future direction. So I think that that will help to foster experimentation and make them more likely to try to take next steps, either within the programs that they've already built, or with some other learning resource to advance to the next level of their capabilities.
Ken Youens-Clark
0:32:26
Yeah, and and, you know, with the structure of the book is every single chapter is the same, it teaches you kind of presents this problem that you're going to write and then I try to give you just enough material to be able to solve it. Like if you need some background on dictionaries, I talked about that, you know, and then I'll give you in as as the programs get more complicated, I say, you know, like, for instance, singing 99 bottles of beer on the wall, okay, that's, that's a pretty common thing to to code and I say, you know, maybe think about here that you Write a function that just produces one verse, and then you can write a test for that function. And you really only need to test one bottle of beer and not one bottle, like two bottles of beer, really, if you test those two, you can probably pass the whole thing, no matter how many bottles and, and I say, you know, you don't have to write it like this, you know, you can write it however you want. But if you're going to write it in a different way, think about how you would write a test for the way that you're going to write it, like, not just the test that I've provided. But if you're going to write a function, you know, could you mimic this thing that I've just showed you to write your own unit test? And I think that I hope that that kind of thinking is really going to make a difference on the reader that that they're always thinking about. I think I understand this thing that I'm writing, but maybe I should write a test for it and really verify and I've been programming for like 24 years, and I have been I'm pretty good at it. And I'm still amazed at how many times I think I understand a function that I just wrote. And I run the test. And I did not I did not think about this, you know, this, you know, aspect of it. And I got back an unexpected result. And I'm like, Oh, right. Yeah, I didn't think about this aspect. No. And so, um, it's, it's a really important skill to verify the things that you think you understand.
Tobias Macey
0:34:28
Yeah, Boolean algebra is definitely difficult to try and do in your head when it gets beyond one or two elements.
Ken Youens-Clark
0:34:35
There's a lot of really tricky things in this in this industry. And, you know, I think that hubris has its place, but you need to keep it in check. You know, you may be good, but you still need to write tests.
Tobias Macey
0:34:48
Absolutely. And as a book that's intended to serve as a learning resource, what are some of the elements of the Python language and its ecosystem that you consciously left out to it? Avoid overwhelming the readers.
Ken Youens-Clark
0:35:01
The biggest one is object oriented programming. So I started programming around 96. And my first language of Visual Basic, and that was pretty much a procedural language and and then, you know, there was object oriented was really coming up. And my next language was Delphi, which was basically object Pascal. And I started doing object oriented programming and learning all those ideas. And then somehow I got into Perl and at some point, Perl decided to bolt on some Opie stuff that was I wouldn't say elegant, but it was functional and and i really wrote a lot of object oriented code in a couple different languages for a lot of years. And, and then at some point, I just kept hearing about purely functional programming. I kept hearing it and I started reading some books about it. And I started teaching myself a little Haskell. And it really made a huge difference in the way I started thinking about code and I started to I just basically stopped writing object oriented code. And I just started writing functions. And I found my code was easier to understand and easier to test. And so when I wrote this book, I just consciously left out objects. And in fact, you I don't think I've written a really Opie kind of system in Python ever, just because I already abandoned that style. By the time I moved into the language of years ago, and I got a little bit of pushback from the reviewers on the book. They're like, you know, I said there at least one chapter on Opie and, and honestly, as I think that it leads to a lot more boilerplate, a lot more complicated code than it needs to be, it hides the very nature of object oriented programming is to hide and encapsulate data inside these, you know, kind of opaque objects. And the data can be mutated in ways that are subtle and visible. And I just, I don't think that that's the necessarily, especially the teach to a beginner, I think and I and I encourage it in the beginning in the intro, I say we don't talk about objects. If I think it's kind of an advanced topic, I think you should learn about objects, you're going to certainly encounter a lot of object oriented code. But like I was just put on a project a couple months ago, to come in and add tests for a lot of this existing Python code, not a single test insight that's ever been written. And most of the code is object oriented. And I'm just like, Whoa, boy, this is this is going to be really, really challenging because you know, now I have to mock up these objects and find ways to pass these ideas around. Whereas if they were just functions, I could just find each function and write a test for it, passing in the parameters and getting back some value or not some value, and I simply find functional style, easier to teach and easier to test and and easier to show on the page. I don't think that we need to get into a lot of object hierarchies. There. You know, a few things, I think some things about the syntax list. comprehensions, I think are very, very elegant solution in Python. And so I really try to show, you know, most people can I think, grok, a for loop. And I show kind of a naive way of using for loops to create structures like lists and strings. And I say, you know, we could really just use this list comprehension and that's something that's really kind of unique to Python. And it's, it's quite elegant. And so I think that dot and also dictionary comprehensions. I think that that is maybe a little advanced, maybe a little difficult to, for beginner to understand, but I show so many examples of it over and over again that I hope by the end of the book, that just seems like a natural tool to reach for if you're if you're going to be working in Python, you know, there's only so much you can do. My publisher said you know, you can only move The reader up to three notches. So if they're added to, you're not going to get them to be an eight by the end of the book, you can you can maybe bump them up to a four or five. And so you know, I was kind of targeting a one or a two. And I definitely want to get you to an intermediate level a four or five. So, you know, I don't really have space to talk about like databases do introduce parsing like CSV files, which I think is really good, real world kind of a thing. And there's a really beautiful, you know, the CSV module that's built into Python makes that really pretty easy to do. But I guess the biggest thing is, is that I do completely stay away from objects.
Tobias Macey
0:39:34
Yeah, functional programming definitely has a lot of benefits. And I think that one of the things that causes people to shy away from it is a lot of the theoretical aspects to it and the very mathematics heavy lingo that goes along with it that doesn't have to be brought in in order for you to be able to actually use it.
Ken Youens-Clark
0:39:52
Yeah, I never talked about trying to really get into category theory and functional programming and stuff. It is It is seriously overwhelming. And I don't think I'm a, you know, a dumb person. And I feel seriously dumb when I read some of that stuff or I try to open up somebody Haskell code to, like, learn something. And I'm like, What are all these operators? You know, I don't understand what is this? And, and so I try to, I try to keep it really yeah, pretty much no lingo and none of that none of the theoretical stuff like, you know, I don't talk about the fact that lists and strings are monoids they are, and these manono operations are really, really fascinating. But you know, I never bring that term in. And I do stress over and over again, how interchangeable in Python strings and lists are, and you know, but I don't talk about like the identity function note zero and the empty string and things like that. We don't ever we can't really get into folds. But I do get into reduce, which is a really, really great idea. It's kind of like a fold, I guess. So. Yeah, I don't I don't want to intimidate anyone. I just want to say Here's a function, and here's how we can test it. And I think that alone is worth the price of admission if you can write code as simply as possible. And you can have tests that verify that each one of your simple little functions works in isolation, then you can start putting your functions together to compose them into larger programs, that you can then have confidence that they will continue to work in the way that you expect.
Tobias Macey
0:41:25
And as you have worked on writing the book, and going through the review process and getting it edited, what are some of the most interesting or unexpected or challenging lessons that you've learned in the process?
Ken Youens-Clark
0:41:36
I thought I was a pretty good writer. And I was, I had a long way to go. So, um, you know, I've learned so much about teaching and, you know, you think you've expressed something clearly, and then, you know, a reviewer, or, you know, my editor asked me to, you know, explain it better. And, you know, you go back and you have to let go of your ego and I probably have as much More than anyone. So it was really a humbling experience to think that I've explained something well, and it just had fallen flat. And so I remember when I was filling out a proposal, they were like, how many chapters? And how many illustrations Do you think you'll have? And I was like, no idea how many illustrations I'll have. And I ended up having like, hundreds, like you just I did I started using omnigraffle is really great tool for creating diagrams. And I just started realizing like how many times you really know like how many times in class, I would stand in front of this giant screen and point to things like here I'm unpacking these variables like if I'm on packing dict, the items call so item dict items is going to return the keys and values and it's going to return to this tables. And in a for loop. You can say for key comma value in dict dot items and it unpacks those and it's a really elegant way of doing that. But you know in class I would point like, this goes to Hear and this goes to hear. And then you don't have that in a book. And you realize, Oh, I need to make an illustration for that I need to have arrows. And I need to make this as clear as possible. So I think these these visual representations of these ideas, I think, are very, very useful. And so I ended up just creating so many more diagrams that I that I ever thought that I would. And then it was just, you know, surprising the number of times a reviewer would just maybe give me some other idea that I hadn't even considered or, you know, find some other way to test the program in a way that would make it break that I hadn't anticipated, like, oh, wow, I'm so glad I had this army of reviewers catching things. So mostly, I would just say it was it was an extremely humbling experience, to to realize that people out there, they really want to learn these things. But it's my job as the author to make this as clear as possible and to remove any ambiguities and to just make sure that Kinda like, you know, you know, I think about Hemingway, you know, I was an English major. And I just think about how short and powerful and clean and concise his sentences were. And and I was just trying to write in as clear a tone as possible. And also still try to make it a little bit funny. I'm sure I have a quirky sense of humor. I hope it comes off Oh, it's a makes it more interesting than just reading another
Tobias Macey
0:44:24
technical book. And for somebody who has made it to the end of your book, what are your thoughts on some of the useful resources or next steps for people who are interested in progressing in their use of Python or getting into programming as a career or at least using it heavily for a tool in whatever endeavor they are engaged with?
Ken Youens-Clark
0:44:45
That's a really good I don't know, I don't explicitly to recommend anything at the end. And maybe that's a shortcoming more than anything. I hope that that they've thought about that. I've hopefully introduced some ideas along the way like test driven development actually. That that phrase comes from this book by Kent back, which I think it was from, like 2002. So I mean, this, this idea has been around a really long time. And I point to the original book, and I'm like, you know, if you, you should just keep learning more about this. And I kind of point to that there's some ambiguities about like, what is what does unit test meaning what does integration like you should read about these things, you should keep looking further, I make a lot of allusions to other languages, like I, you know, I mentioned rich Hickey is the creator of the closure language. And I and I talked about, you know, that this kind of approach would be something that probably a c++ programmer would look at immediately recognize this four loop kind of approach, whereas this other approach with map and filter would probably be more more recognizable to a Haskell Haskell programmer. And so I hope that by kind of planting these seeds, oh, and then regular expressions to them, and my God, what a deep subject that is, and so I hope that I'm kind of, maybe planted the seed that the reader would go off and start looking At all these other things, I joke that, that if you if you only program in one language, then you kind of suffer from Stockholm Syndrome like you, you start thinking that the warts of your language are actually things to love and be proud of. And I think that it's healthier to expand your knowledge, I actually literally recommend that you try to write these solutions in Python, and then write them in another language that you know, like if you know JavaScript, write it, write it in JavaScript. And if the if the program is called the same thing, it's possible you could even use the test suite that I include to run your other program and test it if it you know, kind of handles the command line parameters in the same way. So I think I really encourage people to to expand their knowledge, not just inside of Python, but you know, in the general world of computer science that we're working in and look at other languages and try to try to understand these bigger ideas like I probably spend like four chapters. on regular expressions, not I mean, they just kind of play a key role for the chapters. It's not like they're just about that, but, and I try to stress that that's a language of itself, you know, a domain specific language that you're going to find inside of databases inside of command line tools inside of pretty much any other programming language you work with. One of the one one of the exercises, actually, I was really struggling with how to craft this regular expression. And I found the answer on a Java board. And I talked about that in the book, like, you know, first of all, you should know professional programmers googled all the time try to find the answer to their problem. And the problem and the solution is not necessarily going to be in you know, from another Python programmer, because there I was trying to find a regular expression. And I found it in a Java board. And you know, what, the exactly the way they typed it work inside a Python because it was a regular expression that I was copying, not the Java and I and I try to stress that so I hope that those elements, the reader would would go would use those ideas to go forward and expand their knowledge specifically with Python, you know, I do recommend that people probably explore object oriented Python or just object oriented programming in general. It's a it's an enormously important paradigm. I think that it may be too much to say that it's falling out of favor, if you know in favor of functional programming, but it certainly functional programming ideas are really taking hold. And I think lambdas have even found a way into Java now. So
Tobias Macey
0:48:33
we'll see. Are there any other aspects of your work on the book or your experience teaching newcomers to Python or your career as a software engineer that we didn't discuss that you'd like to cover before we close out the show?
Ken Youens-Clark
0:48:46
I think that it's always the case that when you teach you learn, things that you just kind of maybe take for granted and you think you understand when you actually have to explain them to someone else. You You realize the limit have what you thought you knew. And so I would say that for anyone, it once you've learned something, pass it on, is it a Maya Angelou I think says something like, you know, once you've kind of gotten to a higher station, you know, your job is to now help other people get there, too. And so I would say that, you know, as you've learned things, maybe find someone to teach that to, it reinforces what you think, you know, and and, and, and so I would say that teaching has been something that has really affected me throughout my career. And in fact, I would like one day, if I can, if I can make this happen to be to become more of a full time teacher helping people to learn how to program. And so that's been an aspect of it, especially going to the University of Arizona and getting a chance to do some teaching in the classroom has been really transformative to me. I think that I would not be the programmer that I am, if I hadn't done that teaching and if I hadn't tried to write these materials, and you know, when you're and the other Yeah, another aspect I would say is that everything that I've done, I would say, since probably 1998, has been open source software. So you can probably find, like pretty much every line of code I've written over the last 20 something years on SourceForge on GitHub, in some supplemental for some paper somewhere, I think when you work in the public eye, just always putting your name out there on your code. I think it forces you to really take pride in your code, and like, you really want to put your best foot forward. And so I would say, Don't hide your work, you know, make make all your GitHub repositories public. You know, who cares if somebody maybe steals that line of code, and think maybe that makes the world a better place because you've shared a solution. And so I would say working in Open Source kind of science and data has really transformed the way that I think in writing this book, obviously, you know, I want to look like a smart person, I don't want to look like some dummy, who doesn't really know how to write Python. So, you know, I really had to dig down and really learn the language. Because, you know, as I've admitted, I really only came to Python like two or three years ago. And I say, throughout the book, you know, I know in Python, there's this idea, there should be one obvious way to do something, but really, I'm more of a Perl guy. And we, you know, the monitor, there's, there's more than one way to do it. And so, you know, I'm going to play around with that idea. I'm going to play around with how many other ways can I find to do this this thing in Python? And so I think that not being stuck in any one technology in any one language. I think that's important for a person's growth. I think that you if you recognize that you've gotten stuck in something you need to unstuck yourself whether that means changing jobs or changing programming language, and or moving to a new city. You know, whatever it takes, you got to keep yourself fresh and moving. And I think sharing your work with others, sharing your knowledge with others is is really crucial to growing yourself and becoming a better person and a better programmer. And yeah, I think I think that's what I would say,
Tobias Macey
0:52:20
for anybody who wants to get in touch with you or follow along with the work that you're doing, I'll have you add your preferred contact information to the show notes. And so with that, I'm going to move us into the pics and this week, I'm gonna choose the Marvel Cinematic Universe, I recently started revisiting those movies, watching them back from the beginning in order with my kids now that they're old enough, and it's interesting and entertaining to revisit the stories and also just see some of the ways that their cinematic style has evolved since they first began and also just some of the background technologies like the types of cellphones that they're using. It's just entertaining to see how much things have changed in those years. So definitely worth taking a look back at some of those original Marvel movies. And revisiting the storylines. So with that, I'll pass it to you can do you have any pics this week?
Ken Youens-Clark
0:53:04
No, it's enchanting. I wasn't prepared for this. But I also have kids. And you know, I don't know when people will be listening to this. But right now we're still in the midst of pandemic in Arizona, we're still very much, at least my family trying to stay very close to home. And so we're pretty bored sometimes with but you know, for some reason, we've gone back and started rewatching, Parks and Rec. And even though I think that the kids have seen the episodes two or three times, and this is the second time for me, there's something comforting in kind of knowing the next punch line, or rediscovering it. And so I don't know, maybe that makes me sound pretty boring to say that I just want to rewatch old television shows. But that's been something that's actually brought me a little bit of comfort and joy during these these pre trying times.
Tobias Macey
0:53:50
All right. Well, thank you very much for taking the time today to join me and discuss the work that you've been doing as somebody who's teaching people how to use Python and as the author of this title Python projects book. It's definitely an interesting book and I appreciate your focus on introducing linting and testing as a first class concern to new programmers. So I appreciate the time and effort you've put into that and I hope you enjoy the rest of your day.
Ken Youens-Clark
0:54:12
Yeah, you too and thanks so much for having me on.
Tobias Macey
0:54:18
Thank you for listening. Don't forget to check out our other show the data engineering podcast at data engineering podcast comm for the latest on modern data management. And visit the site at Python podcasts comm to subscribe to the show, sign up for the mailing list and read the show notes. And if you've learned something or tried out a project from the show, then tell us about it. Email hosts@podcasting.com with your story. To help other people find the show please leave a review on iTunes and tell your friends and co workers
Liked it? Take a second to support Podcast.__init__ on Patreon!