Code Health

Managing Application Secrets with Brian Kelly - Episode 181

Summary

Any application that communicates with other systems or services will at some point require a credential or sensitive piece of information to operate properly. The question then becomes how best to securely store, transmit, and use that information. The world of software secrets management is vast and complicated, so in this episode Brian Kelly, engineering manager at Conjur, aims to help you make sense of it. He explains the main factors for protecting sensitive information in your software development and deployment, ways that information might be leaked, and how to get the whole team on the same page.

Preface

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • When you’re ready to launch your next app you’ll need somewhere to deploy it, so check out Linode. With private networking, shared block storage, node balancers, and a 40Gbit network, all controlled by a brand new API you’ve got everything you need to scale up. Go to podcastinit.com/linode to get a $20 credit and launch a new server in under a minute.
  • Visit the site to subscribe to the show, sign up for the newsletter, and read the show notes. And if you have any questions, comments, or suggestions I would love to hear them. You can reach me on Twitter at @Podcast__init__ or email [email protected])
  • To help other people find the show please leave a review on iTunes, or Google Play Music, tell your friends and co-workers, and share it on social media.
  • Join the community in the new Zulip chat workspace at podcastinit.com/chat
  • Your host as usual is Tobias Macey and today I’m interviewing Brian Kelly about how to store, deploy, and use sensitive information in your applications

Interview

  • Introductions
  • How did you get introduced to Python?
  • To begin with, how do you define a secret in the context of an application?
  • What are the broad categories for solutions to secrets management?
  • What are the different aspects of secrets management in the lifecycle of developing, deploying, and maintaining an application?
  • How does the scale of a project or organization impact the strategies that are reasonable for secrets management?
  • What are some of the most challenging aspects of secrets management at the different stages of usage?
    • What are some of the common reasons that secrets management strategies fail?
    • What are some of the vulnerabilities or attack vectors that development teams should be thinking about when working with credentials?
  • What are your thoughts on versioning of secrets?
  • Beyond storing and deploying sensitive information, what are some of the secondary concerns around secrets management that development teams should be thinking about?
  • How does the use of multiple environments (e.g. dev, QA, production, etc.) affect the strategies used for secrets management?
  • What are some of the most useful resources that you have found for anyone looking to learn more about this subject?

Keep In Touch

Picks

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Keep Your Code Clean Using pre-commit with Anthony Sottile - Episode 178

Summary

Maintaining the health and well-being of your software is a never-ending responsibility. Automating away as much of it as possible makes that challenge more achievable. In this episode Anthony Sottile describes his work on the pre-commit framework to simplify the process of writing and distributing functions to make sure that you only commit code that meets your definition of clean. He explains how it supports tools and repositories written in multiple languages, enforces team standards, and how you can start using it today to ship better software.

Preface

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • When you’re ready to launch your next app you’ll need somewhere to deploy it, so check out Linode. With private networking, shared block storage, node balancers, and a 40Gbit network, all controlled by a brand new API you’ve got everything you need to scale up. Go to podcastinit.com/linode to get a $20 credit and launch a new server in under a minute.
  • Visit the site to subscribe to the show, sign up for the newsletter, and read the show notes. And if you have any questions, comments, or suggestions I would love to hear them. You can reach me on Twitter at @Podcast__init__ or email [email protected])
  • To help other people find the show please leave a review on iTunes, or Google Play Music, tell your friends and co-workers, and share it on social media.
  • Join the community in the new Zulip chat workspace at podcastinit.com/chat
  • Your host as usual is Tobias Macey and today I’m interviewing Anthony Sottile about pre-commit, a framework for managing and maintaining hooks for multiple languages

Interview

  • Introductions
  • How did you get introduced to Python?
  • Can you start by describing what a pre-commit hook is and some of the ways that they are useful for developers?
  • What was you motivation for creating a framework to manage your pre-commit hooks?
    • How does it differ from other projects built to manage these hooks?
  • What are the steps for getting someone started with pre-commit in a new project?
  • Which other event hooks would be most useful to implement for maintaining the health of a repository?
  • What types of operations are most useful for ensuring the health of a project?
  • What types of routines should be avoided as a pre-commit step?
  • Installing the hooks into a user’s local environment is a manual step, so how do you ensure that all of your developers are using the configured hooks?
    • What factors have you found that lead to developers skipping or disabling hooks?
  • How is pre-commit implemented and how has that design evolved from when you first started?
    • What have been the most difficult aspects of supporting multiple languages and package managers?
    • What would you do differently if you started over today?
    • Would you still use Python?
  • For someone who wants to write a plugin for pre-commit, what are the steps involved?
  • What are some of the strangest or most unusual uses of pre-commit hooks that you have seen?
  • What are your plans for the future of pre-commit?

Keep In Touch

Picks

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Continuous Delivery For Complex Systems Using Zuul with Monty Taylor - Episode 172

Summary

Continuous integration systems are important for ensuring that you don’t release broken software. Some projects can benefit from simple, standardized platforms, but as you grow or factor in additional projects the complexity of checking your deployments grows. Zuul is a deployment automation and gating system that was built to power the complexities of OpenStack so it will grow and scale with you. In this episode Monty Taylor explains how he helped start Zuul, how it is designed for scale, and how you can start using it for your continuous delivery systems. He also discusses how Zuul has evolved and the directions it will take in the future.

Preface

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • When you’re ready to launch your next app you’ll need somewhere to deploy it, so check out Linode. With private networking, shared block storage, node balancers, and a 200Gbit network, all controlled by a brand new API you’ve got everything you need to scale up. Go to podcastinit.com/linode to get a $20 credit and launch a new server in under a minute.
  • Visit the site to subscribe to the show, sign up for the newsletter, and read the show notes. And if you have any questions, comments, or suggestions I would love to hear them. You can reach me on Twitter at @Podcast__init__ or email [email protected])
  • To help other people find the show please leave a review on iTunes, or Google Play Music, tell your friends and co-workers, and share it on social media.
  • Join the community in the new Zulip chat workspace at podcastinit.com/chat
  • Your host as usual is Tobias Macey and today I’m interviewing Monty Taylor about Zuul, a platform that drives continuous integration, delivery, and deployment systems with a focus on project gating and interrelated projects.

Interview

  • Introductions
  • How did you get introduced to Python?
  • Can you start by explaining what Zuul is and how the project got started?
  • How do you view Zuul in the broader landscape of CI/CD systems (e.g. GoCD, Jenkins, Travis, etc.)?
  • What is the workflow for someone who is defining a pipeline in Zuul?
    • How are the pipelines tested and promoted?
    • One of the problems that are often encountered in CI/CD systems is the difficulty of testing changes locally. What kind of support is available in Zuul for that?
  • Can you describe the project architecture?
    • What aspects of the architecture enable it to scale to large projects and teams?
  • How difficult would it be to swap the Ansible integration for another orchestration tool?
  • What would be involved in adding support for additional version control systems?
  • What are your plans for the future of the project?

Keep In Touch

Picks

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Laboratory: Safer Refactoring with Joe Alcorn - Episode 150

Summary

Every piece of software that has been around long enough ends up with some piece of it that needs to be redesigned and refactored. Often the code that needs to be updated is part of the critical path through the system, increasing the risks associated with any change. One way around this problem is to compare the results of the new code against the existing logic to ensure that you aren’t introducing regressions. This week Joe Alcorn shares his work on Laboratory, how the engineers at GitHub inspired him to create it as an analog to the Scientist gem, and how he is using it for his day job.

Preface

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • I would like to thank everyone who supports us on Patreon. Your contributions help to make the show sustainable.
  • When you’re ready to launch your next project you’ll need somewhere to deploy it. Check out Linode at podastinit.com/linode and get a $20 credit to try out their fast and reliable Linux virtual servers for running your awesome app. And now you can deliver your work to your users even faster with the newly upgraded 200 GBit network in all of their datacenters.
  • If you’re tired of cobbling together your deployment pipeline then it’s time to try out GoCD, the open source continuous delivery platform built by the people at ThoughtWorks who wrote the book about it. With GoCD you get complete visibility into the life-cycle of your software from one location. To download it now go to podcatinit.com/gocd. Professional support and enterprise plugins are available for added piece of mind.
  • Visit the site to subscribe to the show, sign up for the newsletter, and read the show notes. And if you have any questions, comments, or suggestions I would love to hear them. You can reach me on Twitter at @Podcast__init__ or email [email protected])
  • To help other people find the show please leave a review on iTunes, or Google Play Music, tell your friends and co-workers, and share it on social media.
  • A brief announcement before we start the show:
    • If you work with data or want to learn more about how the projects you have heard about on the show get used in the real world then join me at the Open Data Science Conference in Boston from May 1st through the 4th. It has become one of the largest events for data scientists, data engineers, and data driven businesses to get together and learn how to be more effective. To save 60% off your tickets go to podcastinit.com/odsc-east-2018 and register.
  • Your host as usual is Tobias Macey and today I’m interviewing Joe Alcorn about using Laboratory as a safety net for your refactoring.

Interview

  • Introductions
  • How did you get introduced to Python?
  • Can you start be explaining what Laboratory is and what motivated you to start the project?
  • How much of the design and implementation were directly inspired by the Scientist project from GitHub and how much of it did you have to figure out from scratch due to differences in the target languages?
  • What have been some of the most challenging aspects of building and maintaining Laboratory, and have you had any opportunities to use it on itself?
  • For someone who would like to use Laboratory in their project, what does the workflow look like and what potential pitfalls should they watch out for?
  • In the documentation you mention that portions of code that perform I/O and create side effects should be avoided. Have you found any strategies to allow for stubbing out the external interactions while still executing the rest of the logic?
  • How do you keep track of the results for active experiments and what sort of reporting is available?
  • What are some examples of the types of routines that would be good candidates for conducting an experiment?
  • What are some of the most complicated or difficult pieces of code that you have refactored with the help of Laboratory?
  • Given the fact that Laboratory is intended to be run in production and provide a certain measure of safety, what methods do you use to ensure that users of the library will not suffer from a drastic increase in overhead or unintended aberrations in the functionality of their software?
  • Are there any new features or improvements that you have planned for future releases of Laboratory?

Keep In Touch

Picks

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Software Architecture For Developers with Neal Ford - Episode 149

Summary

Whether it is intentional or accidental, every piece of software has an existing architecture. In this episode Neal Ford discusses the role of a software architect, methods for improving the design of your projects, pitfalls to avoid, and provides some resources for continuing to learn about how to design and build successful systems.

Preface

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • I would like to thank everyone who supports us on Patreon. Your contributions help to make the show sustainable.
  • When you’re ready to launch your next project you’ll need somewhere to deploy it. Check out Linode at podastinit.com/linode and get a $20 credit to try out their fast and reliable Linux virtual servers for running your awesome app. And now you can deliver your work to your users even faster with the newly upgraded 200 GBit network in all of their datacenters.
  • If you’re tired of cobbling together your deployment pipeline then it’s time to try out GoCD, the open source continuous delivery platform built by the people at ThoughtWorks who wrote the book about it. With GoCD you get complete visibility into the life-cycle of your software from one location. To download it now go to podcatinit.com/gocd. Professional support and enterprise plugins are available for added piece of mind.
  • Visit the site to subscribe to the show, sign up for the newsletter, and read the show notes. And if you have any questions, comments, or suggestions I would love to hear them. You can reach me on Twitter at @Podcast__init__ or email [email protected])
  • To help other people find the show please leave a review on iTunes, or Google Play Music, tell your friends and co-workers, and share it on social media.
  • A few announcements before we start the show:
    • There is still time to register for the O’Reilly Software Architecture Conference in New York. Use the link podcastinit.com/sacon-new-york to register and save 20%
    • If you work with data or want to learn more about how the projects you have heard about on the show get used in the real world then join me at the Open Data Science Conference in Boston from May 1st through the 4th. It has become one of the largest events for data scientists, data engineers, and data driven businesses to get together and learn how to be more effective. To save 60% off your tickets go to podcastinit.com/odsc-east-2018 and register.
  • With many thanks to O’Reilly Media, I have two items to give away. To sign up you just need to subscribe to the mailing list at podcastinit.com and you will have the chance to win either a copy of Neal’s book, Building Evolutionary Architectures, or a Bronze ticket to the O’Reilly Software Architecture Conference in New York. I will be picking the winners on February 21st.
  • Your host as usual is Tobias Macey and today I’m interviewing Neal Ford about principles of software architecture for developers

Interview

  • Introductions
  • How did you get introduced to Python?
  • A majority of your work has been focused on software architectures and how that can be used to facilitate delivery of working systems. Can you start by giving a high level description of what software architecture is and how it fits into the overall development process?
  • One of the difficulties that arise in long-lived projects is that technical debt accrues to the point that forward progress stagnates due to fear that any changes will cause the system to stop functioning. What are some methods that developers can use to either guard against that eventuality, or address it when it happens?
  • What are some of the broad categories of architectural patterns that developers should be aware of?
  • Are there aspects of the language that a system or application is being implemented in which influence the style of architecture that is commonly used?
  • What are some architectural anti-patterns that you have found to be the most commonly occurring?
  • Software is useless if there is no way to deliver it to the end user. What are some of the challenges that are most often overlooked by engineering teams and how do you solve for them?
  • Beyond the purely technological aspects, what other elements of software production and delivery are necessary for a successful architecture?
  • What resources can you recommend for someone who is interested in learning more about software architecture, whether as an individual contributor or in a full time architect role?

Keep In Touch

Picks

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

MonkeyType with Carl Meyer and Matt Page - Episode 146

Summary

One of the draws of Python is how dynamic and flexible the language can be. Sometimes, that flexibility can be problematic if the format of variables at various parts of your program is unclear or the descriptions are inaccurate. The growing middle ground is to use type annotations as a way of providing some verification of the format of data as it flows through your application and enforcing gradual typing. To make it simpler to get started with type hinting, Carl Meyer and Matt Page, along with other engineers at Instagram, created MonkeyType to analyze your code as it runs and generate the type annotations. In this episode they explain how that process works, how it has helped them reduce bugs in their code, and how you can start using it today.

Preface

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • I would like to thank everyone who supports us on Patreon. Your contributions help to make the show sustainable.
  • When you’re ready to launch your next project you’ll need somewhere to deploy it. Check out Linode at podastinit.com/linode and get a $20 credit to try out their fast and reliable Linux virtual servers for running your awesome app. And now you can deliver your work to your users even faster with the newly upgraded 200 GBit network in all of their datacenters.
  • If you’re tired of cobbling together your deployment pipeline then it’s time to try out GoCD, the open source continuous delivery platform built by the people at ThoughtWorks who wrote the book about it. With GoCD you get complete visibility into the life-cycle of your software from one location. To download it now go to podcatinit.com/gocd. Professional support and enterprise plugins are available for added piece of mind.
  • Visit the site to subscribe to the show, sign up for the newsletter, and read the show notes. And if you have any questions, comments, or suggestions I would love to hear them. You can reach me on Twitter at @Podcast__init__ or email [email protected])
  • To help other people find the show please leave a review on iTunes, or Google Play Music, tell your friends and co-workers, and share it on social media.
  • A few announcements before we start the show:
    • There’s still time to get your tickets for PyCon Colombia, happening February 9th and 10th. Go to pycon.co to learn more and register.
    • There is also still time to register for the O’Reilly Software Architecture Conference in New York Feb 25-28. Use the link podcastinit.com/sacon-new-york to register and save 20%
    • If you work with data or want to learn more about how the projects you have heard about on the show get used in the real world then join me at the Open Data Science Conference in Boston from May 1st through the 4th. It has become one of the largest events for data scientists, data engineers, and data driven businesses to get together and learn how to be more effective. To save 60% off your tickets go to podcastinit.com/odsc-east-2018 and register.
  • Your host as usual is Tobias Macey and today I’m interviewing Carl Meyer and Matt Page about MonkeyType, a system to collect type information at runtime for your Python 3 code

Interview

  • Introductions
  • How did you get introduced to Python?
  • What is MonkeyType and how did the project get started?
  • How much overhead does the MonkeyType tracing add to the running system, and what techniques have you used to minimize the impact on production systems?
  • Given that the type information is collected from call traces at runtime, and some functions may accept multiple different types for the same arguments (e.g. add), do you have any logic that will allow for combining that information into a higher-order type that gets set as the annotation?
  • How does MonkeyType function internally and how has the implementation evolved over the time that you have been working on it?
  • Once the type annotations are present in your code base, what other tooling are you using to take advantage of that information?
  • It seems as though using MonkeyType to trace your running production systems could be a way to inadvertantly identify dead sections of code that aren’t being executed. Have you investigated ways to use the collected type information perform that analysis?
  • What have been some of the most challenging aspects of building, using, and maintaining MonkeyType?
  • What have been some of the most interesting or noteworthy things that you have learned in the process of working on and with MonkeyType?
  • What have you found to be the most useful and most problematic aspects of the typing capabilities provided in recent versions of Python?
  • For someone who wants to start using MonkeyType today, what is involved in getting it set up and using it in a new or existing codebase?
  • What features or improvements do you have planned for future releases of MonkeyType?

Keep In Touch

Picks

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Eliot: Effective Logging with Itamar Turner-Trauring - Episode 133

Summary

Understanding what is happening in a software system can be difficult, especially when you have inconsistent log messages. Itamar Turner-Trauring created Eliot to make it possible for your project to tell you a story about how transactions flow through your program. In this week’s episode we go deep on proper logging practices, anti patterns, and how to improve your ability to debug your software with log messages.

Preface

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • I would like to thank everyone who supports us on Patreon. Your contributions help to make the show sustainable.
  • When you’re ready to launch your next project you’ll need somewhere to deploy it. Check out Linode at podastinit.com/linode and get a $20 credit to try out their fast and reliable Linux virtual servers for running your awesome app. And now you can deliver your work to your users even faster with the newly upgraded 200 GBit network in all of their datacenters.
  • If you’re tired of cobbling together your deployment pipeline then it’s time to try out GoCD, the open source continuous delivery platform built by the people at ThoughtWorks who wrote the book about it. With GoCD you get complete visibility into the life-cycle of your software from one location. To download it now go to podcatinit.com/gocd. Professional support and enterprise plugins are available for added piece of mind.
  • Visit the site to subscribe to the show, sign up for the newsletter, and read the show notes. And if you have any questions, comments, or suggestions I would love to hear them. You can reach me on Twitter at @Podcast__init__ or email [email protected])
  • To help other people find the show please leave a review on iTunes, or Google Play Music, tell your friends and co-workers, and share it on social media.
  • Your host as usual is Tobias Macey and today I’m interviewing Itamar Turner-Trauring about Eliot, a library for managing complex logs across multiple processes.

Interview

  • Introductions
  • How did you get introduced to Python?
  • What is Eliot and what problem were you trying to solve by creating it?
  • How is Eliot implemented and how has the design evolved since you first started working on it?
  • Why is it so important to have a standardized format for your application logs?
  • What are some of the anti-patterns that you consider to be the most harmful when developers are setting up logging in their projects?
  • What have been the most challenging aspects of building and maintaining Eliot?
  • How does Eliot compare to some of the other third party logging libraries available such as structlog or logbook?
  • What are some of the improvements or additional features that you have planned for the future of Eliot?

Keep In Touch

Picks

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Jedi Code Completion with David Halter - Episode 113

Summary

When you’re writing python code and your editor offers some suggestions, where does that suggestion come from? The most likely answer is Jedi! This week David Halter explains the history of how the Jedi auto completion library was created, how it works under the hood, and where he plans on taking it.

Preface

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • I would like to thank everyone who supports us on Patreon. Your contributions help to make the show sustainable.
  • When you’re ready to launch your next project you’ll need somewhere to deploy it. Check out Linode at www.podastinit.com/linode and get a $20 credit to try out their fast and reliable Linux virtual servers for running your awesome app.
  • Visit the site to subscribe to the show, sign up for the newsletter, read the show notes, and get in touch.
  • To help other people find the show please leave a review on iTunes, or Google Play Music, tell your friends and co-workers, and share it on social media.
  • Your host as usual is Tobias Macey and today I’m interviewing David Halter about Jedi, an awesome autocompletion and static analysis library for Python

Interview

  • Introductions
  • How did you get introduced to Python?
  • Can you explain what Jedi is and what problem you were trying to solve when you created it?
    • What is the story behind the name?
  • While reading through the documentation I noticed that there is alpha support for linting with Jedi. Can you compare the linting approach and capabilities with those found in other tools such as pylint and flake8?
  • What does the internal architecture and design look like?
  • From the research that I did for the show it seems that, rather than use the AST to determine the structure of the code being completed you built your own parser and recursive evaluation of the other methods that you use for determining accurate completion?
    • What was lacking in existing parsers that led you to build your own?
    • What are some of the difficulties that you have encountered building and maintaining the grammar definitions and higher level API for parsing multiple versions of Python, including the 2 vs 3 split?
  • What are some of the biggest challenges associated with introspecting user code?
  • What are some of the ways that Jedi can be confounded by a user’s project?
  • What are some of the most difficult technical hurdles that you have been faced with while building Jedi?
  • What are some unusual or unexpected uses of Jedi that you have seen?
  • What do you have planned for the future of Jedi?

Keep In Touch

Picks

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Tech Debt and Refactoring at Yelp! with Andrew Mason - Episode 110

Summary

Healthy code makes for happy coders, and there are many ways to measure the health of a project. This week Andrew Mason talks about the Undebt project from Yelp!, as well as some of the other tools and practices that have been developed to make sure that the balance on their technical debt card stays low. Give it a listen to learn how and why to measure and address the painful parts of your software.

Preface

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • I would like to thank everyone who supports us on Patreon. Your contributions help to make the show sustainable.
  • When you’re ready to launch your next project you’ll need somewhere to deploy it. Check out Linode at www.podastinit.com/linode and get a $20 credit to try out their fast and reliable Linux virtual servers for running your awesome app.
  • Visit the site to subscribe to the show, sign up for the newsletter, read the show notes, and get in touch.
  • To help other people find the show please leave a review on iTunes, or Google Play Music, tell your friends and co-workers, and share it on social media.
  • Your host as usual is Tobias Macey and today I’m interviewing Andrew Mason about technical debt and refactoring with Undebt.

Interview

  • Introductions
  • How did you get introduced to Python?
  • How do you define technical debt and why is it an important aspect of a project to keep track of?
  • How would you characterize refactoring in general and when you might want to do it?
  • What is Undebt and what was the problem that you were facing at Yelp when it was created?
  • For someone who wants to get started with using Undebt what does that process look like and how does it work under the covers?
  • What are some of the other tools and techniques available for refactoring Python code and how do they differ from what is possible in Undebt?
  • What are some of the other tools and methods that you use to maintain the overall health of your codebase?
  • What are some of the limitations and edge cases that you have experiemced working with Undebt?
  • It is often a difficult balancing act when working in a team to determine how much time to spend paying down technical debt and building tools that will act as force multipliers vs doing feature work that will be visible to end-users. In your experience, what are some ways to manage that tension?

Keep In Touch

Picks

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA