Tools

Unpacking The Python Toolkit For Chaos Engineering - Episode 199

Summary

Chaos engineering is the practice of injecting failures into your production systems in a controlled manner to identify weaknesses in your applications. In order to build, run, and report on chaos experiments Sylvain Hellegouarch created the Chaos Toolkit. In this episode he explains his motivation for creating the toolkit, how to use it for improving the resiliency of your systems, and his plans for the future. He also discusses best practices for building, running, and learning from your own experiments.

Announcements

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • When you’re ready to launch your next app or want to try a project you hear about on the show, you’ll need somewhere to deploy it, so take a look at our friends over at Linode. With 200 Gbit/s private networking, scalable shared block storage, node balancers, and a 40 Gbit/s public network, all controlled by a brand new API you’ve got everything you need to scale up. Go to pythonpodcast.com/linode to get a $20 credit and launch a new server in under a minute. And don’t forget to thank them for their continued support of this show!
  • And to keep track of how your team is progressing on building new features and squashing bugs, you need a project management system designed by software engineers, for software engineers. Clubhouse lets you craft a workflow that fits your style, including per-team tasks, cross-project epics, a large suite of pre-built integrations, and a simple API for crafting your own. Podcast.__init__ listeners get 2 months free on any plan by going to pythonpodcast.com/clubhouse today and signing up for a trial.
  • Visit the site to subscribe to the show, sign up for the newsletter, and read the show notes. And if you have any questions, comments, or suggestions I would love to hear them. You can reach me on Twitter at @Podcast__init__ or email [email protected])
  • To help other people find the show please leave a review on iTunes, or Google Play Music, tell your friends and co-workers, and share it on social media.
  • Join the community in the new Zulip chat workspace at pythonpodcast.com/chat
  • Your host as usual is Tobias Macey and today I’m interviewing Sylvain Hellegouarch about Chaos Toolkit, a framework for building and automating chaos engineering experiments

Interview

  • Introductions
  • How did you get introduced to Python?
  • Can you start by explaining what Chaos Engineering is?
  • What is the Chaos Toolkit and what motivated you to create it?
    • How does it compare to the Gremlin platform?
  • What is the workflow for using Chos Toolkit to build and run an experiment?
    • What are the best practices for building a useful experiment?
    • Once you have an experiment created, how often should it be executed?
  • When running an experiment, what are some strategies for identifying points of failure, particularly if they are unexpected?
    • What kinds of reporting and statistics are captured during a test run?
  • Can you describe how Chaos Toolkit is implemented and how it has evolved since you began working on it?
  • What are some of the most challenging aspects of ensuring that the experiments run via the Chaos Toolkit are safe and have a reliable rollback available?
  • What have been some of the most interesting/useful/unexpected lessons that you have learned in the process of building and maintaining the Chaos Toolkit project and community?
  • What do you have planned for the future of the project?

Keep In Touch

Picks

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Computational Musicology For Python Programmers - Episode 198

Summary

Music is a part of every culture around the world and throughout history. Musicology is the study of that music from a structural and sociological perspective. Traditionally this research has been done in a manual and painstaking manner, but the advent of the computer age has enabled an increase of many orders of magnitude in the scope and scale of analysis that we can perform. The music21 project is a Python library for computer aided musicology that is written and used by MIT professor Michael Scott Cuthbert. In this episode he explains how the project was started, how he is using it personally, professionally, and in his lectures, as well as how you can use it for your own exploration of musical analysis.

Announcements

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • When you’re ready to launch your next app or want to try a project you hear about on the show, you’ll need somewhere to deploy it, so take a look at our friends over at Linode. With 200 Gbit/s private networking, scalable shared block storage, node balancers, and a 40 Gbit/s public network, all controlled by a brand new API you’ve got everything you need to scale up. Go to pythonpodcast.com/linode to get a $20 credit and launch a new server in under a minute. And don’t forget to thank them for their continued support of this show!
  • And to keep track of how your team is progressing on building new features and squashing bugs, you need a project management system designed by software engineers, for software engineers. Clubhouse lets you craft a workflow that fits your style, including per-team tasks, cross-project epics, a large suite of pre-built integrations, and a simple API for crafting your own. Podcast.__init__ listeners get 2 months free on any plan by going to pythonpodcast.com/clubhouse today and signing up for a trial.
  • Visit the site to subscribe to the show, sign up for the newsletter, and read the show notes. And if you have any questions, comments, or suggestions I would love to hear them. You can reach me on Twitter at @Podcast__init__ or email [email protected])
  • To help other people find the show please leave a review on iTunes, or Google Play Music, tell your friends and co-workers, and share it on social media.
  • Join the community in the new Zulip chat workspace at pythonpodcast.com/chat
  • Your host as usual is Tobias Macey and today I’m interviewing Michael Cuthbert about music21, a toolkit for computer aided musicology

Interview

  • Introductions
  • How did you get introduced to Python?
  • Can you start by explaining what computational musicology is?
  • What is music21 and what motivated you to create it?
    • What are some of the use cases that music21 supports, and what are some common requests that you purposefully don’t support?
  • How much knowledge of musical notation, structure, and theory is necessary to be able to work with music21?
  • Can you talk through a typical workflow for doing analysis of one or more pieces of existing music?
    • What are some of the common challenges that users encounter when working with it (either on the side of Python or musicology/musical theory)?
    • What about for doing exploration of new musical works?
  • As a professor at MIT, what are some of the ways that music21 has been incorporated into your classroom?
    • What have they enjoyed most about it?
  • How is music21 implemented, and how has its structure evolved since you first started it?
    • What have been the most challenging aspects of building and maintaining the music21 project and community?
  • What are some of the most interesting, unusual, or unexpected ways that you have seen music21 used?
    • What are some analyses that you have performed which yielded unexpected results?
  • What do you have planned for the future of music21?
  • Beyond computational analysis of musical theory, what are some of the other ways that you are using Python in your academic and professional pursuits?

Keep In Touch

Picks

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Counteracting Code Complexity With Wily - Episode 195

Summary

As we build software projects, complexity and technical debt are bound to creep into our code. To counteract these tendencies it is necessary to calculate and track metrics that highlight areas of improvement so that they can be acted on. To aid in identifying areas of your application that are breeding grounds for incidental complexity Anthony Shaw created Wily. In this episode he explains how Wily traverses the history of your repository and computes code complexity metrics over time and how you can use that information to guide your refactoring efforts.

Preface

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • When you’re ready to launch your next app or want to try a project you hear about on the show, you’ll need somewhere to deploy it, so take a look at our friends over at Linode. With 200 Gbit/s private networking, scalable shared block storage, node balancers, and a 40 Gbit/s public network, all controlled by a brand new API you’ve got everything you need to scale up. Go to pythonpodcast.com/linode to get a $20 credit and launch a new server in under a minute. And don’t forget to thank them for their continued support of this show!
  • And to keep track of how your team is progressing on building new features and squashing bugs, you need a project management system designed by software engineers, for software engineers. Clubhouse lets you craft a workflow that fits your style, including per-team tasks, cross-project epics, a large suite of pre-built integrations, and a simple API for crafting your own. Podcast.__init__ listeners get 2 months free on any plan by going to pythonpodcast.com/clubhouse today and signing up for a trial.
  • Visit the site to subscribe to the show, sign up for the newsletter, and read the show notes. And if you have any questions, comments, or suggestions I would love to hear them. You can reach me on Twitter at @Podcast__init__ or email [email protected])
  • To help other people find the show please leave a review on iTunes, or Google Play Music, tell your friends and co-workers, and share it on social media.
  • Join the community in the new Zulip chat workspace at pythonpodcast.com/chat
  • Your host as usual is Tobias Macey and today I’m interviewing Anthony Shaw about Wily, a command-line application for tracking and reporting on complexity of Python tests and applications

Interview

  • Introductions
  • How did you get introduced to Python?
  • Can you start by describing what Wily is and what motivated you to create it?
  • What is software complexity and why should developers care about it?
    • What are some methods for measuring complexity?
  • I know that Python has the McCabe tool, but what other methods are there for determining complexity, both in Python and for other languages?
  • What kinds of useful signals can you derive from evaluating historical trends of complexity in a codebase?
  • What are some other useful metrics for tracking and maintaining the health of a software project?
  • Once you have established the points of complexity in your software, what are some strategies for remediating it?
  • What are your favorite tools for refactoring?
  • What are some of the aspects of developer-oriented tools that you have found to be most important in your own projects?
  • What are your plans for the future of Wily, or any other tools that you have in mind to aid in producing healthy software?

Keep In Touch

Picks

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Using Calibre To Keep Your Digital Library In Order with Kovid Goyal - Episode 187

Summary

Digital books are convenient and useful ways to have easy access to large volumes of information. Unfortunately, keeping track of them all can be difficult as you gain more books from different sources. Keeping your reading device synchronized with the material that you want to read is also challenging. In this episode Kovid Goyal explains how he created the Calibre digital library manager to solve these problems for himself, how it grew to be the most popular application for organizing ebooks, and how it works under the covers. Calibre is an incredibly useful piece of software with a lot of hidden complexity and a great story behind it.

Preface

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • When you’re ready to launch your next app or want to try a project you hear about on the show, you’ll need somewhere to deploy it, so check out Linode. With 200 Gbit/s private networking, scalable shared block storage, node balancers, and a 40 Gbit/s public network, all controlled by a brand new API you’ve got everything you need to scale up. Go to podcastinit.com/linode to get a $20 credit and launch a new server in under a minute.
  • Visit the site to subscribe to the show, sign up for the newsletter, and read the show notes. And if you have any questions, comments, or suggestions I would love to hear them. You can reach me on Twitter at @Podcast__init__ or email [email protected])
  • To help other people find the show please leave a review on iTunes, or Google Play Music, tell your friends and co-workers, and share it on social media.
  • Join the community in the new Zulip chat workspace at podcastinit.com/chat
  • Your host as usual is Tobias Macey and today I’m interviewing Kovid Goyal about Calibre, the powerful and free ebook management tool

Interview

  • Introductions
  • How did you get introduced to Python?
  • Can you start by explaining what Calibre is and how the project got started?
  • How are you able to keep up to date with device support in Calibre, given the continual release of new devices and platforms that a user can read ebooks on?
  • What are the main features of Calibre?
    • What are some of the most interesting and most popular plugins that have been creatd for Calibre?
  • Can you describe the software architecture for the project and how it has evolved since you first started working on it?
  • You have been maintaining and improving Calibre for a long time now. What is your motivation to keep working on it?
    • How has the focus of the project and the primary use cases changed over the years that you have been working on it?
  • In addition to its longevity, Calibre has also become a de-facto standard for ebook management. What is your opinion as to why it has gained and kept its popularity?
    • What are some of the competing options and how does Calibre differentiate from them?
  • In addition to the myriad devices and platforms, there is a significant amount of complexity involved in supporting the different ebook formats. What have been the most challenging or complex aspects of managing and converting between the formats?
  • One of the challenges around maintaining a private library of electronic resources is the prevalence of DRM restricted content available through major publishers and retailers. What are your thoughts on the current state of digital book marketplaces?
  • What was your motivation for implementing Calibre in Python?
    • If you were to start the project over today would you make the same choice?
    • Are there any aspects of the project that you would implement differently if you were starting over?
  • What are your plans for the future of Calibre?

Keep In Touch

Picks

  • Tobias
  • Kovid
    • Into Thin Air by John Krakauer
      About how an expedition to climb Everest went wrong. Wonderful account of the difficulties of high altitude mountaineering and the determination it needs.
    • The Steerswoman’s Road by Rosemary Kirstein
      About the spirit of scientific enquiry in a fallen civilization on an alien planet with partial terraforming that is slowly failing.

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Entity Extraction, Document Processing, And Knowledge Graphs For Investigative Journalists with Friedrich Lindenberg - Episode 186

Summary

Investigative reporters have a challenging task of identifying complex networks of people, places, and events gleaned from a mixed collection of sources. Turning those various documents, electronic records, and research into a searchable and actionable collection of facts is an interesting and difficult technical challenge. Friedrich Lindenberg created the Aleph project to address this issue and in this episode he explains how it works, why he built it, and how it is being used. He also discusses his hopes for the future of the project and other ways that the system could be used.

Preface

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • When you’re ready to launch your next app or want to try a project you hear about on the show, you’ll need somewhere to deploy it, so check out Linode. With 200 Gbit/s private networking, scalable shared block storage, node balancers, and a 40 Gbit/s public network, all controlled by a brand new API you’ve got everything you need to scale up. Go to podcastinit.com/linode today to get a $20 credit and launch a new server in under a minute.
  • Visit the site to subscribe to the show, sign up for the newsletter, and read the show notes. And if you have any questions, comments, or suggestions I would love to hear them. You can reach me on Twitter at @Podcast__init__ or email [email protected])
  • To help other people find the show please leave a review on iTunes, or Google Play Music, tell your friends and co-workers, and share it on social media.
  • Join the community in the new Zulip chat workspace at podcastinit.com/chat
  • Registration for PyCon US, the largest annual gathering across the community, is open now. Don’t forget to get your ticket and I’ll see you there!
  • Your host as usual is Tobias Macey and today I’m interviewing Friedrich Lindenberg about Aleph, a tool to perform entity extraction across documents and structured data

Interview

  • Introductions
  • How did you get introduced to Python?
  • Can you start by explaining what Aleph is and how the project got started?
  • What is investigative journalism?
    • How does Aleph fit into their workflow?
    • What are some other tools that would be used alongside Aleph?
    • What are some ways that Aleph could be useful outside of investigative journalism?
  • How is Aleph architected and how has it evolved since you first started working on it?
  • What are the major components of Aleph?
    • What are the types of documents and data formats that Aleph supports?
  • Can you describe the steps involved in entity extraction?
    • What are the most challenging aspects of identifying and resolving entities in the documents stored in Aleph?
  • Can you describe the flow of data through the system from a document being uploaded through to it being displayed as part of a search query?
  • What is involved in deploying and managing an installation of Aleph?
  • What have been some of the most interesting or unexpected aspects of building Aleph?
  • Are there any particularly noteworthy uses of Aleph that you are aware of?
  • What are your plans for the future of Aleph?

Keep In Touch

Picks

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Of Checklists, Ethics, and Data with Emily Miller and Peter Bull - Episode 184

Summary

As data science becomes more widespread and has a bigger impact on the lives of people, it is important that those projects and products are built with a conscious consideration of ethics. Keeping ethical principles in mind throughout the lifecycle of a data project helps to reduce the overall effort of preventing negative outcomes from the use of the final product. Emily Miller and Peter Bull of Driven Data have created Deon to improve the communication and conversation around ethics among and between data teams. It is a Python project that generates a checklist of common concerns for data oriented projects at the various stages of the lifecycle where they should be considered. In this episode they discuss their motivation for creating the project, the challenges and benefits of maintaining such a checklist, and how you can start using it today.

Preface

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • When you’re ready to launch your next app you’ll need somewhere to deploy it, so check out Linode. With private networking, shared block storage, node balancers, and a 40Gbit network, all controlled by a brand new API you’ve got everything you need to scale up. Go to podcastinit.com/linode to get a $20 credit and launch a new server in under a minute.
  • Visit the site to subscribe to the show, sign up for the newsletter, and read the show notes. And if you have any questions, comments, or suggestions I would love to hear them. You can reach me on Twitter at @Podcast__init__ or email [email protected])
  • To help other people find the show please leave a review on iTunes, or Google Play Music, tell your friends and co-workers, and share it on social media.
  • Join the community in the new Zulip chat workspace at podcastinit.com/chat
  • Your host as usual is Tobias Macey and today I’m interviewing Emily Miller and Peter Bull about Deon, an ethics checklist for data projects

Interview

  • Introductions
  • How did you get introduced to Python?
  • Can you start by describing what Deon is and your motivation for creating it?
  • Why a checklist, specifically? What’s the advantage of this over an oath, for example?
  • What is unique to data science in terms of the ethical concerns, as compared to traditional software engineering?
  • What is the typical workflow for a team that is using Deon in their projects?
  • Deon ships with a default checklist but allows for customization. What are some common addendums that you have seen?
    • Have you received pushback on any of the default items?
  • How does Deon simplify communication around ethics across team boundaries?
  • What are some of the most often overlooked items?
  • What are some of the most difficult ethical concerns to comply with for a typical data science project?
  • How has Deon helped you at Driven Data?
  • What are the customer facing impacts of embedding a discussion of ethics in the product development process?
  • Some of the items on the default checklist coincide with regulatory requirements. Are there any cases where regulation is in conflict with an ethical concern that you would like to see practiced?
  • What are your hopes for the future of the Deon project?

Keep In Touch

Picks

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Keep Your Code Clean Using pre-commit with Anthony Sottile - Episode 178

Summary

Maintaining the health and well-being of your software is a never-ending responsibility. Automating away as much of it as possible makes that challenge more achievable. In this episode Anthony Sottile describes his work on the pre-commit framework to simplify the process of writing and distributing functions to make sure that you only commit code that meets your definition of clean. He explains how it supports tools and repositories written in multiple languages, enforces team standards, and how you can start using it today to ship better software.

Preface

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • When you’re ready to launch your next app you’ll need somewhere to deploy it, so check out Linode. With private networking, shared block storage, node balancers, and a 40Gbit network, all controlled by a brand new API you’ve got everything you need to scale up. Go to podcastinit.com/linode to get a $20 credit and launch a new server in under a minute.
  • Visit the site to subscribe to the show, sign up for the newsletter, and read the show notes. And if you have any questions, comments, or suggestions I would love to hear them. You can reach me on Twitter at @Podcast__init__ or email [email protected])
  • To help other people find the show please leave a review on iTunes, or Google Play Music, tell your friends and co-workers, and share it on social media.
  • Join the community in the new Zulip chat workspace at podcastinit.com/chat
  • Your host as usual is Tobias Macey and today I’m interviewing Anthony Sottile about pre-commit, a framework for managing and maintaining hooks for multiple languages

Interview

  • Introductions
  • How did you get introduced to Python?
  • Can you start by describing what a pre-commit hook is and some of the ways that they are useful for developers?
  • What was you motivation for creating a framework to manage your pre-commit hooks?
    • How does it differ from other projects built to manage these hooks?
  • What are the steps for getting someone started with pre-commit in a new project?
  • Which other event hooks would be most useful to implement for maintaining the health of a repository?
  • What types of operations are most useful for ensuring the health of a project?
  • What types of routines should be avoided as a pre-commit step?
  • Installing the hooks into a user’s local environment is a manual step, so how do you ensure that all of your developers are using the configured hooks?
    • What factors have you found that lead to developers skipping or disabling hooks?
  • How is pre-commit implemented and how has that design evolved from when you first started?
    • What have been the most difficult aspects of supporting multiple languages and package managers?
    • What would you do differently if you started over today?
    • Would you still use Python?
  • For someone who wants to write a plugin for pre-commit, what are the steps involved?
  • What are some of the strangest or most unusual uses of pre-commit hooks that you have seen?
  • What are your plans for the future of pre-commit?

Keep In Touch

Picks

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Infection Monkey Vulnerability Scanner with Daniel Goldberg - Episode 177

Summary

How secure are your servers? The best way to be sure that your systems aren’t being compromised is to do it yourself. In this episode Daniel Goldberg explains how you can use his project Infection Monkey to run a scan of your infrastructure to find and fix the vulnerabilities that can be taken advantage of. He also discusses his reasons for building it in Python, how it compares to other security scanners, and how you can get involved to keep making it better.

Preface

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • When you’re ready to launch your next app you’ll need somewhere to deploy it, so check out Linode. With private networking, shared block storage, node balancers, and a 40Gbit network, all controlled by a brand new API you’ve got everything you need to scale up. Go to podcastinit.com/linode to get a $20 credit and launch a new server in under a minute.
  • Visit the site to subscribe to the show, sign up for the newsletter, and read the show notes. And if you have any questions, comments, or suggestions I would love to hear them. You can reach me on Twitter at @Podcast__init__ or email [email protected])
  • To help other people find the show please leave a review on iTunes, or Google Play Music, tell your friends and co-workers, and share it on social media.
  • Join the community in the new Zulip chat workspace at podcastinit.com/chat
  • Your host as usual is Tobias Macey and today I’m interviewing Daniel Goldberg about Infection Monkey, an open source system breach simulation tool for evaluating the security of your network

Interview

  • Introductions
  • How did you get introduced to Python?
  • What is infection monkey and what was the reason for building it?
    • What was the reasoning for building it in Python?
    • If you were to start over today what would you do differently?
  • Penetration testing is typically an endeavor that requires a significant amount of knowledge and experience of security practices. What have been some of the most difficult aspects of building an automated vulnerability testing system?
    • How does a deployed instance keep up to date with recent exploits and attack vectors?
  • How does Infection Monkey compare to other tools such as Nessus and Nexpose?
  • What are some examples of the types of vulnerabilities that can be discovered by Infection Monkey?
  • What kinds of information can Infection Monkey discover during a scan?
    • How does that information get reported to the user?
    • How much security experience is necessary to understand and address the findings in a given report generated from a scan?
  • What techniques do you use to ensure that the simulated compromises can be safely reverted?
  • What are some aspects of network security and system vulnerabilities that Infection Monkey is unable to detect and/or analyze?
  • For someone who is interested in using Infection Monkey what are the steps involved in getting it set up?
    • What is the workflow for running a scan?
    • Is Infection Monkey intended to be run continuously, or only with the interaction of an operator?
  • What are your plans for the future of Infection Monkey?

Keep In Touch

Picks

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Fast Stream Processing In Python Using Faust with Ask Solem - Episode 176

Summary

The need to process unbounded and continually streaming sources of data has become increasingly common. One of the popular platforms for implementing this is Kafka along with its streams API. Unfortunately, this requires all of your processing or microservice logic to be implemented in Java, so what’s a poor Python developer to do? If that developer is Ask Solem of Celery fame then the answer is, help to re-implement the streams API in Python. In this episode Ask describes how Faust got started, how it works under the covers, and how you can start using it today to process your fast moving data in easy to understand Python code. He also discusses ways in which Faust might be able to replace your Celery workers, and all of the pieces that you can replace with your own plugins.

Preface

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • When you’re ready to launch your next app you’ll need somewhere to deploy it, so check out Linode. With private networking, shared block storage, node balancers, and a 40Gbit network, all controlled by a brand new API you’ve got everything you need to scale up. Go to podcastinit.com/linode to get a $20 credit and launch a new server in under a minute.
  • Visit the site to subscribe to the show, sign up for the newsletter, and read the show notes. And if you have any questions, comments, or suggestions I would love to hear them. You can reach me on Twitter at @Podcast__init__ or email [email protected])
  • To help other people find the show please leave a review on iTunes, or Google Play Music, tell your friends and co-workers, and share it on social media.
  • Join the community in the new Zulip chat workspace at podcastinit.com/chat
  • Your host as usual is Tobias Macey and today I’m interviewing Ask Solem about Faust, a library for building high performance, high throughput streaming systems in Python

Interview

  • Introductions
  • How did you get introduced to Python?
  • What is Faust and what was your motivation for building it?
    • What were the initial project requirements that led you to use Kafka as the primary infrastructure component for Faust?
  • Can you describe the architecture for Faust and how it has changed from when you first started writing it?
    • What mechanism does Faust use for managing consensus and failover among instances that are working on the same stream partition?
  • What are some of the lessons that you learned while building Celery that were most useful to you when designing Faust?
  • What have you found to be the most common areas of confusion for people who are just starting to build an application on top of Faust?
  • What has been the most interesting/unexpected/difficult aspects of building and maintaining Faust?
  • What have you found to be the most challenging aspects of building streaming applications?
  • What was the reason for releasing Faust as an open source project rather than keeping it internal to Robinhood?
  • What would be involved in adding support for alternate queue or stream implementations?
  • What do you have planned for the future of Faust?

Keep In Touch

Picks

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Continuous Delivery For Complex Systems Using Zuul with Monty Taylor - Episode 172

Summary

Continuous integration systems are important for ensuring that you don’t release broken software. Some projects can benefit from simple, standardized platforms, but as you grow or factor in additional projects the complexity of checking your deployments grows. Zuul is a deployment automation and gating system that was built to power the complexities of OpenStack so it will grow and scale with you. In this episode Monty Taylor explains how he helped start Zuul, how it is designed for scale, and how you can start using it for your continuous delivery systems. He also discusses how Zuul has evolved and the directions it will take in the future.

Preface

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • When you’re ready to launch your next app you’ll need somewhere to deploy it, so check out Linode. With private networking, shared block storage, node balancers, and a 200Gbit network, all controlled by a brand new API you’ve got everything you need to scale up. Go to podcastinit.com/linode to get a $20 credit and launch a new server in under a minute.
  • Visit the site to subscribe to the show, sign up for the newsletter, and read the show notes. And if you have any questions, comments, or suggestions I would love to hear them. You can reach me on Twitter at @Podcast__init__ or email [email protected])
  • To help other people find the show please leave a review on iTunes, or Google Play Music, tell your friends and co-workers, and share it on social media.
  • Join the community in the new Zulip chat workspace at podcastinit.com/chat
  • Your host as usual is Tobias Macey and today I’m interviewing Monty Taylor about Zuul, a platform that drives continuous integration, delivery, and deployment systems with a focus on project gating and interrelated projects.

Interview

  • Introductions
  • How did you get introduced to Python?
  • Can you start by explaining what Zuul is and how the project got started?
  • How do you view Zuul in the broader landscape of CI/CD systems (e.g. GoCD, Jenkins, Travis, etc.)?
  • What is the workflow for someone who is defining a pipeline in Zuul?
    • How are the pipelines tested and promoted?
    • One of the problems that are often encountered in CI/CD systems is the difficulty of testing changes locally. What kind of support is available in Zuul for that?
  • Can you describe the project architecture?
    • What aspects of the architecture enable it to scale to large projects and teams?
  • How difficult would it be to swap the Ansible integration for another orchestration tool?
  • What would be involved in adding support for additional version control systems?
  • What are your plans for the future of the project?

Keep In Touch

Picks

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA