Infrastructure

Gnocchi: A Scalable Time Series Database For Your Metrics with Julien Danjou - Episode 189

Summary

Do you know what your servers are doing? If you have a metrics system in place then the answer should be “yes”. One critical aspect of that platform is the timeseries database that allows you to store, aggregate, analyze, and query the various signals generated by your software and hardware. As the size and complexity of your systems scale, so does the volume of data that you need to manage which can put a strain on your metrics stack. Julien Danjou built Gnocchi during his time on the OpenStack project to provide a time oriented data store that would scale horizontally and still provide fast queries. In this episode he explains how the project got started, how it works, how it compares to the other options on the market, and how you can start using it today to get better visibility into your operations.

Preface

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • When you’re ready to launch your next app or want to try a project you hear about on the show, you’ll need somewhere to deploy it, so check out Linode. With 200 Gbit/s private networking, scalable shared block storage, node balancers, and a 40 Gbit/s public network, all controlled by a brand new API you’ve got everything you need to scale up. Go to pythonpodcast.com/linode to get a $20 credit and launch a new server in under a minute.
  • And to keep track of how your team is progressing on building new features and squashing bugs, you need a project management system designed by software engineers, for software engineers. Clubhouse lets you craft a workflow that fits your style, including per-team tasks, cross-project epics, a large suite of pre-built integrations, and a simple API for crafting your own. Podcast.__init__ listeners get 2 months free on any plan by going to pythonpodcast.com/clubhouse today and signing up for a trial.
  • Visit the site to subscribe to the show, sign up for the newsletter, and read the show notes. And if you have any questions, comments, or suggestions I would love to hear them. You can reach me on Twitter at @Podcast__init__ or email [email protected])
  • To help other people find the show please leave a review on iTunes, or Google Play Music, tell your friends and co-workers, and share it on social media.
  • Join the community in the new Zulip chat workspace at pythonpodcast.com/chat
  • Your host as usual is Tobias Macey and today I’m interviewing Julien Danjou about Gnocchi, an open source time series database built to handle large volumes of system metrics

Interview

  • Introductions
  • How did you get introduced to Python?
  • Can you start by describing what Gnocchi is and how the project got started?
    • What was the motivation for moving Gnocchi out of the Openstack organization and into its own top level project?
  • The space of time series databases and metrics as a service platforms are both fairly crowded. What are the unique features of Gnocchi that would lead someone to deploy it in place of other options?
    • What are some of the tools and platforms that are popular today which hadn’t yet gained visibility when you first began working on Gnocchi?
  • How is Gnocchi architected?
    • How has the design changed since you first started working on it?
    • What was the motivation for implementing it in Python and would you make the same choice today?
  • One of the interesting features of Gnocchi is its support of resource history. Can you describe how that operates and the types of use cases that it enables?
    • Does that factor into the multi-tenant architecture?
  • What are some of the drawbacks of pre-aggregating metrics as they are being written into the storage layer (e.g. loss of fidelity)?
    • Is it possible to maintain the raw measures after they are processed into aggregates?
  • One of the challenging aspects of building a scalable metrics platform is support for high-cardinality data. What sort of labelling and tagging of metrics and measures is available in Gnocchi?
  • For someone who wants to implement Gnocchi for their system metrics, what is involved in deploying, maintaining, and upgrading it?
    • What are the available integration points for extending and customizing Gnocchi?
  • Once metrics have been stored, aggregated, and indexed, what are the options for querying and analyzing the collected data?
  • When is Gnocchi the wrong choice?
  • What do you have planned for the future of Gnocchi?

Keep In Touch

Picks

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Infection Monkey Vulnerability Scanner with Daniel Goldberg - Episode 177

Summary

How secure are your servers? The best way to be sure that your systems aren’t being compromised is to do it yourself. In this episode Daniel Goldberg explains how you can use his project Infection Monkey to run a scan of your infrastructure to find and fix the vulnerabilities that can be taken advantage of. He also discusses his reasons for building it in Python, how it compares to other security scanners, and how you can get involved to keep making it better.

Preface

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • When you’re ready to launch your next app you’ll need somewhere to deploy it, so check out Linode. With private networking, shared block storage, node balancers, and a 40Gbit network, all controlled by a brand new API you’ve got everything you need to scale up. Go to podcastinit.com/linode to get a $20 credit and launch a new server in under a minute.
  • Visit the site to subscribe to the show, sign up for the newsletter, and read the show notes. And if you have any questions, comments, or suggestions I would love to hear them. You can reach me on Twitter at @Podcast__init__ or email [email protected])
  • To help other people find the show please leave a review on iTunes, or Google Play Music, tell your friends and co-workers, and share it on social media.
  • Join the community in the new Zulip chat workspace at podcastinit.com/chat
  • Your host as usual is Tobias Macey and today I’m interviewing Daniel Goldberg about Infection Monkey, an open source system breach simulation tool for evaluating the security of your network

Interview

  • Introductions
  • How did you get introduced to Python?
  • What is infection monkey and what was the reason for building it?
    • What was the reasoning for building it in Python?
    • If you were to start over today what would you do differently?
  • Penetration testing is typically an endeavor that requires a significant amount of knowledge and experience of security practices. What have been some of the most difficult aspects of building an automated vulnerability testing system?
    • How does a deployed instance keep up to date with recent exploits and attack vectors?
  • How does Infection Monkey compare to other tools such as Nessus and Nexpose?
  • What are some examples of the types of vulnerabilities that can be discovered by Infection Monkey?
  • What kinds of information can Infection Monkey discover during a scan?
    • How does that information get reported to the user?
    • How much security experience is necessary to understand and address the findings in a given report generated from a scan?
  • What techniques do you use to ensure that the simulated compromises can be safely reverted?
  • What are some aspects of network security and system vulnerabilities that Infection Monkey is unable to detect and/or analyze?
  • For someone who is interested in using Infection Monkey what are the steps involved in getting it set up?
    • What is the workflow for running a scan?
    • Is Infection Monkey intended to be run continuously, or only with the interaction of an operator?
  • What are your plans for the future of Infection Monkey?

Keep In Touch

Picks

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Continuous Delivery For Complex Systems Using Zuul with Monty Taylor - Episode 172

Summary

Continuous integration systems are important for ensuring that you don’t release broken software. Some projects can benefit from simple, standardized platforms, but as you grow or factor in additional projects the complexity of checking your deployments grows. Zuul is a deployment automation and gating system that was built to power the complexities of OpenStack so it will grow and scale with you. In this episode Monty Taylor explains how he helped start Zuul, how it is designed for scale, and how you can start using it for your continuous delivery systems. He also discusses how Zuul has evolved and the directions it will take in the future.

Preface

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • When you’re ready to launch your next app you’ll need somewhere to deploy it, so check out Linode. With private networking, shared block storage, node balancers, and a 200Gbit network, all controlled by a brand new API you’ve got everything you need to scale up. Go to podcastinit.com/linode to get a $20 credit and launch a new server in under a minute.
  • Visit the site to subscribe to the show, sign up for the newsletter, and read the show notes. And if you have any questions, comments, or suggestions I would love to hear them. You can reach me on Twitter at @Podcast__init__ or email [email protected])
  • To help other people find the show please leave a review on iTunes, or Google Play Music, tell your friends and co-workers, and share it on social media.
  • Join the community in the new Zulip chat workspace at podcastinit.com/chat
  • Your host as usual is Tobias Macey and today I’m interviewing Monty Taylor about Zuul, a platform that drives continuous integration, delivery, and deployment systems with a focus on project gating and interrelated projects.

Interview

  • Introductions
  • How did you get introduced to Python?
  • Can you start by explaining what Zuul is and how the project got started?
  • How do you view Zuul in the broader landscape of CI/CD systems (e.g. GoCD, Jenkins, Travis, etc.)?
  • What is the workflow for someone who is defining a pipeline in Zuul?
    • How are the pipelines tested and promoted?
    • One of the problems that are often encountered in CI/CD systems is the difficulty of testing changes locally. What kind of support is available in Zuul for that?
  • Can you describe the project architecture?
    • What aspects of the architecture enable it to scale to large projects and teams?
  • How difficult would it be to swap the Ansible integration for another orchestration tool?
  • What would be involved in adding support for additional version control systems?
  • What are your plans for the future of the project?

Keep In Touch

Picks

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

The Pulp Artifact Repository with Bihan Zhang and Austin Macdonald - Episode 168

Summary

Hosting your own artifact repositories can have a huge impact on the reliability of your production systems. It reduces your reliance on the availability of external services during deployments and ensures that you have access to a consistent set of dependencies with known versions. Many repositories only support one type of package, thereby requiring multiple systems to be maintained, but Pulp is a platform that handles multiple content types and is easily extendable to manage everything you need for running your applications. In this episode maintainers Bihan Zhang and Austin Macdonald explain how the Pulp project works, the exciting new changes coming in version 3, and how you can get it set up to use for your deployments today.

Preface

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • When you’re ready to launch your next app you’ll need somewhere to deploy it, so check out Linode. With private networking, shared block storage, node balancers, and a 200Gbit network, all controlled by a brand new API you’ve got everything you need to scale up. Go to podcastinit.com/linode to get a $20 credit and launch a new server in under a minute.
  • To get worry-free releases download GoCD, the open source continous delivery server built by Thoughworks. You can use their pipeline modeling and value stream map to build, control and monitor every step from commit to deployment in one place. And with their new Kubernetes integration it’s even easier to deploy and scale your build agents. Go to podcastinit.com/gocd to learn more about their professional support services and enterprise add-ons.
  • Visit the site to subscribe to the show, sign up for the newsletter, and read the show notes. And if you have any questions, comments, or suggestions I would love to hear them. You can reach me on Twitter at @Podcast__init__ or email [email protected])
  • To help other people find the show please leave a review on iTunes, or Google Play Music, tell your friends and co-workers, and share it on social media.
  • Your host as usual is Tobias Macey and today I’m interviewing Austin Macdonald and Bihan Zhang about Pulp, a platform for hosting and managing software package repositories

Interview

  • Introductions
  • How did you get introduced to Python?
  • What is Pulp and how did the project get started?
  • What are the use cases/benefits for hosting your own artifact repository?
  • What is the high level architecture of the platform?
    • Pulp 3 appears to be a fairly substantial change in architecture and design. What will be involved in migrating an existing installation to the new version when it is released?
  • What is involved in adding support for a new type of artifact/package?
  • How does Pulp compare to other artifact repositories?
  • What are the major pieces of work that are required before releasing Pulp 3?
  • What have been some of the most interesting/unexpected/challenging aspects of building and maintaining Pulp?
  • What are your plans for the future of Pulp?

Keep In Touch

Picks

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Scaling Deep Learning Using Polyaxon with Mourad Mourafiq - Episode 158

Summary

With libraries such as Tensorflow, PyTorch, scikit-learn, and MXNet being released it is easier than ever to start a deep learning project. Unfortunately, it is still difficult to manage scaling and reproduction of training for these projects. Mourad Mourafiq built Polyaxon on top of Kubernetes to address this shortcoming. In this episode he shares his reasons for starting the project, how it works, and how you can start using it today.

Preface

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • When you’re ready to launch your next app you’ll need somewhere to deploy it, so check out Linode. With private networking, shared block storage, node balancers, and a 200Gbit network, all controlled by a brand new API you’ve got everything you need to scale up. Go to podcastinit.com/linode to get a $20 credit and launch a new server in under a minute.
  • Finding a bug in production is never a fun experience, especially when your users find it first. Airbrake error monitoring ensures that you will always be the first to know so you can deploy a fix before anyone is impacted. With open source agents for Python 2 and 3 it’s easy to get started, and the automatic aggregations, contextual information, and deployment tracking ensure that you don’t waste time pinpointing what went wrong. Go to podcastinit.com/airbrake today to sign up and get your first 30 days free, and 50% off 3 months of the Startup plan.
  • To get worry-free releases download GoCD, the open source continous delivery server built by Thoughworks. You can use their pipeline modeling and value stream map to build, control and monitor every step from commit to deployment in one place. And with their new Kubernetes integration it’s even easier to deploy and scale your build agents. Go to podcastinit.com/gocd to learn more about their professional support services and enterprise add-ons.
  • Visit the site to subscribe to the show, sign up for the newsletter, and read the show notes. And if you have any questions, comments, or suggestions I would love to hear them. You can reach me on Twitter at @Podcast__init__ or email [email protected])
  • Your host as usual is Tobias Macey and today I’m interviewing Mourad Mourafiq about Polyaxon, a platform for building, training and monitoring large scale deep learning applications.

Interview

  • Introductions
  • How did you get introduced to Python?
  • Can you give a quick overview of what Polyaxon is and your motivation for creating it?
  • What is a typical workflow for building and testing a deep learning application?
  • How is Polyaxon implemented?
    • How has the internal architecture evolved since you first started working on it?
    • What is unique to deep learning workloads that makes it necessary to have a dedicated tool for deploying them?
    • What does Polyaxon add on top of the existing functionality in Kubernetes?
  • It can be difficult to build a docker container that holds all of the necessary components for a complex application. What are some tips or best practices for creating containers to be used with Polyaxon?
  • What are the relative tradeoffs of the various deep learning frameworks that you support?
  • For someone who is getting started with Polyaxon what does the workflow look like?
    • What is involved in migrating existing projects to run on Polyaxon?
  • What have been the most challenging aspects of building Polyaxon?
  • What are your plans for the future of Polyaxon?

Keep In Touch

Picks

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Healthchecks.io: Open Source Alerting For Your Cron Jobs with Pēteris Caune - Episode 144

Summary

Your backups are running every day, right? Are you sure? What about that daily report job? We all have scripts that need to be run on a periodic basis and it is easy to forget about them, assuming that they are working properly. Sometimes they fail and in order to know when that happens you need a tool that will let you know so that you can find and fix the problem. Pēteris Caune wrote Healthchecks to be that tool and made it available both as an open source project and a hosted version. In this episode he discusses his motivation for starting the project, the lessons he has learned while managing the hosting for it, and how you can start using it today.

Preface

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • I would like to thank everyone who supports us on Patreon. Your contributions help to make the show sustainable.
  • When you’re ready to launch your next project you’ll need somewhere to deploy it. Check out Linode at podastinit.com/linode and get a $20 credit to try out their fast and reliable Linux virtual servers for running your awesome app. And now you can deliver your work to your users even faster with the newly upgraded 200 GBit network in all of their datacenters.
  • If you’re tired of cobbling together your deployment pipeline then it’s time to try out GoCD, the open source continuous delivery platform built by the people at ThoughtWorks who wrote the book about it. With GoCD you get complete visibility into the life-cycle of your software from one location. To download it now go to podcatinit.com/gocd. Professional support and enterprise plugins are available for added piece of mind.
  • Visit the site to subscribe to the show, sign up for the newsletter, and read the show notes. And if you have any questions, comments, or suggestions I would love to hear them. You can reach me on Twitter at @Podcast__init__ or email [email protected])
  • To help other people find the show please leave a review on iTunes, or Google Play Music, tell your friends and co-workers, and share it on social media.
  • Your host as usual is Tobias Macey and today I’m interviewing Pēteris Caune about Healthchecks, a Django app which serves as a watchdog for your cron tasks

Interview

  • Introductions
  • How did you get introduced to Python?
  • Can you start by explaining what Healthchecks is and what motivated you to build it?
  • How does Healthchecks compare with other cron monitoring projects such as Cronitor or Dead Man’s Snitch?
  • Your pricing on the hosted service for Healthchecks.io is quite generous so I’m curious how you arrived at that cost structure and whether it has proven to be profitable for you?
  • How is Healthchecks functionality implemented and how has the design evolved since you began working on and using it?
  • What have been some of the most challenging aspects of working on Healthchecks and managing the hosted version?
  • For someone who wants to run their own instance of the service what are the steps and services involved?
  • What are some of the most interesting or unusual uses of Healtchecks that you are aware of?
  • Given that Healthchecks is intended to be used as part of an operations management and alerting system, what are the considerations that users should be aware of when deploying it in a highly available configuration?
  • What improvements or features do you have planned for the future of Healthchecks?

Keep In Touch

Picks

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Graphite Metrics Stack with Jason Dixon and Dan Cech - Episode 136

Summary

Do you know what is happening in your production systems right now? If you have a comprehensive metrics platform then the answer is yes. If your answer is no, then this episode is for you. Jason Dixon and Dan Cech, core maintainers of the Graphite project, talk about how graphite is architected to capture your time series data and give you the ability to use it for answering questions. They cover the challenges that have been faced in evolving the project, the strengths that have let it stand the tests of time, and the features that will be coming in future releases.

Preface

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • I would like to thank everyone who supports us on Patreon. Your contributions help to make the show sustainable.
  • When you’re ready to launch your next project you’ll need somewhere to deploy it. Check out Linode at podastinit.com/linode and get a $20 credit to try out their fast and reliable Linux virtual servers for running your awesome app. And now you can deliver your work to your users even faster with the newly upgraded 200 GBit network in all of their datacenters.
  • If you’re tired of cobbling together your deployment pipeline then it’s time to try out GoCD, the open source continuous delivery platform built by the people at ThoughtWorks who wrote the book about it. With GoCD you get complete visibility into the life-cycle of your software from one location. To download it now go to podcatinit.com/gocd. Professional support and enterprise plugins are available for added piece of mind.
  • Visit the site to subscribe to the show, sign up for the newsletter, and read the show notes. And if you have any questions, comments, or suggestions I would love to hear them. You can reach me on Twitter at @Podcast__init__ or email [email protected])
  • To help other people find the show please leave a review on iTunes, or Google Play Music, tell your friends and co-workers, and share it on social media.
  • Now is a good time to start planning your conference schedule for 2018. To help you out with that, guest Jason Dixon is offering a $100 discount for Monitorama in Portland, OR on June 4th – 6th and guest Dan Cech is offering a €50 discount to Grafanacon in Amsterdam, Netherlands March 1st and 2nd. There is also still time to get your tickets to PyCascades in Vancouver, BC Canada January 22nd and 23rd. All of the details are in the show notes
  • Your host as usual is Tobias Macey and today I’m interviewing Jason Dixon and Dan Cech about Graphite

Interview

  • Introductions
  • How did you get introduced to Python?
  • What is Graphite and how did you each get involved in the project?
  • Why should developers be thinking about collecting and reporting on metrics from their software and systems?
  • How do you think the Graphite project has contributed to or influenced the overall state of the art in systems monitoring?
  • There are a number of different projects that comprise a fully working Graphite deployment. Can you list each of them and describe how they fit together?
  • What are some of the early design choices that have proven to be problematic while trying to evolve the project?
  • What are some of the challenges that you have been faced with while maintaining and improving the various Graphite projects?
  • What will be involved in porting Graphite to run on Python 3?
  • If you were to start the project over would you still use Python?
  • What are the options for scaling Graphite and making it highly available?
  • Given the level of importance to a companies visibility into their systems, what development practices do you use to ensure that Graphite can operate reliably and fail gracefully?
  • What are some of the biggest competitors to Graphite?
  • When is Graphite not the right choice for tracking your system metrics?
  • What are some of the most interesting or unusual uses of Graphite that you are aware of?
  • What are some of the new features and enhancements that are planned for the future of Graphite?

Keep In Touch

Picks

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Cloud-Init with Scott Moser - Episode 126

Summary

Server administration is a complex endeavor, but there are some tools that can make life easier. If you are running your workload in a cloud environment then cloud-init is here to help. This week Scott Moser explains what cloud-init is, how it works, and how it became the de-facto tool for configuring your Linux servers at boot.

Preface

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • I would like to thank everyone who supports us on Patreon. Your contributions help to make the show sustainable.
  • When you’re ready to launch your next project you’ll need somewhere to deploy it. Check out Linode at www.podastinit.com/linode and get a $20 credit to try out their fast and reliable Linux virtual servers for running your awesome app.
  • Need to learn more about how to scale your apps or learn new techniques for building them? Pluralsight has the training and mentoring you need to level up your skills. Go to www.podcastinit.com/pluralsight to start your free trial today.
  • Visit the site to subscribe to the show, sign up for the newsletter, read the show notes, and get in touch.
  • To help other people find the show please leave a review on iTunes, or Google Play Music, tell your friends and co-workers, and share it on social media.
  • If you work with data for your job or want to learn more about how open source is powering the latest innovations in data science then make your way to the Open Data Science Conference, happening in London in October and San Francisco in November. Follow the links in the show notes to register and help support the show in the process.
  • Your host as usual is Tobias Macey and today I’m interviewing Scott Moser about cloud-init, a set of python scripts and utilities to make your cloud images be all they can be!

Interview

  • Introductions
  • How did you get introduced to Python?
  • What is cloud-init and how did the project get started?
  • Why was Python chosen as the language for implementing cloud-init?
  • How has cloud-init come to be the de-facto utility for managing cloud instances across vendors and distributions?
  • Are there any viable competitors to cloud-init? coreos-cloudinit, some others.
  • How much overlap is there between cloud-init and configuration management tools such as SaltStack, Ansible, Chef, etc.
  • How have you architected cloud-init to allow for compatibility across operating system distributions?
  • What is the most difficult or complex aspect of building and maintaining cloud-init? [os integration, networking, goal of “do stuff without reboot”]
  • Given that it is used as a critical component of the production deployment mechanics for a large number of people, how do you ensure an appropriate level of stability and security while developing cloud-init?
  • How do you think the status of cloud-init as a Canonical project has affected the level of contributions that you receive?
  • How much of the support and roadmap is contributed by individual vs corporate users such as AWS and Azure?
  • What are some of the most unexpected or creative uses of cloud-init that you have seen? [https://wiki.ubuntu.com/OpenCompute “disposable use os”]
  • In your experience, what has been the biggest stumbling block for new users of cloud-init?
  • Do you have any notable features or improvements planned for the future of cloud-init, or do you feel that it has reached a state of feature-completeness?

Keep In Touch

Picks

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Duplicity with Kenneth Loafman - Episode 103

Summary

Everyone who uses a computer on a regular basis knows the importance of backups. Duplicity is one of the most widely used backup technologies, and it’s written in Python! This week Kenneth Loafman shares how Duplicity got started, how it works, and why you should be using it every day.

Preface

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • I would like to thank everyone who has donated to the show. Your contributions help us make the show sustainable.
  • When you’re ready to launch your next project you’ll need somewhere to deploy it. Check out Linode at linode.com/podcastinit and get a $20 credit to try out their fast and reliable Linux virtual servers for running your awesome app.
  • Visit our site to subscribe to our show, sign up for our newsletter, read the show notes, and get in touch.
  • To help other people find the show you can leave a review on iTunes, or Google Play Music, and tell your friends and co-workers
  • Your host as usual is Tobias Macey and today I’m interviewing Kenneth Loafman about Duplicity, the Python based backup tool

Interview

  • Introduction
  • How did you get introduced to Python?
  • Can you share some of the history of Duplicity?
  • What is duplicity and how does it differ from other available backup tools?
  • Many backup solutions are written in Java or lower level languages such as C, what is the motivation for using Python as the language for implementing Duplicity?
  • At face value backing up files seems like a straightforward task but there is a lot of incidental complexity. Can you describe the architecture and internals of Duplicity that allow for it to handle a wide variety of use cases?
  • It has been shown in a number of contexts that people will generally use the default settings, so by forcing people to opt out of encrypting their backups you are promoting security best practices in Duplicity. Why is it so important to have the archive encrypted, even if the storage medium is fully under the control of the person doing the backup?
  • Given that backups need to be highly reliable what are the steps that you take during the development process to ensure that there are no regressions?
  • What mechanisms are built into duplicity to prevent data corruption?
  • What are some of the most difficult or complex aspects of the problem space that Duplicity is dealing with?
  • I noticed that you have a proposal for a new archive format to replace Tar. Can you describe the motivation for that and the design choices that have been made?

Contact

Picks

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Crossbar.io with Tobias Oberstein and Alexander Gödde - Episode 101

Summary

As our system architectures and the Internet of Things continue to push us towards distributed logic we need a way to route the traffic between those various components. Crossbar.io is the original implementation of the Web Application Messaging Protocol (WAMP) which combines Remote Procedure Calls (RPC) with Publish/Subscribe (PubSub) communication patterns into a single communication layer. In this episode Tobias Oberstein describes the use cases and design patterns that become possible when you have event-based RPC in a high-throughput and low-latency system.

Preface

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • I would like to thank everyone who has donated to the show. Your contributions help us make the show sustainable.
  • When you’re ready to launch your next project you’ll need somewhere to deploy it. Check out Linode at linode.com/podcastinit and get a $20 credit to try out their fast and reliable Linux virtual servers for running your awesome app.
  • Visit our site to subscribe to our show, sign up for our newsletter, read the show notes, and get in touch.
  • To help other people find the show you can leave a review on iTunes, or Google Play Music, and tell your friends and co-workers
  • Your host as usual is Tobias Macey and today I’m interviewing Tobias Oberstein and Alexander Gödde about Crossbar.io, a high throughput asynchronous router for the WAMP protocol

Interview

  • Introductions
  • How did you get introduced to Python?
  • What is Crossbar and what is the problem that you were trying to solve when you created it?
  • What is the status of the IETF WAMP protocol proposal?
  • Why have an open protocol – and how do you see the ecosystem?
  • Python isn’t typically considered to be a high-performance language so what led you to use it for building Crossbar?
  • How is Crossbar architected for proxying requests from a highly distributed set of clients with low latency and high throughput?
  • How do you handle authorization between the various clients of the router so that potentially sensitive messages don’t get published to the wrong component?
  • Does Crossbar encapsulate any business logic or is that all pushed to the edges of the system?
  • What are some of the typical kinds of applications that Crossbar is designed for?
  • What are some common design paradigms that would be better suited for a WAMP implementation?
  • What are some of the most interesting or surprising uses of Crossbar that you have seen?
  • What do you have planned for the future of Crossbar?

Keep In Touch

Picks

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA