Technology

Pip and the Python Package Authority with Donald Stufft - Episode 54

Visit our site to listen to past episodes, support the show, join our community, and sign up for our mailing list.

Summary

As Python developers we have all used pip to install the different libraries and projects that we need for our work, but have you ever wondered about who works on pip and how the package archive we all know and love is maintained? In this episode we interviewed Donald Stufft who is the primary maintainer of pip and the Python Package Index about how he got involved with the projects, what kind of work is involved, and what is on the roadmap. Give it a listen and then give him a big thank you for all of his hard work!

Brief Introduction

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • Subscribe on iTunes, Stitcher, TuneIn or RSS
  • Google Play Music just launched support for podcasts, so now you can check us out there and subscribe to the show.
  • Follow us on Twitter or Google+
  • Give us feedback! Leave a review on iTunes, Tweet to us, send us an email or leave us a message on Google+
  • Join our community! Visit discourse.pythonpodcast.com for your opportunity to find out about upcoming guests, suggest questions, and propose show ideas.
  • I would like to thank everyone who has donated to the show. Your contributions help us make the show sustainable. For details on how to support the show you can visit our site at pythonpodcast.com
  • Linode is sponsoring us this week. Check them out at linode.com/podcastinit and get a $20 credit to try out their fast and reliable Linux virtual servers for your next project
  • We also have a new sponsor this week. Rollbar is a service for tracking and aggregating your application errors so that you can fix the bugs in your application before your users notice they exist. Use the link rollbar.com/podcatinit to get 90 days and 300,000 errors for free on their bootstrap plan.
  • The Open Data Science Conference in Boston is happening on May 21st and 22nd. If you use the code EP during registration you will save 20% off of the ticket price. If you decide to attend then let us know, we’ll see you there!
  • Your hosts as usual are Tobias Macey and Chris Patti
  • Today we are interviewing Donald Stufft about Pip and the Python Packaging Authority

Interview with Donald Stufft

  • Introductions
  • How did you get introduced to Python? – Chris
  • How did you get involved with the Pip project? – Tobias
  • What is the Python Packaging Authority and what does it do? – Tobias
  • How is PyPi / the Python Packaging Authority funded? – Chris
  • What is your opinion on the current state of Python packaging? Are there lessons from other languages and package managers that you think should be adopted by Python? – Tobias
  • What was involved in getting pip into the standard Python distribution? Was there any controversy around this? – Chris
  • Can you describe some of the mechanics of Pip and how it differs from the other packaging systems that Python has used in the past? – Tobias
  • Does pip interact at all with virtualenv, pyenv and the like? – Chris
  • The newest package format for Python is the wheel system. Can you describe what that is and what its benefits are? – Tobias
  • What are the biggest challenges that you have encountered while working on Pip? – Tobias
  • What does the infrastructure for the Python Package Index look like? – Tobias
  • What have been some of the challenges around scaling Pypi’s infrastructure to meet demand? – Chris
  • You’re currently working on a replacement for the PyPI site with the Warehouse project. Can you explain your motivation for that and how it improves on the current system? – Tobias
  • Where do you see the future of dependency management in Python headed? – Chris
  • A few days ago there was a big story about how an NPM library was removed from the index, breaking a large number of dependent projects and applications. Do you think that anything like that could happen in the Python ecosystem? – Tobias
  • What’s on the roadmap for Pip? – Tobias

Keep In Touch

Picks

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

StackStorm with Tomaž Muraus and Patrick Hoolboom - Episode 53

Visit our site to listen to past episodes, support the show, join our community, and sign up for our mailing list.

Summary

If you are responsible for managing any amount of servers, then you know that automation is critical for maintaining your sanity. This week we spoke with Tomaž Muraus and Patrick Hoolboom about their work on StackStorm, which is a platform for tracking and reacting to events in your infrastructure. By allowing you to register actions with event triggers it frees you from having to worry about a whole class of concerns so that you can focus on building new capabilities rather than babysitting what you already have.

Brief Introduction

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • Subscribe on iTunes, Stitcher, TuneIn or RSS
  • Follow us on Twitter or Google+
  • Give us feedback! Leave a review on iTunes, Tweet to us, send us an email or leave us a message on Google+
  • Join our community! Visit discourse.pythonpodcast.com for your opportunity to find out about upcoming guests, suggest questions, and propose show ideas.
  • I would like to thank everyone who has donated to the show. Your contributions help us make the show sustainable. For details on how to support the show you can visit our site at pythonpodcast.com
  • Linode is sponsoring us this week. Check them out at linode.com/podcastinit and get a $20 credit to try out their fast and reliable Linux virtual servers for your next project
  • I would also like to thank Hired, a job marketplace for developers and designers, for sponsoring this episode of Podcast.__init__. Use the link hired.com/podcastinit to double your signing bonus.
  • ODSC East in Boston is happening on May 21st – 22nd. Use the discount code EP for 20% off when you register
  • Your hosts as usual are Tobias Macey and Chris Patti
  • Today we are interviewing Tomaž Muraus and Patrick Hoolboom about the StackStorm project, which is an event-driven system automation framework.

Interview

  • Introductions
  • How did you get introduced to Python? – Chris
  • What is StackStorm and what problems does it solve? – Tobias
  • What was your inspiration for creating StackStorm and what were some of the biggest architectural and design challenges? – Tobias
  • What made you choose Python for StackStorm’s implementation rather than another language like Go? – Chris
  • Can you describe the architecture of StackStorm and what the setup looks like? – Tobias
  • Other than chat driven events, what types of event sources does StackStorm support, and what use cases do those alternate event streams enable? – Chris
  • The home page describes StackStorm as being an event-driven framework for automating the users infrastructure. What kinds of capabilities are made possible by this and do you think that it simplifies or complicates the work of operations engineers? – Tobias
  • Is there a minimum or maximum size of infrastructure for which it would make sense to use StackStorm? – Tobias
  • It looks like StackStorm is made up of a number of discrete components. What do the components use to communicate, and how did those choices influence the design of StackStorm’s overall architecture? – Chris
  • I use SaltStack in my work which is a tool that also focuses on event-driven architecture. Can you compare and contrast the capabilities and focus of StackStorm with the features of SaltStack? Would it make sense to use both frameworks in the same infrastructure? – Tobias
  • One of the advertised features of StackStorm is a strong focus on ChatOps. Can you explain that concept for people who might not be familiar with it and describe why it is such a useful paradigm? – Tobias
  • Extensibility is a critical capability for an operations platform due to the wide variety of environments that people are inclined to build. In StackStorm the unit of extensibility is a pack. Can you describe what a pack is and how you arrived at that abstraction? – Tobias
    • Have you encountered any situations in which the concept of a pack has been the wrong abstraction and made something more difficult than it may have been otherwise? – Tobias
  • In very large scale environments like Netflix, how would one build a StackStorm cluster to handle the immense load. More specifically, how does one determine what kinds of machine resources each component needs? – Chris
  • Management of credentials is always a difficult problem in operations. Does StackStorm attempt to tackle that issue or does it defer that responsibility to other systems, such as the user’s configuration management platform? – Tobias
  • Does StackStorm interface with Kibana, Splunk or other log / metric aggregation packages? – Chris
  • What are some of the most surprising uses that you have heard of from people using the platform? – Tobias

Keep In Touch

Picks

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Pyjion with Dino Viehland and Brett Cannon - Episode 51

Visit our site to listen to past episodes, support the show, join our community, and sign up for our mailing list.

Summary

In an attempt to improve the performance characteristics of the CPython implementation, Dino Viehland began work on a patch to allow for a pluggable interface to a JIT (Just In Time) compiler. His employer, Microsoft, decided to sponsor his efforts and the result is the Pyjion project. In this episode we spoke with Dino Viehland and Brett Cannon about the goals of the project, the progress they have made so far, and the issues they have encountered along the way. We also made an interesting detour to discuss the general state of performance in the Python ecosystem and why the GIL isn’t the bogeyman it’s made out to be.

Brief Introduction

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • Subscribe on iTunes, Stitcher, TuneIn or RSS
  • Follow us on Twitter or Google+
  • Give us feedback! Leave a review on iTunes, Tweet to us, send us an email or leave us a message on Google+
  • Join our community! Visit discourse.pythonpodcast.com for your opportunity to find out about upcoming guests, suggest questions, and propose show ideas.
  • I would like to thank everyone who has donated to the show. Your contributions help us make the show sustainable. For details on how to support the show you can visit our site at pythonpodcast.com
  • Linode is sponsoring us this week. Check them out at linode.com/podcastinit and get a $20 credit to try out their fast and reliable Linux virtual servers for your next project
  • I would also like to thank Hired, a job marketplace for developers and designers, for sponsoring this episode of Podcast.__init__. Use the link hired.com/podcastinit to double your signing bonus.
  • Your hosts as usual are Tobias Macey and Chris Patti
  • Open Data Science Conference, Boston MA May 21st – 22nd, use the discount code EP at registration for 20% off
  • Today we are interviewing Brett Cannon and Dino Viehland about their work on Pyjion, a CPython extension that provides an API to allow for plugging a JIT compilation engine into the CPython runtime.

Interview with Brett Cannon and Dino Viehland

  • Introductions
  • How did you get introduced to Python? – Chris
  • What was the inspiration for the Pyjion project and what are its goals? – Tobias
  • The FAQ mentions that Pyjion could easily be made cross platform, but this being a Microsoft project it was bootstrapped on Windows. Have any of the discrete tasks required to get Pyjion running under OSX or Linux been laid out even in outline form? – Chris
  • Given that this is a Microsoft backed project it makes sense that the first JIT engine to be implemented is for the CoreCLR. What would an alternative implementation provide and in what ways can a JIT framework be tuned for particular workloads? – Tobias
  • What kinds of use cases and problem domains that were previously impractical will be enabled by this? – Tobias
  • Does Microsoft’s recent acquisition of Xamarin and the Mono project change things for the Pyjion project at all? – Chris
  • What are the challenges associated with your work on Pyjion? Are there certain aspects of the Python language and the CPython implementation that make the work more difficult than it might be otherwise? – Tobias
  • When I think of Microsoft and programming languages I generally think of C++ and C#. Did your team have to go through an approval process in order to utilize Python, and further to open source your work on Pyjion? – Chris
  • How does Pyjion hook into the CPython runtime and what kinds of primitives does it expose to JIT engines for them to be able to work with? – Tobias
  • Would an entire project be run through the JIT engine during runtime or is it possible to target a subset of the code being executed? – Tobias
  • In what ways can a JIT compiler implementation be purpose-built for a given workload and how would someone go about creating one? – Tobias
  • Could a JIT plugin be designed with different trade-offs, like no C API compatibility, but that worked around the GIL to provide real concurrency in Python? – Chris
  • One of the most notable benefits of having a JIT implementation for the CPython runtime is the fact that modules with C extensions can be used, such as NumPy. Does that pose any difficulties in the compilation methods used for optimizing the Python portion of the code? – Tobias
  • What kinds of performance improvements have you seen in your experimentation? – Tobias
  • Which release of Python do you hope to have Pyjion incorporated into? – Tobias
  • Has any thought been given to making Python a first class citizen in Visual Studio Code? – Chris
  • What areas of the project could use some help from our listeners? – Chris

Keep In Touch

Picks

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Transcrypt with Jacques de Hooge - Episode 50

Visit our site to listen to past episodes, support the show, join our community, and sign up for our mailing list.

Summary

Any programmer who has dealt with a website for any length of time knows that writing JavaScript isn’t always the most enjoyable. Wouldn’t you rather write that code in Python and just have it work on your website? In this episode we learn about Transcrypt with its creator Jacques de Hooge. Transcrypt is a Python to JavaScript transpiler that embraces the JavaScript ecosystem while letting you use the familiar syntax of Python for writing your logic, rather than trying to shoehorn a Python runtime into your browser.

Brief Introduction

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • Subscribe on iTunes, Stitcher, TuneIn or RSS
  • Follow us on Twitter or Google+
  • Give us feedback! Leave a review on iTunes, Tweet to us, send us an email or leave us a message on Google+
  • Join our community! Visit discourse.pythonpodcast.com for your opportunity to find out about upcoming guests, suggest questions, and propose show ideas.
  • I would like to thank everyone who has donated to the show. Your contributions help us make the show sustainable. For details on how to support the show you can visit our site at pythonpodcast.com
  • Linode is sponsoring us this week. Check them out at linode.com/podcastinit and get a $20 credit to try out their fast and reliable Linux virtual servers for your next project
  • I would also like to thank Hired, a job marketplace for developers and designers, for sponsoring this episode of Podcast.__init__. Use the link hired.com/podcastinit to double your signing bonus.
  • ODSC East in Boston is happening on May 21st – 22nd. Use the discount code EP for 20% off when you register
  • Your host today is Tobias Macey
  • Today I am interviewing Jacques de Hooge about his work on the Transcrypt Project

Interview with Jacques de Hooge

  • Introductions
  • How did you get introduced to Python? – Tobias
  • What is Transcrypt and what inspired you to create it? – Tobias
  • As you mention in the documentation, there are a number of projects that attempt to shoehorn Python into the browser. What makes Transcrypt different? – Tobias
  • I like that you decided to embrace the web environment by calling into JavaScript libraries. What are some of the challenges that you encountered while creating that functionality? – Tobias
  • How is the transpilation performed and what are some of the methods that you used to get the build size as small as it is? – Tobias
  • Given the nature of JavaScripts prototypical inheritance and differences in class semantics, I imagine that adding support for multiple inheritance and reflecting the structure of Python classes must have been challenging. Can you describe that process and how you arrived at your current solution? – Tobias
  • Which aspects of the language were most difficult to translate to JavaScript? – Tobias
  • Is Transcrypt complete and stable enough to be used in production? – Tobias

Keep in Touch

Picks

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

VPython with Ruth Chabay and Bruce Sherwood - Episode 49

Visit our site to listen to past episodes, support the show, join our community, and sign up for our mailing list.

Summary

Wouldn’t it be nice to be able to generate interactive 3D visualizations of physical systems in a declarative manner with Python? In this episode we spoke with Ruth Chabay and Bruce Sherwood about the VPython project which does just that. They tell us about how the use VPython in their classrooms, how the project got started, and the work they have done to bring it into the browser.

Brief Introduction

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • Subscribe on iTunes, Stitcher, TuneIn or RSS
  • Follow us on Twitter or Google+
  • Give us feedback! Leave a review on iTunes, Tweet to us, send us an email or leave us a message on Google+
  • Join our community! Visit discourse.pythonpodcast.com for your opportunity to find out about upcoming guests, suggest questions, and propose show ideas.
  • I would like to thank everyone who has donated to the show. Your contributions help us make the show sustainable. For details on how to support the show you can visit our site at pythonpodcast.com
  • Linode is sponsoring us this week. Check them out at linode.com/podcastinit and get a $20 credit to try out their fast and reliable Linux virtual servers for your next project
  • I would also like to thank Hired, a job marketplace for developers and designers, for sponsoring this episode of Podcast.__init__. Use the link hired.com/podcastinit to double your signing bonus.
  • Your hosts as usual are Tobias Macey and Chris Patti
  • Today we are interviewing Ruth Chabay and Bruce Sherwood about their work on VPython

Interview

  • Introductions
  • How did you get introduced to Python? – Chris
  • What is VPython and how did it get started? – Tobias
  • What problems inspired you to create VPython? – Chris
  • How do you design an API that allows for such powerful 3D visualization while still making it accessible to students who are focusing on learning new concepts in mathematics and physics so that they don’t get overwhelmed by the tool? – Tobias
  • I know many schools have embraced the open curriculum idea, have any of your physics courses using VPython been made available to the non matriculating public? – Chris
  • How does VPython perform its rendering? If you were to reimplement it would you do anything differently? – Tobias
  • One of the remarkable points about VPython is its ability to execute the simulations in a browser environment. Can you explain the technologies involved to make that work? – Tobias
  • Given the real-time rendering capabilities in VPython I’m sure that performance is a core concern for the project. What are some of the methods that are used to ensure an appropriate level of speed and does the cross-platform nature of the package pose any additional challenges? – Tobias
  • How does collision detection work in VPython, and does it handle more complex assemblies of component objects? – Chris
  • Can you talk a little bit about VPython’s design, and perhaps walk us through how a simple scene is rendered, say the results of the sphere() call? – Chris

Keep In Touch

Picks

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

PyData London with Ian Ozsvald and Emlyn Clay - Episode 48

Visit our site to listen to past episodes, support the show, join our community, and sign up for our mailing list.

Summary

Ian Ozsvald and Emlyn Clay are co-chairs of the London chapter of the PyData organization. In this episode we talked to them about their experience managing the PyData conference and meetup, what the PyData organization does, and their thoughts on using Python for data analytics in their work.

Brief Introduction

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • Subscribe on iTunes, Stitcher, TuneIn or RSS
  • Follow us on Twitter or Google+
  • Give us feedback! Leave a review on iTunes, Tweet to us, send us an email or leave us a message on Google+
  • Join our community! Visit discourse.pythonpodcast.com for your opportunity to find out about upcoming guests, suggest questions, and propose show ideas.
  • I would like to thank everyone who has donated to the show. Your contributions help us make the show sustainable. For details on how to support the show you can visit our site at pythonpodcast.com
  • Linode is sponsoring us this week. Check them out at linode.com/podcastinit and get a $20 credit to try out their fast and reliable Linux virtual servers for your next project
  • I would also like to thank Hired, a job marketplace for developers and designers, for sponsoring this episode of Podcast.__init__. Use the link hired.com/podcastinit to double your signing bonus.
  • Your hosts as usual are Tobias Macey and Chris Patti
  • Today we are interviewing Ian Ozsvald and Emlyn Clay about their work with PyData London, a group within the PyData organization. PyData London represents the largest Python group in London at ~2850 members, they hold regular monthly meetups for ~200 members at AHL near Bank and a yearly conference for around ~300 members. Last year, they and their sponsors raised over £26,000 to sponsor the development of core numerical libraries in Python.
Linode Sponsor Banner
Use the promo code podcastinit20 to get a $20 credit when you sign up!

Hired Logo

On Hired software engineers & designers can get 5+ interview requests in a week and each offer has salary and equity upfront. With full time and contract opportunities available, users can view the offers and accept or reject them before talking to any company. Work with over 2,500 companies from startups to large public companies hailing from 12 major tech hubs in North America and Europe. Hired is totally free for users and If you get a job you’ll get a $2,000 “thank you” bonus. If you use our special link to signup, then that bonus will double to $4,000 when you accept a job. If you’re not looking for a job but know someone who is, you can refer them to Hired and get a $1,337 bonus when they accept a job.

Interview

  • Introductions
  • How did you get introduced to Python? – Chris
  • What is the PyData organization, how does PyData London fit into it and what is your relationship with it? – Tobias
  • In what ways does a PyData conference differ from a PyCon? – Tobias
  • Does PyData do anything in particular to encourage users from disciplines that might not be aware of how much our community has to offer to choose the Python suite of data analysis tools? – Chris
  • You have both spent a good portion of your careers using Python for working with and analyzing data from various domains. How has that experience evolved over the past several years as newer tools have become available? – Tobias
  • For someone who is just getting started in the data analytics space, what advice can you give? – Tobias
  • How can conferences like PyData help strengthen the bonds and synergies between the Python software community and the sciences? – Chris
  • There are a number of different subtopics within the blanket categorization of data science. Is it difficult to balance the subject matter in PyData conferences and meetups to keep members of the audience from being alienated? – Tobias
  • Data science is a young field and we’ve yet to see lots of examples of the successful use of data. How are London-based companies using data with Python? – Ian
  • Is there a Python data science library you think needs a little love? – Emlyn

Keep In Touch

Picks

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Efene with Mariano Guerra - Episode 47

Visit our site to listen to past episodes, support the show, join our community, and sign up for our mailing list.

Summary

Efene is a language that runs on the Erlang Virtual Machine (BEAM) and is inspired by the Zen of Python. It is intended as a bridge language that serves to ease the transition into the Erlang ecosystem for people who are coming from languages like Python. In this episode I spoke with Mariano Guerra, the creator of Efene, about how Python influenced his design choices, why you might want to use it, and when Python is the better tool.

Brief Introduction

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • Subscribe on iTunes, Stitcher, TuneIn or RSS
  • Follow us on Twitter or Google+
  • Give us feedback! Leave a review on iTunes, Tweet to us, send us an email or leave us a message on Google+
  • Join our community! Visit discourse.pythonpodcast.com for your opportunity to find out about upcoming guests, suggest questions, and propose show ideas.
  • I would like to thank everyone who has donated to the show. Your contributions help us make the show sustainable. For details on how to support the show you can visit our site at pythonpodcast.com
  • Linode is sponsoring us this week. Check them out at linode.com/podcastinit and get a $20 credit to try out their fast and reliable Linux virtual servers for your next project
  • I would also like to thank Hired, a job marketplace for developers, for sponsoring this episode of Podcast.__init__. Use the link hired.com/podcastinit to double your signing bonus.
  • Your host today is Tobias Macey
  • Today we are interviewing Mariano Guerra about his work on the Efene language.

Interview with Mariano Guerra

  • Introductions
  • How did you get introduced to Python? – Chris
  • So Efene is a language that runs on the BEAM VM which you say was at least partially inspired by the Zen of Python. Can you explain in greater detail in what form that inspiration manifested and some of the process involved in the creation of Efene? – Tobias
  • What inspired you to create Efene and what problems does it solve? – Tobias
  • How does Efene compare to other BEAM based languages such as Elixir? – Tobias
  • When would a Python developer want to consider using Efene? – Tobias
  • What benefits does the BEAM provide that can’t be easily replicated in the Python ecosystem? – Tobias
  • Does the Efene language ease the transition to a more functional mindset for developers who are already familiar with Python paradigms? – Tobias
  • I understand that you are experimenting with another language implementation that runs on the BEAM. Can you describe that project and compare it to Efene? What were your inspirations? – Tobias

Keep In Touch

Picks

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Functional Python with Matthew Rocklin and Alexander Schepanovsky - Episode 46

Visit our site to listen to past episodes, support the show, join our community, and sign up for our mailing list.

Summary

What is functional programming, why would you want to use it, and how can you get started with it in Python? Our guests this week, Matthew Rocklin and Alexander Schepanovsky, help us understand all of that and more. Matthew and Alexander have each created their own Python libraries to make it easier to employ functional paradigms in your Python code. In this episode they help us understand the benefits that functional styles can have and the benefits that can be realized by trying them out for yourself.

Brief Introduction

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • Subscribe on iTunes, Stitcher, TuneIn or RSS
  • Follow us on Twitter or Google+
  • Give us feedback! Leave a review on iTunes, Tweet to us, send us an email or leave us a message on Google+
  • Join our community! Visit discourse.pythonpodcast.com for your opportunity to find out about upcoming guests, suggest questions, and propose show ideas.
  • I would like to thank everyone who has donated to the show. Your contributions help us make the show sustainable. For details on how to support the show you can visit our site at pythonpodcast.com
  • Linode is sponsoring us this week. Check them out at linode.com/podcastinit and get a $20 credit to try out their fast and reliable Linux virtual servers for your next project
  • I would also like to thank Hired, a job marketplace for developers and designers, for sponsoring this episode of Podcast.__init__. Use the link hired.com/podcastinit to double your signing bonus.
  • Your host today is Tobias Macey
  • Today we are interviewing Matthew Rocklin and Alexander Schepanovski about their work on functional libraries for Python.

Interview with Alexander and Matthew

  • Introductions
  • How did you get introduced to Python? – Chris
  • Can you first explain what functional programming is and how it differs from the procedural or object oriented programming that most Pythonistas are familiar with? – Tobias
  • How did you get started with functional programming? – Tobias
  • What are the benefits of functional programming and when might someone want to use functional paradigms in their projects? – Tobias
  • What is it about functional programming that people find so intimidating and what do you think has led to its recent rise in popularity? – Tobias
  • What aspects of the Python language lend themselves to being used in a functional manner and where does it fall down? – Tobias
  • Can you each describe what your respective libraries provide in terms of functional capabilities and what their particular focus is? Are they distinct enough from each other that it would make sense to use them both in a single project? – Tobias
  • What inspired each of you to create your respective libraries? – Tobias
  • There is a functools module in the Python standard library that provides some methods that enable functional paradigms. Where does that module fall short and how do your respective libraries augment or replace the functionality in that module? – Tobias
  • There is also a library named fn.py which provides functional paradigms for use in Python. Can you each compare and contrast it with your own work? – Tobias
  • There are a number of concepts involved in functional programming such as currying, function composition, immutable data, and pure functions. Can you describe some of those concepts and then explain which of them you tried to incorporate into your libraries? – Tobias
  • What are some of the resources that you have found to be most helpful when trying to learn and apply functional principles to your programs? – Tobias

Keep In Touch

Picks

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Cython with Craig Citro and Robert Bradshaw - Episode 45

Visit our site to listen to past episodes, support the show, join our community, and sign up for our mailing list.

Summary

Do you find yourself reaching for a different language when you need some extra speed? With Cython you can get the best of both worlds by writing your code in Python and executing it as compiled code. In this episode we were joined by Craig Citro and Robert Bradshaw from the Cython project to discuss how and when you might want to incorporate it into your applications.

Brief Introduction

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • Subscribe on iTunes, Stitcher, TuneIn or RSS
  • Follow us on Twitter or Google+
  • Give us feedback! Leave a review on iTunes, Tweet to us, send us an email or leave us a message on Google+
  • Join our community! Visit discourse.pythonpodcast.com for your opportunity to find out about upcoming guests, suggest questions, and propose show ideas.
  • I would like to thank everyone who has donated to the show. Your contributions help us make the show sustainable. For details on how to support the show you can visit our site at pythonpodcast.com
  • Linode is sponsoring us this week. Check them out at linode.com/podcastinit and get a $20 credit to try out their fast and reliable Linux virtual servers for your next project
  • I would also like to thank Hired, a job marketplace for developers, for sponsoring this episode of Podcast.__init__. Use the link hired.com/podcastinit to double your signing bonus.
  • Your hosts as usual are Tobias Macey and Chris Patti
  • Today we are interviewing Craig Citro and Robert Bradshaw

Interview with Craig Citro and Robert Bradshaw

  • Introductions
  • How did you get introduced to Python? – Chris
  • What is Cython and how did the project get started? – Tobias
  • My understanding is that Cython can improve the performance of a Python program without even having to provide any type annotations. How does it manage to do that? – Tobias
  • Can a Cython module be used as a way to sidestep the GIL? What are some of the pitfalls that can be caused by doing so? – Tobias
  • Can you give some examples of how Cython can be used to improve the perfomance of Python programs? – Tobias
  • How does Cython work under the covers? – Tobias
  • What were some of the challenges during the creation of Cython and what design decisions were made to overcome them? – Tobias
  • Does Python’s cross platform nature create any unique challenges when compiling down to the C level? – Chris
  • What processor and system architectures does Cython support and are there plans to expand that support? – Tobias
  • How do generators and list comprehensions map to C, and did those higher level language constructs pose any special challenges in Cython’s design? – Chris
  • Would Rust ever be a potential compile target for performance and safety optimized modules? – Tobias

Keep In Touch

Picks

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Airflow with Maxime Beauchemin - Episode 44

Visit our site to listen to past episodes, support the show, join our community, and sign up for our mailing list.

Summary

Are you struggling with trying to manage a series of related, interdependent batch jobs? Then you should check out Airflow. In this episode we spoke with the project’s creator Maxime Beauchemin about what inspired him to create it, how it works, and why you might want to use it. Airflow is a data pipeline management tool that will simplify how you build, deploy, and monitor your complex data processing tasks so that you can focus on getting the insights you need from your data.

Brief Introduction

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • Subscribe on iTunes, Stitcher, TuneIn or RSS
  • Follow us on Twitter or Google+
  • Give us feedback! Leave a review on iTunes, Tweet to us, send us an email or leave us a message on Google+
  • Join our community! Visit discourse.pythonpodcast.com for your opportunity to find out about upcoming guests, suggest questions, and propose show ideas.
  • I would like to thank everyone who has donated to the show. Your contributions help us make the show sustainable. For details on how to support the show you can visit our site at pythonpodcast.com
  • Linode is sponsoring us this week. Check them out at linode.com/podcastinit and get a $20 credit to try out their fast and reliable Linux virtual servers for your next project
  • I would also like to thank Hired, a job marketplace for developers and designers, for sponsoring this episode of Podcast.__init__. Use the link hired.com/podcastinit to double your signing bonus.
  • Your hosts as usual are Tobias Macey and Chris Patti
  • Today we are interviewing Maxime Beauchemin about his work on the Airflow project.

Interview with Maxime Beauchemin

  • Introductions
  • How did you get introduced to Python? – Chris
  • What is Airflow and what are some of the kinds of problems it can be used to solve? – Chris
  • What are some of the biggest challenges that you have seen when implementing a data pipeline with a workflow engine? – Tobias
  • What are some of the signs that a workflow engine is needed? – Tobias
  • Can you share some of the design and architecture of Airflow and how you arrived at those decisions? – Tobias
  • How does Airflow compare to other workflow management solutions, and why did you choose to write your own? – Chris
  • One of the features of Airflow that is emphasized in the documentation is the ability to dynamically generate pipelines. Can you describe how that works and why it is useful? – Tobias
  • For anyone who wants to get started with using Airflow, what are the infrastructure requirements? – Tobias
  • Airflow, like a number of the other tools in the space, support interoperability with Hadoop and its ecosystem. Can you elaborate on why JVM technologies have become so prevalent in the big data space and how Python fits into that overall problem domain? – Tobias
  • Airflow comes with a web UI for visualizing workflows, as do a few of the other Python workflow engines. Why is that an important feature for this kind of tool and what are some of the tasks and use cases that are supported in the Airflow web portal? – Tobias
  • One problem with data management is tracking the provenance of data as it is manipulated and shuttled between different systems. Does Airflow have any support for maintaining that kind of information and if not do you have recommendations for how practitioners can approach the issue? – Tobias
  • What other kinds of metadata can Airflow track as it executes tasks and what are some of the interesting uses you have seen or created for that information? – Tobias
  • With all the other languages competing for mindshare, what made you choose Python when you built Airflow? – Chris
  • I notice that Airflow supports Kerberos. It’s an incredibly capable security model but that comes at a high price in terms of complexity. What were the challenges and was it worth the additional implementation effort? – Chris
  • When does the data pipeline/workflow management paradigm break down and what other approaches or tools can be used in those cases? – Tobias
  • So, you wrote another tool recently called Panoramix. Can you describe what it is and maybe explain how it fits in the data management domain in relation to Airflow? – Tobias

Keep In Touch

Picks

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA