Summary
Dependency management in Python has taken a long and winding path, which has led to the current dominance of Pip. One of the remaining shortcomings is the lack of a robust mechanism for resolving the package and version constraints that are necessary to produce a working system. Thankfully, the Python Software Foundation has funded an effort to upgrade the dependency resolution algorithm and user experience of Pip. In this episode the engineers working on these improvements, Pradyun Gedam, Tzu-Ping Chung, and Paul Moore, discuss the history of Pip, the challenges of dependency management in Python, and the benefits that surrounding projects will gain from a more robust resolution algorithm. This is an exciting development for the Python ecosystem, so listen now and then provide feedback on how the new resolver is working for you.
Do you want to try out some of the tools and applications that you heard about on Podcast.__init__? Do you have a side project that you want to share with the world? With Linode’s managed Kubernetes platform it’s now even easier to get started with the latest in cloud technologies. With the combined power of the leading container orchestrator and the speed and reliability of Linode’s object storage, node balancers, block storage, and dedicated CPU or GPU instances, you’ve got everything you need to scale up. Go to pythonpodcast.com/linode today and get a $100 credit to launch a new cluster, run a server, upload some data, or… And don’t forget to thank them for being a long time supporter of Podcast.__init__!
Did you know Data science is a fast-growing career field, with a 650% growth in jobs since 2012 and a median salary of around $125,000? Springboard has identified that data careers are going to shape the future, and has responded to that need by creating the Springboard School of Data, comprehensive, end-to-end data career programs that encompass data science, data analytics, data engineering, and machine learning.
Each Springboard course is 100% online and remote, and each course curriculum is tailored to fit the schedule of working professionals. This means flexible hours and a project-based methodology designed to get real world experience: every Springboard student graduates with a portfolio of projects to showcase their skills to potential employers. Springboard’s unique approach to learning is centered on the very simple idea that mentorship and one-on-one human support is the fastest and most efficient way to learn new skills. That’s why all of Springboard’s data courses are supported by a vast network of industry expert mentors, who are carefully vetted to ensure the right fit for each program. Mentors provide valuable guidance, coaching, and support to help keep Springboard students motivated through weekly, 1:1 video calls for the duration of the program.
Before graduation, Springboard’s career services team supports students in their job search, helping prepare them for interviews and networking, and facilitates their transition in the tech or data industry. Springboard’s tuition-back guarantee allows students to secure the role of their dreams and invest in themselves without risk. Meaning students are not charged if they don’t get a job offer in the field they study. Springboard’s support does not end when students graduate. All Springboard graduates benefit from an extensive support network encompassing career services, 1:1 career coaching, networking tips, resume assistance, interview prep, and salary negotiation.
Since Springboard was founded in 2013, around 94% of eligible graduates secured a job within one year, earning an average salary increase of $26,000. Want to learn more? Springboard is exclusively offering up to 20 scholarships of $500 to listeners of Podcast.__init__. Simply go to pythonpodcast.com/springboard for more information.
Announcements
- Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
- When you’re ready to launch your next app or want to try a project you hear about on the show, you’ll need somewhere to deploy it, so take a look at our friends over at Linode. With 200 Gbit/s private networking, node balancers, a 40 Gbit/s public network, fast object storage, and a brand new managed Kubernetes platform, all controlled by a convenient API you’ve got everything you need to scale up. And for your tasks that need fast computation, such as training machine learning models, they’ve got dedicated CPU and GPU instances. Go to pythonpodcast.com/linode to get a $20 credit and launch a new server in under a minute. And don’t forget to thank them for their continued support of this show!
- You listen to this show because you love Python and want to keep your skills up to date, and machine learning is finding its way into every aspect of software engineering. Springboard has partnered with us to help you take the next step in your career by offering a scholarship to their Machine Learning Engineering career track program. In this online, project-based course every student is paired with a Machine Learning expert who provides unlimited 1:1 mentorship support throughout the program via video conferences. You’ll build up your portfolio of machine learning projects and gain hands-on experience in writing machine learning algorithms, deploying models into production, and managing the lifecycle of a deep learning prototype. Springboard offers a job guarantee, meaning that you don’t have to pay for the program until you get a job in the space. Podcast.__init__ is exclusively offering listeners 20 scholarships of $500 to eligible applicants. It only takes 10 minutes and there’s no obligation. Go to pythonpodcast.com/springboard and apply today! Make sure to use the code AISPRINGBOARD when you enroll.
- Your host as usual is Tobias Macey and today I’m interviewing Tzu-ping Chung, Pradyun Gedam, and Paul Moore about their work to improve the dependency resolution capabilities of Pip and its user experience
Interview
- Introductions
- How did you get introduced to Python?
- Can you start by describing the focus of the work that you are doing?
- What is the scope of the work, and what is the established criteria for when it is considered complete?
- What is your history with working on the Pip source code and what interests you most about this project?
- What are the main sources or manifestations of technical debt that exist in Pip as of today?
- How does it currently handle dependency resolution?
- What are some of the workarounds that developers have had to resort to in the absence of a robust dependency resolver in Pip?
- How is the new dependency resolver implemented?
- How has your initial design evolved or shifted as you have gotten further along in its implementation?
- What are the pieces of information that the resolver will rely on for determining which packages and versions to install? (e.g. will it install setuptools > 45.x in a Python 2 virtualenv?)
- What are the new capabilities in Pip that will be enabled by this upgrade to the dependency resolver?
- What projects or features in the encompassing ecosystem will be unblocked with the introduction of this upgrade?
- What are some of the changes that users will need to make to adopt the updated Pip?
- How do you anticipate the changes in Pip impacting the viability or adoption of Python and its ecosystem within different communities or industries?
- What are some of the additional changes or improvements that you would like to see in Pip or other core elements of the Python landscape?
- What are some of the most interesting, unexpected, or challenging lessons that you have learned while working on these updates to Pip?
Keep In Touch
- Pradyun
- Website
- pradyunsg on GitHub
- @pradyunsg on Twitter
- Paul
- pfmoore on GitHub
- Tzu-Ping
Picks
- Tzu-ping
- Paul
- Pradyun
- because my picks can be anything — things that have kept me sane in this lockdown world
- Music: Chris Daughtry
- Video Game: Parkitect
- because my picks can be anything — things that have kept me sane in this lockdown world
- Tobias
Closing Announcements
- Thank you for listening! Don’t forget to check out our other show, the Data Engineering Podcast for the latest on modern data management.
- Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
- If you’ve learned something or tried out a project from the show then tell us about it! Email hosts@podcastinit.com) with your story.
- To help other people find the show please leave a review on iTunes and tell your friends and co-workers
- Join the community in the new Zulip chat workspace at pythonpodcast.com/chat
Links
- Pip
- Macdown
- Taiwan
- Pipenv
- PyPI
- TOML
- Python Package Metadata Standards
- iBook G4
- Acorn Computer
- distutils
- easy_install
- Python Eggs
- setuptools
- Python Wheels
- CPAN
- Conda
- Inside The Cheeseshop
- Google Summer of Code
- Zazo
- PEP517
- pip-tools
- Poetry
- resolvelib
- SAT Solver
- Trove Classifiers
- PyPA
- pyproject.toml
The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA