PyCon India 2025 Keynote: Fellowship of the Stack: Scientific Discovery with Python - Dawn Wages
Watch on YouTubeVideo summary
Dawn Wages, an outgoing chair of the Python Software Foundation and a dedicated advocate for open-source software, delivered a keynote titled "Fellowship of the Stack: Scientific Discovery with Python." She framed the complex landscape of scientific computing not as a static set of tools, but as an epic journey requiring a fellowship of diverse companions. Her presentation utilized a fantasy metaphor to explain the evolution of the Python ecosystem, where developers must gather supplies like language distributions and IDEs, cast spells through documentation, and navigate treacherous terrain defined by evolving software dependencies and the sheer versatility of Python itself. She emphasized that while Python's flexibility allows for rapid development across many domains, this very strength creates a fragmented environment with competing dependencies that can be difficult to manage without careful planning and community support.
The core of her talk explored the historical and technical challenges inherent in maintaining scientific Python environments, contrasting the early days of distribution via Usenet with modern complexities involving C API extensions, just-in-time compilation, and diverse interpreters like PyPy and GraalPython. Wages highlighted specific tools that act as essential companions on this journey, such as Polars for high-performance data manipulation, NVIDIA Rapids for GPU acceleration, and various project management tools like Poetry and uv. She explained the intricacies of how Python code is interpreted into bytecode and executed by the virtual machine, noting recent advancements like PEP 744 that offer better visibility into memory layout and execution paths. Furthermore, she addressed the notorious packaging headaches caused by legacy C libraries in packages like Pillow and GDAL, advocating for a shift toward reproducible research environments and modular workflows to ensure code remains maintainable and scientifically accurate.
Beyond technical specifics, Wages discussed the critical state of the Python Software Foundation's funding and governance. She transparently addressed the recent pause in PSF grant cycles due to rising conference costs and a lack of proportional investment from large corporations, urging the community to engage with upcoming elections and consider sustaining memberships to support the foundation's mission. She also clarified the relationship between the US-based PSF and regional organizations like Python Asia, explaining that while they are separate entities, they collaborate informally to support the global language. The keynote concluded with an optimistic outlook on Python's future, particularly regarding WebAssembly for browser optimization and just-in-time compilation, reinforcing the idea that the community must remain vigilant yet collaborative to ensure Python continues to thrive as a premier tool for scientific discovery.
Read the full video transcript
Yeah, folks. So, let me introduce our
last uh keynote speaker, Don Wedges. Um,
I'll give a small introduction. Uh, Don
is a dedicated uh member of Python
community community currently serving as
a chair of Python software foundation
board of directors and a member of
organization for ethical source advisory
board. She is the director of community
and develop developer relationship at
Anaconda, a leading provider of data
scient science tools and opensource
distribution. Dawn is a vocal champion
for open-source software and a
passionate advocate for building
welcoming and inclusive space for
developers and contributors. She brings
her experience supporting Python
developers tools at Microsoft and a
focus on nurturing the longstanding
community that supports Anaconda's
open-source core to the conference.
Let's welcome her with a huge round of
applause.
Hey everyone, I'm a bit short, so I'm
going to be spending part-time behind
the podium and a little bit of time out
here. Also, it's the end of the day, and
I appreciate seeing so many faces in the
audience. This is going to be a little
bit interactive. I'm going to ask for
you to raise your hand um and kind of do
a back and forth with me. So, if we
could keep the energy up for the last
part of the day, I would appreciate it.
that would make the keynote go great.
So, as we mentioned, my name is Don
Wages, she her pronouns. I came all the
way from Philadelphia to Pyon, India,
and it's been a fantastic occasion
already, and this is my first time in
India. So, I really appreciate the warm
welcome I've had.
>> [applause]
>> So today we'll be talking about the
fellowship of the stack uh a scientic
scientific discovery with Python and
this is going to be a talk that's going
to be really accessible to almost all of
the audience regardless of your
expertise. Um, but this talk actually
started out and I want to give a shout
out to to Pur Purdue uh school of
advanced computing uh for letting me do
this talk in a different iteration um
before coming here. It actually had no
narrative arc at all. And this is almost
entirely about characters. Uh an arc on
how an epic journey is going to get you
to your in destination of scientific
discovery with Python and how we can do
a better together story.
So my name is Don Wages. I am the
outgoing chair of the Python Software
Foundation. If you have questions at the
end, um hopefully I go with enough time
so you can give questions. Please feel
free to ask you questions about the PSF.
Uh we have ongoing elections going on
right now. Um I will ask you to get your
phone out if you can uh because there
will be two QR codes that I would love
for you to action. Um I also uh am a
Python engineer and I have been for
uh going on a little bit over a decade.
Um, I also organize Python events and
I've been doing it as long as I've been
learning Python. Um, and I also uh
contribute to the Wagtail CMS core team
and the other things we talked about. I
lead a team at Anaconda uh for developer
and community relations.
So, if you could all take out your
phones, I Oh, I see you doing it. Yes,
great audience. Yeah. So there are two
CTAs main CTAs in this event. The first
of which is I would love for everyone in
this audience to become a Python
Software Foundation member. There are
two different ways to become a Python
Software Foundation member. One is you
can be a contributing member which means
that you give five hours on average a
month or more contributing to an open
source project or contributing to an
open source community in Python. If you
do that, it is self uh nomination. You
get to attest that yourself. No one's
really going to like follow up on you to
make sure you're doing it. It is honor
system. But please give back to the
community and you can also be a Python
Software Foundation member. And then the
second way is you can also uh be a
sustaining member which means there is a
sliding scale membership and you can pay
to be a member of the Python Software
Foundation with as little as $25 US.
The other CTA is the taking the Python
packaging survey. Uh this is a labor of
love of mine. It's been years and we've
had trouble getting off the ground a
Python packaging survey. There's the
JetBrain survey that is general Python
ecosystem. This one's specifically on
Python packaging. If you've ever pip
installed, if you've ever installed, if
you've ever uh
cloned and and built a Docker container
and with an automation script and your
Python packages were already in there
and you didn't have to touch it, you're
still welcome to take this survey. We'd
love to know how people are using Python
and how they're interacting with their
PI um packages in the ecosystem. We
currently have 16,000 respondents, which
is really big deal. The survey closes at
the end of December and then there's
going to be an open data set in early
2026 with all of the responses
available. So your responses could be
included in that data set and it takes
usually on average like 10 to 15 minutes
to complete. Uh,
perfect. All right. So this is going to
be our epic journey. We're going to
start with why the scientific Python
community uh and ecosystem is so
treacherous to maintain. It is something
that's widely known across the ecosystem
that scientific Python and Python in
particular has environments that are
kind of hairy and have some rough edges.
When it works, it works. Um but when it
doesn't, it's really hard to navigate
how to rectify and sometimes you just
want to uh blow up the environment and
start over. Why is that? um this
discovery the the presentation through
our scientific discovery it always
starts with a spark uh in this journey
we'll start to understand why the Python
ecosystem is so treacherous and I posit
today that it's one uh that there's two
reasons for this one there's the
evolving nature of software um and it
happens across software not just in
Python and we're in a critical
inflection in in the software
development ecosystem because there's an
incredible investment in making the
development process easier for us. This
is from the soaring success of VS Code.
I mean, we probably have some Emacs and
some Vim users out here. Um, but IDEES
are really the centerfold of how people
are interacting with their code these
days. And also the more recent
advancement of AI assistance making it
even more efficient for us to engage in
what we do. And the second uh is because
you can do so much with Python. So in
the early 2000s with the proliferation
of Python across many different uh
subject matter expertise, you saw
because of the way that Python provides
you a way to interact and build with a
level of abstraction away from the
machine, you're able to build faster and
more performantly uh performant in a
different way. It's not the fastest code
language, but it is more performant to
build something quickly. Um and we're
empowering users to build faster. That
also means that there is different and
disperate directions that the community
can to uh can go to build one thing and
it makes managing those uh competing
dependencies and environment sometimes
more difficult. So it provides an
immense amount of good and also allows
for a diverse group of developers to
create their very specific scenarios.
We also want to go into uh oh let's has
I would love a raise of hands. This is
first of interaction part. Who's heard
of the Python documentary?
Oh, yay. Good. Who's watched the Python
documentary?
Okay, good number of you. So, we're
going to do a little clip of the Python
documentary right now about how the
first Python uh preversion 1.0 was
distributed.
If I
the release which turned out to be an
incredible pain because Usenet was the
only thing we had to release it.
[music] Usenet was a network of you
could say bulletin boards. People would
write messages. They would get
distributed over the world and then
there were the groups for [music] all
the different computer languages. Usenit
had a very strict limit on the size of a
single post. First [music] put your
entire source tree in a tar bottle. Then
compress it. Then you have to do another
step where [music] you turn the binary
data into ASI encoding.
So you have this huge file with
gobblygook, but it's at least printable
gobly gobblygook. And then you apply
another tool that snips [music] it into
pieces.
Then there is a script where if [music]
you have downloaded all 21 parts, you
can undo all those operations in
reverse. That was what people put up
with. So we did all the work on our side
and [music] plenty of people did all the
work on their side to unpack this
because I had apparently written a very
good teaser about what this was.
And that's a little bit of the um
let's put this back into presentation
mode.
Perfect. All right, we're back.
Okay, we're back. Perfect. So, um that's
a little excerpt of the documentary
which I really enjoyed watching.
Hopefully that's enough to get the rest
of you to want to watch the rest of the
documentary, too. And it's all on
YouTube.
Also, we're going to dive into what the
ecosystem is doing currently to address
it. So, from uh really difficult
developer experiences like we just saw
all the way to what we're going through
now, we're iterating on a better
developer experience every day. Um but
you know there's much more to do and
then where do you fit in a scientific
Python fellow um how can you advance
this?
So in the grand scheme of this uh
metaphor we're going to get our
supplies. We're going to cast our first
spells. We're going to gather companions
and address some of the trials and
tribulations of the Python ecosystem.
uh we'll form a fellowship along the way
and then we'll scale our quest. We'll
give you some extra activities that
you're going to do to continue this
advancement and take record of our
exploration.
So you'll need a language distribution.
There's many different ways that you can
get your Python. Most people are more
familiar with uh CPython. That is the uh
default distribution that's done by the
uh Python steering council. There's a
language interpreter manage package
manager development environment. You'll
also want your version control. Um, and
if you're doing uh exploration with
data, you'll want your data input and
your pipeline for ETL. And then your
specific domain is going to determine
how your exploratory toolkit or your
analysis engine is going to work.
So we uh stumble upon a guide and we a
shopkeeper of sorts um and in their own
right is experienced with all these very
and difficult tools to help us on our
journey. Your IDE is your interactive
development environment will transport
you to another environment and keep
track of your it'll keep track of your
variables um of that universe and
connect symbols across your files um
into other portals.
Uh I recommend a health check of sorts.
Tools to provide uh health checks on
your code are invaluable. For example,
cond forge and mamba have environment
health checks. You can also consider
hypothesis testing, data alerts on your
pipeline uh to identify data drift um or
data ch data quality track uh quality
experiments um and keep tracking of
those.
And this may look like a simple ball of
twine uh but it is uh what holds your
code together uh from your developer
environment to your staging and
production scenarios. Twine the tool
that um that is under GitHub under the
hood uh keeps your CI/CD moving from one
environment to the other.
And don't forget code for matters
llinters and style checkers. uh organize
your imports in security focus on
linting uh in security focused linting
like bandit.
Your spell book or your documentation
starts with writing spells so that you
can recite them truthfully every time
you need them um to sit down and embark
upon your own journey. But soon you'll
expand that spell book to uh be helpful
for others. Share it freely and reduce
the barriers for your fellow journeymen.
uh and people across all levels will
have the same reach that you do.
And think of this like a time machine of
sorts. You have mysticism, but beyond
what you know, specifically in
scientific computing and scripting in
Python, it's easy to jump into code
without justifying why you should
include version control. Think of it
like your time machine.
And the magic of data. Don't forget your
data. your data is gold.
So, when we're casting our first spells,
another interactive portion, I'm going
to ask you all to raise your hand. Um,
as you're starting your projects, are
you who who in this audience is using
languages other than Python in their
Python projects?
Okay, I see a number of hands up. Um,
are you able to or do you typically
anticipate the dependency considerations
before you start? Um, or are you uh
stumbling upon the dependencies and
solving it as you get to it? So, who's
considering it before they even start
the dependencies?
Who stumbles upon the dependency uh
contradictions and then fixes it as they
go? Okay, more hands there. What about
sharing environments with your peers?
Are you predicting or expecting to share
environments with your peers hands?
Okay, a good number of people are are
doing that. Um, and what about tools
that use projectbased approaches like
Pippen, Pixie, UV, poetry?
Hands.
Okay, good number of people doing that
as well. Cool.
So this is like a common journey that
you'll have in your data science and
scientific computing um process. You'll
want to start with identifying your
problem and your scope and then get
deeper into your data understanding.
Many times cleaning and uh normalizing
and wrangling your data. You'll model
your model your data when you uh have
data science variations in scientific
computing. Um and then you'll evaluate
um that uh those models and then check
back with your production results.
And here we we met a friend with deep
knowledge driven by many years of p
pursuit wise in many years seemingly
magnanimous. Uh but it's their desire to
understand the universe that makes them
a great companion. And they're
well-versed in the new and the old
magic. They hear about your journey and
they're immediately interested in
helping. There's so many different ways
people use Python and there's a huge
ecosystem with hundreds of thousands of
packages available in open source to
exped expedite your innovation. So
you're able to leverage these packages
for your accomplishments, speeding up
the time to finish your task and pushing
away from your desk. Uh the first
generation focused on integration with
Python uh Python's ease of use with the
performance of compiled languages.
Roughly from the mid 1990s to the early
2000s uh tools like F2PI which would be
for Python swig and later CPython um
really fit that brill uh there was an
emphasis on arrayentric computing
introducing in dimensional arrays that
addressed uh Python's native list
limitations and numerical data enabled
mathematical operations uh that are on
entire arrays without explicit loops. So
these free tools that were alternative
to expensive pri proprietary tools um
like mat lab for example and this is all
within the academic and research
origins. These are developed by
scientists for scientists. Um and that
signaled um a a big shift in the Python
ecosystem.
And by the end of this first generation
uh we we see a movement into some of
this new magic. The most newly adopted
and widely appreciated packages in 2024
2025, excuse me. Um Polers is one of the
rising stars that hit version 1.0 0 and
is rapidly gaining adoption in potential
pandas replacement. It's lightning fast
uh dataf frame library that offers
significant performance improvements.
Uh performance particularly like polars
and ductb are in a league of their own
um being in an order of magnitude faster
than dask and pispark. Uh there are
streaming engines that are can seem to
be three to seven times faster in
polar's in-memory engine. And then
there's also a huge wave with GPU
accelerations. So for example, I've been
playing with the NVIDIA Rapids library
which has been really exciting and that
uh accelerates, excuse me, accesses your
uh GPU whether remote or cloud um and
provides up to 13 times performance
improvements inside the Rapids
accelerator polar's GPU engine.
Um another uh approach has also been uh
with ecosystem integration. So for
example, polars uh works with major
packages including scikitlearn and
plotting libraries uh through narwhal's
compatibility layer. There's also high
performance data manipulation in Python.
Um so all of these new features have led
to this new new magic of sort um that's
accelerating where we're going.
We're seeing some changes with a
performance first approach. Uh there's
also a zero code change philosophy. The
most successful new packages of 2025
focus on providing performance
improvements with minimal code changes.
That would include the Rapids library.
That's really exciting. Uh AutoML and
democratization. So automated machine
learning tools are becoming more
sophisticated. Um and they're making
advanced analytics access to be uh to
non-experts.
So, we talked about our scientific
companion. We have a new companion that
we've met ready for this adventure,
showing their fortitude. They're
bringing in software competencies. This
includes uh numerical accuracy, testing,
uh being thoughtful on resource
optimization, creating dry code. For
those who may not be familiar with dry,
that's when you don't repeat yourself in
your code. So that's levels of
abstraction and uh uh areas of um
ownership and expertise of your code uh
division of responsibilities of your
code also platform independent. So it
can it run on one machine versus another
maintainability which includes code
quality dependency management your CI/CD
pip pipeline and your documentation.
Some of the key insights is that uh
scientific computing serves these dual
purposes. It must produce produce
correct scientific results and it must
be maintainable as a software system. So
this dual requirement makes these
principles especially important in the
scientific computing ecosystem.
Um I do have a call out at the bottom.
Who of this audience I'd love a raise of
hands who's familiar with PI open sigh?
Okay great a few of y'all. So hopefully
I am introducing even more. So if you
are of uh in a scientific computing
background and may or may not have
experience with Python packaging for
example um this is an organization led
by Leo Waser um and they're really
fantastic group that have tons of
trainings and they bring in experts all
over the ecosystem to help you with your
Python packaging specifically in a
scientific computing context.
So let's dive into some of the trials
and tribulations that we may find in our
scary ecosystem that is that can be
Python. Python support is a lot to learn
especially in scientific computing when
your first domain may not be computing.
Python's considered a glue language. So
for example um there's a lot of people
who call Python the second best language
for everything. uh that would mean that
maybe it's not the fastest language. Um
but because it can do a lot of different
things, it's very flexible and it is a a
great middle layer language between your
many languages. Um it is definitely a a
popular language to glue your two
languages together. Maybe you'll have a
Java backend and a JavaScript frontend
and you're running pipelines or
integrations with your pipeline.
It's used in a lot of different use
cases. It's very flexible and extensible
and it can outsource a lot of the
performance bottlenecks. We'll get into
uh the next conversation we'll get into
the Python interpreter and we'll talk
about how the C API uh works in Python.
And also it's human readable code. So I
think that's a large part of even why
the documentary but a long time why
people say that Python has gotten so
much adoption is because you can write
your code walk away and not look at it
for a year come back to it and still
decipher what you were doing.
So we have a side quest and we have a
special friend to help us through our
side quest. We're learning the intric
intricacies of the CPython interpreter.
who would consider themselves among this
audience pretty familiar with the how
the CPython interpreter works? Raise a
hand.
Awesome. Great. So, it sounds like
there's a good number of people who are
coming along for the ride and some
people who may learn something from uh
from this little side quest. So, we'll
uh there are many Python interpreters.
There's Jython which is written for Java
and uh written in Java for Python.
There's Iron Python which is written
for.net. Uh there's pi which is for just
in time compiling of python. Uh it
directly executes the code rather than
needing a a separate compilation layer.
So that so that is why um python um is
directly as as this. Um so either
through hitting a play button in your
editor or shift enter or uh python main
on your CLI. There may be other ways
that you can execute your Python. We
have this friend whom we met along the
way who can help us break down this
process. Uh Python is an interpreted
language. So not all languages are
considered interpreted. Uh some are
compiled and that means the program that
you run is delivered as source code and
you can just run the source code. It'll
start out uh with your Python file uh by
the Python interpreter without any
fiscal physical compilation. This is
human readable code. This is peri Python
performing the lexical evaluations and
then breaking it down into character
tokens along with keywords, identifiers
and operation operators and literals.
The tokens are now representing the
shape of the code.
Then it means uh excuse me and then you
go into your Python interpreter. The
interpreter takes and executes each
statement written in your source code.
This will be line by line. uh the
program gets compiled by the Python
compiler um and then checks for errors
and if the compiler finds an error it
will show the error in the console and
if there's no error uh the source code
is well formatted then they'll be
converted into byte code.
Byte code is the low-level unbiased
representation of your source code. So
the compilation to the bite code is
accomplished through the Python compiler
and the bite code the Python bite code
compiler.
The bite uh code is then processed
inside the Python virtual machine and it
is interpreted to get give actual
machine code. So in order for this
portability of the source code to be
valuable, this Python virtual machine
now allows us to bring uh allows us
things that happen at runtime like
dynamic typing and runtime
introspection.
Uh this is also where the memory manager
and the garbage collection lives.
Uh
PEP 744 proposes a departure from uh the
PEP 659 which is specializing with the
adaptive interpreter and this rewrites
the bite code uh instructions in place.
So we're able to have more visibility
into the profiling of the code. For
example, we would be able to see what
paths are being executed the most or the
memory layout of particular objects.
And this is where the C API happens. So
if we're offloading highly uh
bottlenecks in our code, we're able to
leverage the C API. This is dynamic
loading of li libraries um in the
standard OS method. So it'll load the
entry point in its naming convention and
then it invokes a functionality where
it's entirely responsible for what the
modules set up and own. This is
important when I talk more about
optimizing with spy or numba. There'll
be a little bit more about that later on
this presentation. Um but this is a a
nice extension point for when you have
C, C++, forran which makes it relatively
easy to lean into performance
improvements.
And then you have your machine readable
code. The interpreter never produces any
intermediate machine code unlike a
compiled programming language. And
Python is easy for development
environment because of the way it that
it shares the source code and interpret
it dynamically. It'll allow you to write
your code in all sorts of ways have all
different types of tedious and difficult
dependencies and Python's interpreter
will just try its best to handle it.
And then there are packaging
complexities. We meet another friend
who's going to help us with another
tedious uh part of our main quest. Uh
this is these are deeply nuanced and
scary complexities that offer and they
decide to offer their help in
understanding in Python packaging.
combining a utility into a reproducible
and distributed version control way that
allows for iterative improvements uh can
work uh can be work fine or it could be
incredibly difficult. Sometimes a
package can be buil built differently
based on the date
or maybe bugs in GCC which leads to loss
of numerical precision based on import
order
or the having the correct configuration
sphsl
in particular their configuration script
is in pearl
or maybe shared symbols across libraries
that can delete reference differences
between versions silently.
In GDAL requirements, this requires
building Chromium, for example.
Sometimes it'll build differently based
on file system issues
or it'll build differently across
architectures. These are nontrivial
considerations.
For example, pillow relies on an
external C libraries like lib JPEG, lib
png and zlib in free type. So that adds
additional complexities to packaging
pillow a commonly known kind of headache
in the ecosystem. And this is not
necessarily to denigrate any of these
packages. Uh these are how legacy
concerns happen and these are known
concerns in the ecosystem and how to
address them are not easy.
in GDAL the geospatial data abstraction
library which is also notoriously
different difficult um it has complex
geospatial toolkit with numerous
dependencies and platform specific
quirks so for example GDAL supports
dozens of geospatial formats which all
require different libraries uh PJ
coordinates transformations geos which
is geometry operations there's SQL light
postgress HD HDF5, net CDF and others.
And it also imports uh different
geospatial formats like geotiff, shape
files, postgis,
etc.
And pietorch uh these conversations are
happening internally and out in the open
uh for the community part to
participate. For example, matching
pietorch versions with your CUDA
versions. PyTorch skips ships with
specific CUDA versions baked in and
there's a huge download size uh with
PyTorch and there's also platform
specific wheels. PyTorch has strict
compatibilities with requirements of
NumPy and other core libraries which
conflict with other packages in your
environment.
And then we meet new friends, ones with
domain expertise are imaginical siblings
which require you to make decisions
before you can deploy their services.
You go down into domain expertise or
does your general purpose and build your
own suit your needs.
uh we'll we could start with a general
purpose uh tools like numpy and sci and
then you can narrow in deeper into the
funnel like broad scientific uh packages
like pandas or mattplot lib uh we have
domain specific like biopython astropy
scikit image and then you'll also have
maybe lab specific uh packages that meet
your specific needs um in custom
analysis these are all tend to be
problem driven and designed and aligned
to your data types and formats and
incorporate domain nuance that allow for
interdisciplinary uh collaboration.
So our fellowship is forged. It's 2025
and scientific computing the
underpinning of AI is finding its most
salient applied use cases in this space.
We have language and package
distribution, your IDE choice, your
framework architecture, um environment
standards and artifacts, and your build
process and maintenance. The creators
and the most active consumers have
forged allegiances over the last 30 odd
years. And some of these trends have
emerged. So when it works, you may not
notice when your Python distribution is
not CPython, but there is a
proliferation of other Python
distributions. I'm most familiar with
CPython and the Anaconda distribution
and PIIE, the just in time uh Python
distribution, but there's about a dozen
more dependent on your use case. People
like development environments. We've
talked about this a little bit before.
It's gotten deeper in your expertise. Um
and then people are also familiar and
interested in tools that or orchestrate
your workflow.
These are seen across the ecosystem and
there are too many environment standards
in artifacts and a huge proliferation of
tooling. I wanted [clears throat] to
highlight three different PEPs for
people who are not familiar. PEP is a
Python enhancement proposal. Um so PEP
440 uh that's about version
identification. Um and if you're
interested you can go deeper and to look
into some of these PEPs. think it's
really interesting to um to follow the
PEPs and where they go and the
conversations around them because you'll
get deep expertise on why in the
architectural decisions um made by the
uh proposers. There's also PEP 621 which
is about product met uh project metadata
and then there's PEP 518 which is build
system requirements. All of these are
more recent or ongoing PEPs that are in
um discussion that people are really
interested in to advance our Python
ecosystem.
And so we met our helpers. We had we
started our journey. We met a friend who
showed us the way and inspired us to
believe what was possible. An epic
journey. We met a shopkeeper who showed
us the tools and a sage friend with deep
scientific knowledge of the old and new
ways.
Uh we had a formidable warrior who
equipped us with the software
engineering pre principles of resilient
code and two deeply technical
companions. One who helped us navigate
the Python interpreter and the other who
helped us dodge the perils of complex
packaging dependencies.
And last but not least are f friends
with deep domain knowledge. They know
the dark arts of every form and can help
us navigate our own domain nuance.
But where do we scale? So the next step
in uh your packaging journey is going to
be um making sure that you have observer
observable, monitored and logged code.
Uh you'll also want to have enhanced
collaboration tools. So this would mean
um having uh reproducible research
environments of sorts. Um sometimes you
can have tools that will allow you to
work on code and jam together. Um, and
then also some tools that will create
hashes or artifacts uh that will tell
you if your code environment have
changed. You'll also be interested maybe
in running on specialized architecture.
So if you have high performance
computing needs um and you have um
compute intensive jobs, you might want
to put that on a compute cluster
somewhere on someone else's code uh
excuse me someone else's computer or you
might want to be have uh use your GPU
and have CUDA enabled computations.
I'm also really interested in the
productize before production movement.
Um this is the uh build it so it works
for your specific need and then do
another round of refactoring to abstract
away some of these dependencies, reduce
repetition and um maybe um pull away
some code that can be uh open sourced
and used and useful for others.
I'm also interested in module workflows,
modular workflows, uh ones that can be
uh cut up and you can document them
individually and hopefully when done
well you can find drop in replacements
as you uh evolve. And then the best
workflows scaling your quest is always
well doumented uh workflows.
So that was scaling your request with
code. I want you to scale your quest
with the PSF. So there is an opportunity
to vote in this upcoming Python Software
Foundation election. You can also join
as a Python Software Foundation member.
And then I'll talk a little bit more
about the funding cycles in the PSF
right now. So if you're not already
involved or have not gotten your email,
if you are a Python Software Foundation
member already, um you should have that
email for OPA vote. closing uh voting
ends uh at on September 16th, 2M UTC.
But if you're not a Python Software
Foundation member and if you're just
interested in seeing how the process
happens from the beginning to the end
for a full calendar year, you're in a
great spot. You can sign up to be a PSF
member today. Here's a QR code. This is
where I got you originally to take out
your phones and you can be as PSF
member. you can see the entire election
cycle and then you can follow along to
see the uh board release minutes and um
and voting procedures and learn a little
bit more about the PSF before you get a
chance to vote in the 20207
elections.
And last but not least, uh I wanted to
talk about the funding in the Python uh
community. So, Python is known, the PSF
is known for giving hundreds of
thousands of dollars every year to
various communities across the world to
help them with their Python events. Um,
it's something that we pride ourselves
on. It's a main tenant of what the PSF
does to help proliferate the the
language across the world. Um, and we've
paused our PSF program. There is a
screen grab of the register talking
about how the growth of the Python
language um has not uh led to the same
growth in investments in the foundation.
And so the pause has been twofold. It's
been because we saw in the tea leaves
we've been paying attention and we knew
that the demand and the request from the
ecosystem on us for more support has
continued to grow. Um but we also saw
that we've gotten into a more expensive
conference environment. Um our the main
driver of funds for uh the PSF which is
PYON is was uh not returning what it
used to. Um and then we need the
reciprocal investments from large
companies and we've seen similar
movements across the ecosystem. All PI
PYON conferences are many PYON
conferences are experiencing the very
same thing. So, we're taking a pause um
for a full calendar year. It is
unprecedented, but we're really excited
to get back to funding the ecosystem in
a new and more sustainable way. And in
the same vein, we have an endofear
fundraiser. So, please keep a uh your
eyes out for pyfound.blogspot.com.
That's where you'll find our
announcement for the end of year
fundraiser. And if you have a company
match, for example, you can fund the
Python Software Foundation or you can
throw in the amount of a coffee every
month um in order to help fund the
organization that uh proliferates Python
across the world.
Great job. We did it together. We
unlocked the scientific uh the secrets
of scientific Python.
>> [applause]
>> There's a few ways this analog Oh,
almost done. There's a few ways this
analogy falls short. So, documentation
exists. It's not dark magic. We iterate.
It's not just one long journey. And you
do not have to understand the entire
ecosystem to be successful. Thank you so
much to Risa Erllin uh who um uh
provided the beautiful art that inspired
this uh presentation. And here's another
uh link to the Python packaging survey
to take and you can follow me on
socials. Now I am all done.
Thank you Don for the amazing note.
Thank you.
>> Um if you want to keep it up on the
screen, I'm open to questions. I had
open questions at a previous event and I
got grilled about the PSF funding
cycles. I'm very happy to talk about the
PSF. I'm happy to talk about Anaconda,
developer relations, uh my journey in
the ecosystem, scientific computing, any
of any and all of the things.
Any questions?
I got one in the front.
There's one right there if you want to
or I can also repeat it for the for the
audience. Awesome.
>> What is the minimum age for becoming a
PSF member?
>> There is no minimum age for becoming a
PSF member.
>> Anyone can become.
Great question. [laughter]
Some like to know what the PSF does.
Some wants to know what it's like being
the chair of the PSF.
Uh some have asked me why I'm not going
to be the chair of the PSF in the next
coming election. Why I didn't run again.
>> Hello. Yeah.
>> Yeah. Okay. Got one.
>> Uh we pretty much talked about uh
packaging.
>> Yes.
>> So pip was part of uh Python, right? Uh
we we we get that as we install the
Python.
>> Yes. And now we have many things UV
poetry.
>> Are we having any proposal to include
any of these tools into Python?
>> No. So it wouldn't be to include any of
the tools. So there is uh PEP 772 which
is the Python packaging council. Um so
originally the Python steering council
which is the organization that manages
all of the PEPs and that will decide
what um exists outside of Python and
what will bring brought into Python.
There was one person who handled all of
the packaging needs that handled PEPs.
It was just not enough bandwidth. So
what they did was they elected to have
spread it out into I believe five people
who will exist on the packaging council
and they will decide um any packaging
standards um right now what I know is on
their docket um but they have a list of
intricacies um is a standard packaging
format. Yeah, back then we had uh easy
initi then we have
>> yeah I don't think they're going to
bring bring in any of the existing
tools. There is the PIPA which supports
some of the tools like PIP and maybe
poetry some others. Yeah. So yeah, I
don't I don't believe they're going to
be bringing in any of those tools into
the Python ecosystem.
>> Thank you.
>> Yeah.
>> Yeah. Hey. Uh, so I uh so I recently
heard of this uh
a new organization that formed called
the Python Asia organization.
>> Yeah.
>> Uh is it separate from the PSF? What's
the relation here?
>> They're separate from the PSF. So the
PSF is a North American nonp or it's a
US nonprofit um that is uh its creed is
to its mission to support Python
everywhere. So it is within our
boundaries to uh support externally but
we are a US nonprofit and so we're
governed by those rules and so there's
uh Python Brazil as well and Euro Python
the Europyon Society for example those
are all uh considered sister
organizations like Python Asia which is
exciting yay Python Asia so that's good
yeah not the PSF
>> so how do the like how does the dynamic
work like do they work with each other
like is there a structure there or
>> there's a lot of people who volunteer
for the Python Software Foundation who
also volunteer for Python Asia um but
they are two separate groups and the
interaction is um completely dependent
upon the organizers and there's no
formal interaction right now it's they
we just leverage each other's resources
and we're friends in the ecosystem and
we share ideas
>> that Okay. Yeah.
>> Hey, good evening. Hi.
>> First of all, a great session. Thank you
for that.
>> Uh, so my question is a little bit
irrelevant to this.
>> So it basically I am working in Golang
as well as Python.
>> Okay.
>> So u we have seen a huge difference
between how much Python can do and how
much Golang can do in terms of
concurrency.
So the point is that uh are we doing
something to improve that in the
upcoming versions of Python or uh how
are we going to do uh you know how do we
optimize it uh you know to get better
performance out of Python.
>> Yeah. Yeah. So there's so I got good
news and and bad news on that one. So
there was a team that was spearheaded by
Guido Van Rossim at Microsoft called the
faster C Python team um that just came
to a close this year. Um a lot of their
performance improvements or some of
their performance improvements will be
um made it into 311 um 312 and 313. Um
we'll also see some of those in 314 that
includes uh garbage collection for
example. I have a whole blog post on the
other four that I think are really cool,
but they're all very performance
improvement heavy. Um, where the rest of
the performance improvements are going
to go, I'm we're not sure, but there's
definitely PEPs um to facilitate this.
People are excited about the JIT. Um,
people are really excited about sub
interpreters. Um, also free threaded
Python. People are really excited about
that. All provide some performance
improvements.
Uh just a followup. So uh I mean we are
all here because we love Python. So
uh any chance we should you know we
should be worried about you know Python
becoming a threatened sort of a language
you know just like the old languages
went away something like that.
>> That's an interesting question.
I am
I'm of the mind that we should always
remain vigilant on uh where um mind
share could go outside of Python. Um I
think there's a strategic advantage
because I've invested so much of my time
into Python. Um there is a strategic
advantage to being careful and
thoughtful and wanting to continue to
make Python the best language there is.
Um, but also I would love to challenge
that question back a bit and my own um
predition predisposition for wanting to
keep Python great and say that whatever
comes next, if there is something that
comes next, a better iteration um is
hopefully comes of the the vein of open
source and innovation and um puts us in
a place all of us in a place to pivot in
a new direction and do more. So I I love
open source. Python, just because I love
Python doesn't mean I necessarily hate
TypeScript um for example. And so if we
do it in open source and if we do it in
the spirit of collaboration, I'm really
excited for whatever comes next. But I
do love Python and I want to keep it
great. So it's fair. Um I'm not
particularly worried. I think that we
absolutely have the mind share right now
and it only looks like we're going to
keep going up. Um, but I think the next
wave is going to be making sure that
Python is optimized for the browser. So,
I'm super excited for Web Assembly being
a P platform for Python.
>> Great. Great. Thank you. Thank you so
much.
>> Any more? Oh, I see another one. Hi.
>> Hi. Uh, I just have one.
>> Okay. Uh,
>> last question or we have to be done.
Okay. I'm so sorry. Let's Let's go find
each other in the hallway and I'm happy
to chat.
>> Okay. All right. Thanks everyone.
>> Thank you, Don.