py2pack ignores requirements.txt?

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

py2pack ignores requirements.txt?

Adam Spiers
Is it just me or does py2pack really ignore requirements.txt files and
assume that dependencies are only stated within setup.py?  I have
filed this as

  https://github.com/saschpe/py2pack/issues/25

but that would seem like such a fundamental feature gap that it seems
more likely that I'm missing something (after all I am a Python
packaging newbie), otherwise presumably py2pack would be useless for
many modules.
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: py2pack ignores requirements.txt?

Jason Craig-3
On 04/01/2015 02:10 PM, Adam Spiers wrote:

> Is it just me or does py2pack really ignore requirements.txt files and
> assume that dependencies are only stated within setup.py?  I have
> filed this as
>
>    https://github.com/saschpe/py2pack/issues/25
>
> but that would seem like such a fundamental feature gap that it seems
> more likely that I'm missing something (after all I am a Python
> packaging newbie), otherwise presumably py2pack would be useless for
> many modules.
>

Packages declare their dependencies in setup.py, so that is where
py2pack looks for them, just like setuptools and pip. As far as I
understand it, a requirements file is more for developing a set of
packages with versions known to work well in a certain
configuration--e.g. if you are deploying a web application with a
complex list of requirements that is only really tested with a specific
set of versions of those requirements, something of that nature.

I think this behavior is by design.

--
Jason Craig
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: py2pack ignores requirements.txt?

Adam Spiers
Jason Craig <[hidden email]> wrote:

> On 04/01/2015 02:10 PM, Adam Spiers wrote:
> >Is it just me or does py2pack really ignore requirements.txt files and
> >assume that dependencies are only stated within setup.py?  I have
> >filed this as
> >
> >   https://github.com/saschpe/py2pack/issues/25
> >
> >but that would seem like such a fundamental feature gap that it seems
> >more likely that I'm missing something (after all I am a Python
> >packaging newbie), otherwise presumably py2pack would be useless for
> >many modules.
> >
>
> Packages declare their dependencies in setup.py, so that is where
> py2pack looks for them, just like setuptools and pip.

But setup.py is just code, so it can invoke code in other files in
order to determine the dependencies.

> As far as I
> understand it, a requirements file is more for developing a set of
> packages with versions known to work well in a certain
> configuration--e.g. if you are deploying a web application with a
> complex list of requirements that is only really tested with a
> specific set of versions of those requirements, something of that
> nature.
>
> I think this behavior is by design.

If so, IMHO this design is broken because it excludes the pbr
approach, and the OpenStack community is a significant contributor to
the Python ecosystem:

  http://docs.openstack.org/developer/pbr/#requirements
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: py2pack ignores requirements.txt?

todd rme
In reply to this post by Jason Craig-3
On Thu, Apr 2, 2015 at 12:04 AM, Jason Craig <[hidden email]> wrote:

> On 04/01/2015 02:10 PM, Adam Spiers wrote:
>>
>> Is it just me or does py2pack really ignore requirements.txt files and
>> assume that dependencies are only stated within setup.py?  I have
>> filed this as
>>
>>    https://github.com/saschpe/py2pack/issues/25
>>
>> but that would seem like such a fundamental feature gap that it seems
>> more likely that I'm missing something (after all I am a Python
>> packaging newbie), otherwise presumably py2pack would be useless for
>> many modules.
>>
>
> Packages declare their dependencies in setup.py, so that is where py2pack
> looks for them, just like setuptools and pip. As far as I understand it, a
> requirements file is more for developing a set of packages with versions
> known to work well in a certain configuration--e.g. if you are deploying a
> web application with a complex list of requirements that is only really
> tested with a specific set of versions of those requirements, something of
> that nature.
>
> I think this behavior is by design.

A lot of packages have a setup.y file that reads in the
requirements.txt file and uses it for the dependencies.  py2pack
doesn't handle this situation.

At the very least, the contents of requirements.txt could be put in
Recommends (or Suggests, although I don't like Suggests because YaST
doesn't support it).
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: py2pack ignores requirements.txt?

Jason Craig-3
In reply to this post by Adam Spiers
On 04/01/2015 04:26 PM, Adam Spiers wrote:
> Jason Craig <[hidden email]> wrote:
 >
>> Packages declare their dependencies in setup.py, so that is where
>> py2pack looks for them, just like setuptools and pip.
>
> But setup.py is just code, so it can invoke code in other files in
> order to determine the dependencies.

Is that a problem?

>> As far as I
>> understand it, a requirements file is more for developing a set of
>> packages with versions known to work well in a certain
>> configuration--e.g. if you are deploying a web application with a
>> complex list of requirements that is only really tested with a
>> specific set of versions of those requirements, something of that
>> nature.
>>
>> I think this behavior is by design.
>
> If so, IMHO this design is broken because it excludes the pbr
> approach, and the OpenStack community is a significant contributor to
> the Python ecosystem:
>
>    http://docs.openstack.org/developer/pbr/#requirements
>

Maybe the pbr approach is broken. I am rather fond of this post
https://caremad.io/2013/07/setup-vs-requirement/ which explains why
requirements.txt perhaps should be used differently than setup.py.
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: py2pack ignores requirements.txt?

Jason Craig-3
In reply to this post by todd rme
On 04/02/2015 02:54 AM, Todd Rme wrote:
>
> A lot of packages have a setup.y file that reads in the
> requirements.txt file and uses it for the dependencies.  py2pack
> doesn't handle this situation.
>
> At the very least, the contents of requirements.txt could be put in
> Recommends (or Suggests, although I don't like Suggests because YaST
> doesn't support it).
>

OK, but that still doesn't necessarily tell me that py2pack should be
changed to read requirements.txt. To me it says that py2pack should be
modified such that it will properly handle a setup.py file that builds
the install_requires argument through any outside means, including by
reading the requirements.txt file.

Also, is the universe of such packages that big? I really have no idea
of that.

--
Jason Craig
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: py2pack ignores requirements.txt?

Adam Spiers
In reply to this post by Jason Craig-3
Jason Craig <[hidden email]> wrote:

> On 04/01/2015 04:26 PM, Adam Spiers wrote:
> >Jason Craig <[hidden email]> wrote:
> >
> >>Packages declare their dependencies in setup.py, so that is where
> >>py2pack looks for them, just like setuptools and pip.
> >
> >But setup.py is just code, so it can invoke code in other files in
> >order to determine the dependencies.
>
> Is that a problem?

It is currently, because py2pack only parses setup.py.  There is a
pull request which fixes this:

  https://github.com/saschpe/py2pack/pull/9

It has been open since October 2013 :-/

> >>As far as I
> >>understand it, a requirements file is more for developing a set of
> >>packages with versions known to work well in a certain
> >>configuration--e.g. if you are deploying a web application with a
> >>complex list of requirements that is only really tested with a
> >>specific set of versions of those requirements, something of that
> >>nature.
> >>
> >>I think this behavior is by design.
> >
> >If so, IMHO this design is broken because it excludes the pbr
> >approach, and the OpenStack community is a significant contributor to
> >the Python ecosystem:
> >
> >   http://docs.openstack.org/developer/pbr/#requirements
>
> Maybe the pbr approach is broken. I am rather fond of this post
> https://caremad.io/2013/07/setup-vs-requirement/ which explains why
> requirements.txt perhaps should be used differently than setup.py.

Very interesting, thanks for that link!  It turns out that the post
author is active in the OpenStack community and sent this to the pbr
author (Monty Taylor, a.k.a. mordred) after writing it:

    2013-07-23T08:17:55  <dstufft> Also this was partially inspired by you guys ;) https://caremad.io/blog/setup-vs-requirements/ /cc mordred

    -- http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2013-07-23.log

The link to the corresponding Ruby post by Yehuda Katz is very
enlightening.  IIUC, this highlights a deficiency in the Python
ecosystem's approach to packaging, due to the absence of any
equivalent of Gemfile.lock.

But yes, I am beginning to come round to the idea that maybe pbr's
approach is broken.
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: py2pack ignores requirements.txt?

Adam Spiers
In reply to this post by Jason Craig-3
Jason Craig <[hidden email]> wrote:

> On 04/02/2015 02:54 AM, Todd Rme wrote:
> >
> >A lot of packages have a setup.y file that reads in the
> >requirements.txt file and uses it for the dependencies.  py2pack
> >doesn't handle this situation.
> >
> >At the very least, the contents of requirements.txt could be put in
> >Recommends (or Suggests, although I don't like Suggests because YaST
> >doesn't support it).
>
> OK, but that still doesn't necessarily tell me that py2pack should
> be changed to read requirements.txt. To me it says that py2pack
> should be modified such that it will properly handle a setup.py file
> that builds the install_requires argument through any outside means,
> including by reading the requirements.txt file.

Right - that's what I said, and that's what

  https://github.com/saschpe/py2pack/pull/9

is for.
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]