Packaging python singlespec applications

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Packaging python singlespec applications

todd rme
According to python packaging guidelines, python-based GUI
applications should not have the "python-", "python2-", or "python3-"
prefix. Ideally this wouldn't be a problem, since these are not
dependent on the python version they use. However, a lot of them turn
out to be either IDEs (e.g. Spyder), shells (e.g. jupyter's qtconsole
or bpython), or other development tools that are dependent on the
specific version of python they are built against.

Such packages are hard to use with singlespec macros. One option would
simply be to change or not follow the package naming guidelines for
such packages. Some packages, such as jupyter, do not follow them, but
others like spyder do.

Another option would be to make the macros more useful with such
packages.  If we go this route, I see four changes that would be
extremely helpful in this regard.

First, it would be nice if %python_subpackages allowed us to specify
additional packages to manage. These would be packages named using
"%package -n foo" that we nevertheless want to have multiple versions
of. The macros would then create a set of packages
"foo-%{python_bin_suffix}". So for example we might say
%{python_subpackages spyder}. If we did that, we would get
"spyder-3.6" and "spyder-2.7", with Requires, Provides, Obsoletes, and
%files all handled appropriately for the given python version.

Second, it would be nice if there was some way to handle "%files -f
file". This is pretty much mandatory for -lang packages. So perhaps
something as bare-bones as allowing %files %{python_files foo -f
foo-%{$python_bin_suffix}.bar}, or something more advanced like
%{python_files foo -f foo.bar} which automatically inserts
%{$python_bin_suffix}

Third, it would be nice if %python_clone handled .desktop files and
appdata.xml files, or there was some alternative macro for these.

Fourth, it would be nice if there was a singlespec-aware version of
%find_lang and perhaps even %lang_package. %find_lang is easy enough
to handle manually with grep, but I have not found a good solution for
%lang_package.
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Packaging python singlespec applications

Robert Schweikert-6
On 09/12/2017 03:28 PM, Todd Rme wrote:

> According to python packaging guidelines, python-based GUI
> applications should not have the "python-", "python2-", or "python3-"
> prefix. Ideally this wouldn't be a problem, since these are not
> dependent on the python version they use. However, a lot of them turn
> out to be either IDEs (e.g. Spyder), shells (e.g. jupyter's qtconsole
> or bpython), or other development tools that are dependent on the
> specific version of python they are built against.
>
> Such packages are hard to use with singlespec macros. One option would
> simply be to change or not follow the package naming guidelines for
> such packages. Some packages, such as jupyter, do not follow them, but
> others like spyder do.
>
> Another option would be to make the macros more useful with such
> packages.  If we go this route, I see four changes that would be
> extremely helpful in this regard.
>
> First, it would be nice if %python_subpackages allowed us to specify
> additional packages to manage. These would be packages named using
> "%package -n foo" that we nevertheless want to have multiple versions
> of. The macros would then create a set of packages
> "foo-%{python_bin_suffix}". So for example we might say
> %{python_subpackages spyder}. If we did that, we would get
> "spyder-3.6" and "spyder-2.7",
But is that really that useful?

For higher level stuff, even if they put things in site-packages I have
started to just simply switch builds to python3. It loads the spec files
with if condiftions but SLES 12 is going away in only 7 years ;) and
then we'll be rid of Python 2.

Later,
Robert

--
Robert Schweikert                   MAY THE SOURCE BE WITH YOU
Distinguished Architect                       LINUX
Team Lead Public Cloud
[hidden email]
IRC: robjo


signature.asc (484 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Packaging python singlespec applications

todd rme
On Tue, Sep 12, 2017 at 4:56 PM, Robert Schweikert <[hidden email]> wrote:

> On 09/12/2017 03:28 PM, Todd Rme wrote:
>> According to python packaging guidelines, python-based GUI
>> applications should not have the "python-", "python2-", or "python3-"
>> prefix. Ideally this wouldn't be a problem, since these are not
>> dependent on the python version they use. However, a lot of them turn
>> out to be either IDEs (e.g. Spyder), shells (e.g. jupyter's qtconsole
>> or bpython), or other development tools that are dependent on the
>> specific version of python they are built against.
>>
>> Such packages are hard to use with singlespec macros. One option would
>> simply be to change or not follow the package naming guidelines for
>> such packages. Some packages, such as jupyter, do not follow them, but
>> others like spyder do.
>>
>> Another option would be to make the macros more useful with such
>> packages.  If we go this route, I see four changes that would be
>> extremely helpful in this regard.
>>
>> First, it would be nice if %python_subpackages allowed us to specify
>> additional packages to manage. These would be packages named using
>> "%package -n foo" that we nevertheless want to have multiple versions
>> of. The macros would then create a set of packages
>> "foo-%{python_bin_suffix}". So for example we might say
>> %{python_subpackages spyder}. If we did that, we would get
>> "spyder-3.6" and "spyder-2.7",
>
> But is that really that useful?
>
> For higher level stuff, even if they put things in site-packages I have
> started to just simply switch builds to python3. It loads the spec files
> with if condiftions but SLES 12 is going away in only 7 years ;) and
> then we'll be rid of Python 2.

Depends on the higher-level stuff in question. For things like IDEs,
shells, and debuggers having both python2 and python3 versions is
usually essential since they are usually tied to the python version
they were built against.
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Packaging python singlespec applications

jan matejek-4
In reply to this post by todd rme
hello,

On 12.9.2017 21:28, Todd Rme wrote:
> First, it would be nice if %python_subpackages allowed us to specify
> additional packages to manage. These would be packages named using
> "%package -n foo" that we nevertheless want to have multiple versions
> of. The macros would then create a set of packages
> "foo-%{python_bin_suffix}". So for example we might say
> %{python_subpackages spyder}. If we did that, we would get
> "spyder-3.6" and "spyder-2.7", with Requires, Provides, Obsoletes, and
> %files all handled appropriately for the given python version.

I kind of like this, but don't quite understand how you would use it.
You're presuming the package is called "spyder"? or "python-spyder"? what is the subpackage here?

> Second, it would be nice if there was some way to handle "%files -f
> file". This is pretty much mandatory for -lang packages. So perhaps
> something as bare-bones as allowing %files %{python_files foo -f
> foo-%{$python_bin_suffix}.bar}, or something more advanced like
> %{python_files foo -f foo.bar} which automatically inserts
> %{$python_bin_suffix}

Trouble with %files -f is that there is no good way to process the filelist -- it is generated at
build time, but singlespec macros run at preprocessing time.
A solution might be having a separate step in the %install section that would generate/convert the
separate filelists?

> Third, it would be nice if %python_clone handled .desktop files and
> appdata.xml files, or there was some alternative macro for these.

That probably can be done. Please file a bug at github.com/opensuse/python-rpm-macros

> Fourth, it would be nice if there was a singlespec-aware version of
> %find_lang and perhaps even %lang_package. %find_lang is easy enough
> to handle manually with grep, but I have not found a good solution for
> %lang_package.

There's an existing bug for %find_lang. %lang_package probably can be folded under that.

regards
m.


signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Packaging python singlespec applications

todd rme
On Wed, Sep 13, 2017 at 7:42 AM, jan matejek <[hidden email]> wrote:

> hello,
>
> On 12.9.2017 21:28, Todd Rme wrote:
>> First, it would be nice if %python_subpackages allowed us to specify
>> additional packages to manage. These would be packages named using
>> "%package -n foo" that we nevertheless want to have multiple versions
>> of. The macros would then create a set of packages
>> "foo-%{python_bin_suffix}". So for example we might say
>> %{python_subpackages spyder}. If we did that, we would get
>> "spyder-3.6" and "spyder-2.7", with Requires, Provides, Obsoletes, and
>> %files all handled appropriately for the given python version.
>
> I kind of like this, but don't quite understand how you would use it.
> You're presuming the package is called "spyder"? or "python-spyder"? what is the subpackage here?

So you would still have "Name: python-spyder" that would be handled
normally (although it might not have a %files section).  There would
be a subpackage  "%package -n spyder". The "spyder" %package,
%description, %files, etc. would be replaced by
"spyder-%{python2_bin_suffix}" and "spyder-%{python3_bin_suffix}".

>> Second, it would be nice if there was some way to handle "%files -f
>> file". This is pretty much mandatory for -lang packages. So perhaps
>> something as bare-bones as allowing %files %{python_files foo -f
>> foo-%{$python_bin_suffix}.bar}, or something more advanced like
>> %{python_files foo -f foo.bar} which automatically inserts
>> %{$python_bin_suffix}
>
> Trouble with %files -f is that there is no good way to process the filelist -- it is generated at
> build time, but singlespec macros run at preprocessing time.
> A solution might be having a separate step in the %install section that would generate/convert the
> separate filelists?

Let's use python-numpy as an example. My understanding is that at
preprocessing time something like "%files %{python_files}" is
converted to, for example, "%files -n python3-numpy". My idea is that
something like "%files %{python_files -f foo.bar}" would be converted
to "%files -n python3-numpy -f foo-%{python3_bin_suffix}.bar". So it
would still be done at the preprocessing stage, as only the file name
would need to be edited.  The actual file contents would be handled by
the normal %files macro.

>> Third, it would be nice if %python_clone handled .desktop files and
>> appdata.xml files, or there was some alternative macro for these.
>
> That probably can be done. Please file a bug at github.com/opensuse/python-rpm-macros

Will do.

>> Fourth, it would be nice if there was a singlespec-aware version of
>> %find_lang and perhaps even %lang_package. %find_lang is easy enough
>> to handle manually with grep, but I have not found a good solution for
>> %lang_package.
>
> There's an existing bug for %find_lang. %lang_package probably can be folded under that.

Okay
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Packaging python singlespec applications

jan matejek-4
On 13.9.2017 17:18, Todd Rme wrote:

> On Wed, Sep 13, 2017 at 7:42 AM, jan matejek <[hidden email]> wrote:
>> hello,
>>
>> On 12.9.2017 21:28, Todd Rme wrote:
>>> First, it would be nice if %python_subpackages allowed us to specify
>>> additional packages to manage. These would be packages named using
>>> "%package -n foo" that we nevertheless want to have multiple versions
>>> of. The macros would then create a set of packages
>>> "foo-%{python_bin_suffix}". So for example we might say
>>> %{python_subpackages spyder}. If we did that, we would get
>>> "spyder-3.6" and "spyder-2.7", with Requires, Provides, Obsoletes, and
>>> %files all handled appropriately for the given python version.
>>
>> I kind of like this, but don't quite understand how you would use it.
>> You're presuming the package is called "spyder"? or "python-spyder"? what is the subpackage here?
>
> So you would still have "Name: python-spyder" that would be handled
> normally (although it might not have a %files section).  There would
> be a subpackage  "%package -n spyder". The "spyder" %package,
> %description, %files, etc. would be replaced by
> "spyder-%{python2_bin_suffix}" and "spyder-%{python3_bin_suffix}".
Why is the master package called "python-spyder"?

Wouldn't it be better, in this case, to change the logic for packages that aren't called "python-foo"?
Like, "python-foo" converts to "python2-foo" and "python3-foo"
whereas "spyder" converts to "spyder-2.7" and "spyder-3.7"
?

Is there a usecase where we want both of these behaviors?
If yes, maybe this should be done by a single option switch to %python_subpackages.

I do have the vague idea that more explicit control of %python_subpackages is desirable, but ISTM i
don't have enough usecases yet to formulate how that would work.

> Let's use python-numpy as an example. My understanding is that at
> preprocessing time something like "%files %{python_files}" is
> converted to, for example, "%files -n python3-numpy". My idea is that
> something like "%files %{python_files -f foo.bar}" would be converted
> to "%files -n python3-numpy -f foo-%{python3_bin_suffix}.bar". So it
> would still be done at the preprocessing stage, as only the file name
> would need to be edited.  The actual file contents would be handled by
> the normal %files macro.

That's certainly possible, but what if the actual file contents need to be modified?
In the most common usecase, I expect that the generated file would contain results for all flavors
rolled together.

...of course, we can add what you're suggesting in the meantime and leave the modification question
to the %install section as-is

regards
m.


signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Packaging python singlespec applications

todd rme
On Mon, Sep 18, 2017 at 2:10 PM, jan matejek <[hidden email]> wrote:

> On 13.9.2017 17:18, Todd Rme wrote:
>> On Wed, Sep 13, 2017 at 7:42 AM, jan matejek <[hidden email]> wrote:
>>> hello,
>>>
>>> On 12.9.2017 21:28, Todd Rme wrote:
>>>> First, it would be nice if %python_subpackages allowed us to specify
>>>> additional packages to manage. These would be packages named using
>>>> "%package -n foo" that we nevertheless want to have multiple versions
>>>> of. The macros would then create a set of packages
>>>> "foo-%{python_bin_suffix}". So for example we might say
>>>> %{python_subpackages spyder}. If we did that, we would get
>>>> "spyder-3.6" and "spyder-2.7", with Requires, Provides, Obsoletes, and
>>>> %files all handled appropriately for the given python version.
>>>
>>> I kind of like this, but don't quite understand how you would use it.
>>> You're presuming the package is called "spyder"? or "python-spyder"? what is the subpackage here?
>>
>> So you would still have "Name: python-spyder" that would be handled
>> normally (although it might not have a %files section).  There would
>> be a subpackage  "%package -n spyder". The "spyder" %package,
>> %description, %files, etc. would be replaced by
>> "spyder-%{python2_bin_suffix}" and "spyder-%{python3_bin_suffix}".
>
> Why is the master package called "python-spyder"?
>
> Wouldn't it be better, in this case, to change the logic for packages that aren't called "python-foo"?
> Like, "python-foo" converts to "python2-foo" and "python3-foo"
> whereas "spyder" converts to "spyder-2.7" and "spyder-3.7"
> ?
>
> Is there a usecase where we want both of these behaviors?
> If yes, maybe this should be done by a single option switch to %python_subpackages.
>
> I do have the vague idea that more explicit control of %python_subpackages is desirable, but ISTM i
> don't have enough usecases yet to formulate how that would work.

I guess either approach is really fine. My thinking was too keep
things as close as possible to how things work currently.

>> Let's use python-numpy as an example. My understanding is that at
>> preprocessing time something like "%files %{python_files}" is
>> converted to, for example, "%files -n python3-numpy". My idea is that
>> something like "%files %{python_files -f foo.bar}" would be converted
>> to "%files -n python3-numpy -f foo-%{python3_bin_suffix}.bar". So it
>> would still be done at the preprocessing stage, as only the file name
>> would need to be edited.  The actual file contents would be handled by
>> the normal %files macro.
>
> That's certainly possible, but what if the actual file contents need to be modified?
> In the most common usecase, I expect that the generated file would contain results for all flavors
> rolled together.
>
> ...of course, we can add what you're suggesting in the meantime and leave the modification question
> to the %install section as-is

Although it is likely that the contents will need to be modified, I
think there is enough variability in the details that handling this in
the %files section is probably too limiting. I would rather have that
logic handled in %install.
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Packaging python singlespec applications

Simon Lees-3
In reply to this post by jan matejek-4


On 19/09/17 03:40, jan matejek wrote:

> On 13.9.2017 17:18, Todd Rme wrote:
>> On Wed, Sep 13, 2017 at 7:42 AM, jan matejek <[hidden email]> wrote:
>>> hello,
>>>
>>> On 12.9.2017 21:28, Todd Rme wrote:
>>>> First, it would be nice if %python_subpackages allowed us to specify
>>>> additional packages to manage. These would be packages named using
>>>> "%package -n foo" that we nevertheless want to have multiple versions
>>>> of. The macros would then create a set of packages
>>>> "foo-%{python_bin_suffix}". So for example we might say
>>>> %{python_subpackages spyder}. If we did that, we would get
>>>> "spyder-3.6" and "spyder-2.7", with Requires, Provides, Obsoletes, and
>>>> %files all handled appropriately for the given python version.
>>>
>>> I kind of like this, but don't quite understand how you would use it.
>>> You're presuming the package is called "spyder"? or "python-spyder"? what is the subpackage here?
>>
>> So you would still have "Name: python-spyder" that would be handled
>> normally (although it might not have a %files section).  There would
>> be a subpackage  "%package -n spyder". The "spyder" %package,
>> %description, %files, etc. would be replaced by
>> "spyder-%{python2_bin_suffix}" and "spyder-%{python3_bin_suffix}".
>
> Why is the master package called "python-spyder"?
>
> Wouldn't it be better, in this case, to change the logic for packages that aren't called "python-foo"?
> Like, "python-foo" converts to "python2-foo" and "python3-foo"
> whereas "spyder" converts to "spyder-2.7" and "spyder-3.7"
> ?
>
> Is there a usecase where we want both of these behaviors?
> If yes, maybe this should be done by a single option switch to %python_subpackages.
I would have thought the better question is do we really need 2 versions
of python applications 1 built with each, as a user of spyder or any
other python app why would I care if its built with python2 or python3
as long as it works. When I was doing Qt app development as my day job I
didn't care which version of Qt qtcreator was built with because it was
independent of the version I was using, maybe all these apps not
starting with python- should just build for whatever we determine is the
default python version at the time, and then a macro could be used to
change to only generating for one fixed version as specified by the
macro ie if you wanted to temporarily keep building with python2 as
python3 has issues. Then once those issues are fixed its just one flag
in a macro to change.

Currently all the python apps I maintain just build with python3 not
using single spec, partly because they were migrated before single spec
but also because there is no point in shipping 2 versions built with
different versions of python.

--

Simon Lees (Simotek)                            http://simotek.net

Emergency Update Team                           keybase.io/simotek
SUSE Linux                           Adelaide Australia, UTC+10:30
GPG Fingerprint: 5B87 DB9D 88DC F606 E489 CEC5 0922 C246 02F0 014B


signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Packaging python singlespec applications

todd rme
On Tue, Sep 19, 2017 at 1:08 AM, Simon Lees <[hidden email]> wrote:

>
>
> On 19/09/17 03:40, jan matejek wrote:
>> On 13.9.2017 17:18, Todd Rme wrote:
>>> On Wed, Sep 13, 2017 at 7:42 AM, jan matejek <[hidden email]> wrote:
>>>> hello,
>>>>
>>>> On 12.9.2017 21:28, Todd Rme wrote:
>>>>> First, it would be nice if %python_subpackages allowed us to specify
>>>>> additional packages to manage. These would be packages named using
>>>>> "%package -n foo" that we nevertheless want to have multiple versions
>>>>> of. The macros would then create a set of packages
>>>>> "foo-%{python_bin_suffix}". So for example we might say
>>>>> %{python_subpackages spyder}. If we did that, we would get
>>>>> "spyder-3.6" and "spyder-2.7", with Requires, Provides, Obsoletes, and
>>>>> %files all handled appropriately for the given python version.
>>>>
>>>> I kind of like this, but don't quite understand how you would use it.
>>>> You're presuming the package is called "spyder"? or "python-spyder"? what is the subpackage here?
>>>
>>> So you would still have "Name: python-spyder" that would be handled
>>> normally (although it might not have a %files section).  There would
>>> be a subpackage  "%package -n spyder". The "spyder" %package,
>>> %description, %files, etc. would be replaced by
>>> "spyder-%{python2_bin_suffix}" and "spyder-%{python3_bin_suffix}".
>>
>> Why is the master package called "python-spyder"?
>>
>> Wouldn't it be better, in this case, to change the logic for packages that aren't called "python-foo"?
>> Like, "python-foo" converts to "python2-foo" and "python3-foo"
>> whereas "spyder" converts to "spyder-2.7" and "spyder-3.7"
>> ?
>>
>> Is there a usecase where we want both of these behaviors?
>> If yes, maybe this should be done by a single option switch to %python_subpackages.
>
> I would have thought the better question is do we really need 2 versions
> of python applications 1 built with each, as a user of spyder or any
> other python app why would I care if its built with python2 or python3
> as long as it works.

The problem is that you cannot use spyder3 to do python2 development
and vice versus.  As long as the spyder upstream supports both python2
and python3 and we as a distro are providing both python2 and python3
in general, I think the spyder IDE should support both as well.
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]