singlespec python and SLE_12_SP2_Backports

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

singlespec python and SLE_12_SP2_Backports

todd rme
In devel:languages:python, the new singlespec approach is causing
problems with SLE_12_SP2_Backports build target.  The issue is that
this build target fails if a package includes files from any package
in SLE, and packages that have this problem include some really
critical ones like setuptools.

This is a problem because in most cases only the python2 versions of
the files are available from SLE, but the build failure means the
python3 versions do not get published.  This results in critical
python3 packages needed for singlespec builds not being available.
Currently more than half of the packages are either disabled or
unresolvable, and that number is going to grow to almost every package
as the singlespec conversion continues.

The simplest solution would be to just disable that repo by default.
However, if there (or could be) some way to selectively disable
building python2 versions of specific packages just for that build
target it might be better.
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: singlespec python and SLE_12_SP2_Backports

jan matejek-4
On 10.4.2017 17:25, Todd Rme wrote:

> In devel:languages:python, the new singlespec approach is causing
> problems with SLE_12_SP2_Backports build target.  The issue is that
> this build target fails if a package includes files from any package
> in SLE, and packages that have this problem include some really
> critical ones like setuptools.
>
> This is a problem because in most cases only the python2 versions of
> the files are available from SLE, but the build failure means the
> python3 versions do not get published.  This results in critical
> python3 packages needed for singlespec builds not being available.
> Currently more than half of the packages are either disabled or
> unresolvable, and that number is going to grow to almost every package
> as the singlespec conversion continues.
Does it mean that we are not allowed to use "python2-setuptools" from local repository because
"python-setuptools" is part of SLE and should be used in the SLE version?
Or does the build of "python2-setuptools" fail because the resulting RPM has file conflicts with a
package present in SLE?

In any case, I'm confused as to what is the purpose of the Backports repository. Are people expected
to add it straight from OBS and install OBS-built RPMs? So they get "python-foo" version from SLE
and "python3-foo" from d:l:py? What if the SLE version is too old, or the d:l:py version is too new?
Would only building "python3-foo" help at all?

> The simplest solution would be to just disable that repo by default.
> However, if there (or could be) some way to selectively disable
> building python2 versions of specific packages just for that build
> target it might be better.

I know that you can add per-repository macros in prjconf. Not sure about per-package. Adding this
configuration to the individual spec files seems ugly but possible if necessary... In any case, it
would be a question of "%undefine have_python2" and fixing the bugs that prevent it from working.

We could also create a custom version of python-rpm-macros that contains the "python2 blacklist" as
part of itself. As long as it is limited to d:l:py(SLE12_Backports), this might very well be the
cleanest solution -- no need to touch individual specs.

m.



signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: singlespec python and SLE_12_SP2_Backports

todd rme
On Wed, Apr 12, 2017 at 8:58 AM, jan matejek <[hidden email]> wrote:

> On 10.4.2017 17:25, Todd Rme wrote:
>> In devel:languages:python, the new singlespec approach is causing
>> problems with SLE_12_SP2_Backports build target.  The issue is that
>> this build target fails if a package includes files from any package
>> in SLE, and packages that have this problem include some really
>> critical ones like setuptools.
>>
>> This is a problem because in most cases only the python2 versions of
>> the files are available from SLE, but the build failure means the
>> python3 versions do not get published.  This results in critical
>> python3 packages needed for singlespec builds not being available.
>> Currently more than half of the packages are either disabled or
>> unresolvable, and that number is going to grow to almost every package
>> as the singlespec conversion continues.
>
> Does it mean that we are not allowed to use "python2-setuptools" from local repository because
> "python-setuptools" is part of SLE and should be used in the SLE version?
> Or does the build of "python2-setuptools" fail because the resulting RPM has file conflicts with a
> package present in SLE?

The latter. It checks individual files and if those files are already
in SLE, it will fail.

> In any case, I'm confused as to what is the purpose of the Backports repository. Are people expected
> to add it straight from OBS and install OBS-built RPMs?

It is described here: https://en.opensuse.org/Portal:Backports

Basically, the idea seems to be to guarantee that no potentially
incompatible updates of files in SLE get included. So you can be 100%
sure that these are only add-ons with nothing that could interfere
with the tested, support SLE packages.

> What if the SLE version is too old, or the d:l:py version is too new?

Then we would probably need to manually disable that package.

> Would only building "python3-foo" help at all?

For most critical packages that should solve the problem.

>> The simplest solution would be to just disable that repo by default.
>> However, if there (or could be) some way to selectively disable
>> building python2 versions of specific packages just for that build
>> target it might be better.
>
> We could also create a custom version of python-rpm-macros that contains the "python2 blacklist" as
> part of itself. As long as it is limited to d:l:py(SLE12_Backports), this might very well be the
> cleanest solution -- no need to touch individual specs.

That is a good idea. Maybe we could create link called
"python-rpm-macros_SLE12_Backports" that includes a package diff that
adds the blacklist? Then we could disable python-rpm-macros on
SLE12_Backports and enable python-rpm-macros_SLE12_Backports only for
SLE12_Backports.
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: singlespec python and SLE_12_SP2_Backports

Bruno Friedmann-2
> >> The simplest solution would be to just disable that repo by default.
> >> However, if there (or could be) some way to selectively disable
> >> building python2 versions of specific packages just for that build
> >> target it might be better.
> >
> > We could also create a custom version of python-rpm-macros that contains
> > the "python2 blacklist" as part of itself. As long as it is limited to
> > d:l:py(SLE12_Backports), this might very well be the cleanest solution --
> > no need to touch individual specs.
>
> That is a good idea. Maybe we could create link called
> "python-rpm-macros_SLE12_Backports" that includes a package diff that
> adds the blacklist? Then we could disable python-rpm-macros on
> SLE12_Backports and enable python-rpm-macros_SLE12_Backports only for
> SLE12_Backports.

+1

--

Bruno Friedmann
 Ioda-Net Sàrl www.ioda-net.ch
 Bareos Partner, openSUSE Member, fsfe fellowship
 GPG KEY : D5C9B751C4653227
 irc: tigerfoot


--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: singlespec python and SLE_12_SP2_Backports

Scott Bahling
In reply to this post by todd rme
On Wed, 2017-04-12 at 10:14 -0400, Todd Rme wrote:

> On Wed, Apr 12, 2017 at 8:58 AM, jan matejek <[hidden email]>
> wrote:
> > On 10.4.2017 17:25, Todd Rme wrote:
> > > In devel:languages:python, the new singlespec approach is causing
> > > problems with SLE_12_SP2_Backports build target.  The issue is
> > > that
> > > this build target fails if a package includes files from any
> > > package
> > > in SLE, and packages that have this problem include some really
> > > critical ones like setuptools.
> > >
> > > This is a problem because in most cases only the python2 versions
> > > of
> > > the files are available from SLE, but the build failure means the
> > > python3 versions do not get published.  This results in critical
> > > python3 packages needed for singlespec builds not being available.
> > > Currently more than half of the packages are either disabled or
> > > unresolvable, and that number is going to grow to almost every
> > > package
> > > as the singlespec conversion continues.
> >
> > Does it mean that we are not allowed to use "python2-setuptools"
> > from local repository because
> > "python-setuptools" is part of SLE and should be used in the SLE
> > version?
> > Or does the build of "python2-setuptools" fail because the resulting
> > RPM has file conflicts with a
> > package present in SLE?
>
> The latter. It checks individual files and if those files are already
> in SLE, it will fail.
>
> > In any case, I'm confused as to what is the purpose of the Backports
> > repository. Are people expected
> > to add it straight from OBS and install OBS-built RPMs?
>
> It is described here: https://en.opensuse.org/Portal:Backports
>
> Basically, the idea seems to be to guarantee that no potentially
> incompatible updates of files in SLE get included. So you can be 100%
> sure that these are only add-ons with nothing that could interfere
> with the tested, support SLE packages.
>
> > What if the SLE version is too old, or the d:l:py version is too new?
>
> Then we would probably need to manually disable that package.
>
> > Would only building "python3-foo" help at all?
>
> For most critical packages that should solve the problem.
>
> > > The simplest solution would be to just disable that repo by default.
> > > However, if there (or could be) some way to selectively disable
> > > building python2 versions of specific packages just for that build
> > > target it might be better.
> >
> > We could also create a custom version of python-rpm-macros that contains the "python2 blacklist" as
> > part of itself. As long as it is limited to d:l:py(SLE12_Backports), this might very well be the
> > cleanest solution -- no need to touch individual specs.
>
> That is a good idea. Maybe we could create link called
> "python-rpm-macros_SLE12_Backports" that includes a package diff that
> adds the blacklist? Then we could disable python-rpm-macros on
> SLE12_Backports and enable python-rpm-macros_SLE12_Backports only for
> SLE12_Backports.
What was the outcome of this? Is there something we can do within the
Backports projects to fix the issue? Currently the multi version build
is blocking us from submitting packages that require the python3
packages, but we conflict with the python2 versions in SLE.

-Scott

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: singlespec python and SLE_12_SP2_Backports

Christian
Am 05.07.2017 um 12:31 schrieb Scott Bahling:
> Currently the multi version build
> is blocking us from submitting packages that require the python3
> packages, but we conflict with the python2 versions in SLE.

which also prevent obviously us from having certbot/fail2ban in Backports.
I would really appreciate having this prob fixed and see
certbot/fail2ban ...

Thank you :)

--

Christian
------------------------------------------------------------
   https://join.worldcommunitygrid.org?recruiterId=177038
------------------------------------------------------------
           http://www.sc24.de - Sportbekleidung
------------------------------------------------------------
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: singlespec python and SLE_12_SP2_Backports

Scott Bahling
On Wed, 2017-07-05 at 13:53 +0200, Christian wrote:
> Am 05.07.2017 um 12:31 schrieb Scott Bahling:
> > Currently the multi version build
> > is blocking us from submitting packages that require the python3
> > packages, but we conflict with the python2 versions in SLE.
>
> which also prevent obviously us from having certbot/fail2ban in Backports.
> I would really appreciate having this prob fixed and see
> certbot/fail2ban ...

Yep! I was just looking into fail2ban based on your other request which
triggered my question here.

-Scott

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: singlespec python and SLE_12_SP2_Backports

Scott Bahling
In reply to this post by Christian
On Wed, 2017-07-05 at 13:53 +0200, Christian wrote:
> Am 05.07.2017 um 12:31 schrieb Scott Bahling:
> > Currently the multi version build
> > is blocking us from submitting packages that require the python3
> > packages, but we conflict with the python2 versions in SLE.
>
> which also prevent obviously us from having certbot/fail2ban in Backports.
> I would really appreciate having this prob fixed and see
> certbot/fail2ban ...

While catching up on the singlespec mail threads I guess the answers
lie in the latest RFC that Jan started a couple weeks ago. I will try
to  figure out how this will work with the Backports project since it
has some special conditions.

-Scott

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: singlespec python and SLE_12_SP2_Backports

Scott Bahling
In reply to this post by Scott Bahling
On Fri, 2017-07-07 at 17:03 +0200, jan matejek wrote:

> hello,
>
> On 5.7.2017 12:31, Scott Bahling wrote:
> > What was the outcome of this? Is there something we can do within the
> > Backports projects to fix the issue? Currently the multi version build
> > is blocking us from submitting packages that require the python3
> > packages, but we conflict with the python2 versions in SLE.
>
> i wrote a proof-of-concept patch for python-rpm-macros that allows building only python3 versions of
> blacklisted packages from SLE, without modifying the packages in question. See it at [1] for the
> time being.
Cool! We could add this to the Backports projects. We currently have a
bot that updates the Backports rpmlint blacklist as new packages are
added to SLE 12. We could piggy back off that to update the blacklist in
the python-rpm-macros package.

> However, there is still a problem: there are conflicts with the src.rpm, because, for instance,
> python-six.src.rpm already exists in SLE.
> I don't have a very good idea what to do about this. Is it possible to somehow change name of the
> src.rpm package? If not, you can fork the package under a different name, but it's going to be a
> fork, so at that point you can modify the spec in any case.
> Alternately, could the multibuild magic somehow fix this?

Maybe we can adjust the checks in the Backports projects to ignore src
packages. I need to discuss that idea with others to identify any risks
of doing so. I gut feeling is that we can do it.

-Scott

signature.asc (849 bytes) Download Attachment