call for volunteers: singlespec auto-conversion

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

call for volunteers: singlespec auto-conversion

jan matejek-4
Hello all,

I have finally done what I promised several months ago and ran an auto-converter over all python-*
packages in Factory. The results can be seen in:
https://build.opensuse.org/project/show/devel:languages:python:singlespec-staging

It is roughly 540 packages, so a small fraction of d:l:py :)
About a third of them build.
Another third fails.
The rest is stuck in unresolvable state, usually because they depend on something from the failed set.

Now is the time to review the packages and decide what to do with them. And I'd like to ask you for
help with this gargantuan task.

I'm taking time off tomorrow, so I won't be handling any SRs until Monday. But I wanted to show you
this anyway, in case you felt like playing with it over the weekend. Also, I think d:l:py
maintainers have rights to the repository?

(I honestly don't know how to organize stuff like this, so for now, I'm leaving it as-is and seeing
what happens. I'll do more on Monday. Maybe some sort of a hackathon? Bruno Friedman (in cc) was
talking about this...)

Anyway.
What needs to be done:

* Some packages simply aren't compatible with python3. These should either be upgraded to a
compatible version, dropped from Factory altogether, or left alone with an added Provides for
"python2-foo".

* (This also holds for various backports and things like configparser, unittest2 etc. We generally
don't want them in singlespec versions if they aren't necessary for Python 3.4 and above.)

* In others, the autoconverter might have messed up. Maybe the spec file needs a human touch to
bring it to a working state.

* Even packages that build successfully must be reviewed to check that post-conversion spec file is
doing the same thing as the pre-conversion one.

* Some packages got there by mistake (such as -doc specs of packages that are already singlespec)
and can be deleted from the singlespec-staging project

* This is also a good opportunity to drop obsolete stuff. For instance, Factory still contains
python-argparse, which has not been necessary since openSUSE 12.1 (because it's in stdlib of all
shipped pythons). Worse, packages still list is as a requirement. This needs to go.

* When a package is reviewed, it should be submitted to its devel project (and deleted from
singlespec-staging when the request is accepted) and then forwarded to Factory. Successfully
singlespec'd packages also need a corresponding delete request for the python3 counterpart.

All in all, thanks in advance to every volunteer. Every single SR helps :)

regards
m.


signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: call for volunteers: singlespec auto-conversion

Sebastian-2
Hi,

A new line has been added to the end of the changelog files, is this by
intention?

On 08/24/2017 06:15 PM, jan matejek wrote:
> * When a package is reviewed, it should be submitted to its devel project (and deleted from
> singlespec-staging when the request is accepted)
Deleting needs an extra request, with "Remove local package if request
is accepted" I get a permission error.

Sebastian

--
python programming - mail server - photo - video - https://sebix.at
cryptographic key at https://sebix.at/DC9B463B.asc and on public keyservers



signature.asc (871 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: call for volunteers: singlespec auto-conversion

Luigi Baldoni
In reply to this post by jan matejek-4
jan matejek-4 wrote
I have finally done what I promised several months ago and ran an auto-converter over all python-*
packages in Factory. The results can be seen in:
https://build.opensuse.org/project/show/devel:languages:python:singlespec-staging
Händel's Hallelujah chorus is playing in my mind:)

jan matejek-4 wrote
* This is also a good opportunity to drop obsolete stuff. For instance, Factory still contains
python-argparse, which has not been necessary since openSUSE 12.1 (because it's in stdlib of all
shipped pythons). Worse, packages still list is as a requirement. This needs to go.
I think I had stumbled upon that a while ago, but thing is the packages themselves often rely on the
separate argparse module and therefore require patching.

Regards
Reply | Threaded
Open this post in threaded view
|

Re: call for volunteers: singlespec auto-conversion

jan matejek-4
In reply to this post by Sebastian-2
hello,

On 25.8.2017 10:15, Sebastian wrote:
> Hi,
>
> A new line has been added to the end of the changelog files, is this by
> intention?

It's a minor bug of the changelog processing library I'm using, didn't seem worth fixing.
If you are touching the changelogs by hand, feel free to remove the extra line (or leave it in, I
don't care either way and I expect that Factory maintainers don't either.)

> On 08/24/2017 06:15 PM, jan matejek wrote:
>> * When a package is reviewed, it should be submitted to its devel project (and deleted from
>> singlespec-staging when the request is accepted)
> Deleting needs an extra request, with "Remove local package if request
> is accepted" I get a permission error.

Right, thanks for noticing.
I think you need to be project admin for this to work. I have added you now, but others will need to
look out for this.

regards
m.

>
> Sebastian
>



signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: call for volunteers: singlespec auto-conversion

jan matejek-4
In reply to this post by Luigi Baldoni
On 26.8.2017 20:16, Luigi Baldoni wrote:
>> Factory still contains
>> python-argparse, which has not been necessary since openSUSE 12.1 (because
>> it's in stdlib of all
>> shipped pythons). Worse, packages still list is as a requirement. This
>> needs to go.
>
> I think I had stumbled upon that a while ago, but thing is the packages
> themselves often rely on the
> separate argparse module and therefore require patching.

I don't think so. python-argparse installs %{sitelib}/argparse.py, so "import argparse" is the same
for the bundled version as for the separate version.
(let me know if you come across a package where argparse must be patched)

What you say is true for (some) backports modules though. The stand-alone ones are imported under
different names.

regards
m.


signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: call for volunteers: singlespec auto-conversion

Luigi Baldoni
jan matejek-4 wrote
On 26.8.2017 20:16, Luigi Baldoni wrote:
>> Factory still contains
>> python-argparse, which has not been necessary since openSUSE 12.1 (because
>> it's in stdlib of all
>> shipped pythons). Worse, packages still list is as a requirement. This
>> needs to go.
>
> I think I had stumbled upon that a while ago, but thing is the packages
> themselves often rely on the
> separate argparse module and therefore require patching.

I don't think so. python-argparse installs %{sitelib}/argparse.py, so "import argparse" is the same
for the bundled version as for the separate version.
(let me know if you come across a package where argparse must be patched)
Would that satisfy requirements.txt ?

jan matejek-4 wrote
What you say is true for (some) backports modules though. The stand-alone ones are imported under
different names.
It was some time ago, perhaps you're right.

Regards
Reply | Threaded
Open this post in threaded view
|

Re: call for volunteers: singlespec auto-conversion

jan matejek-4
On 28.8.2017 16:44, Luigi Baldoni wrote:

> jan matejek-4 wrote
>> On 26.8.2017 20:16, Luigi Baldoni wrote:
>>>> Factory still contains
>>>> python-argparse, which has not been necessary since openSUSE 12.1
>>>> (because
>>>> it's in stdlib of all
>>>> shipped pythons). Worse, packages still list is as a requirement. This
>>>> needs to go.
>>>
>>> I think I had stumbled upon that a while ago, but thing is the packages
>>> themselves often rely on the
>>> separate argparse module and therefore require patching.
>>
>> I don't think so. python-argparse installs %{sitelib}/argparse.py, so
>> "import argparse" is the same
>> for the bundled version as for the separate version.
>> (let me know if you come across a package where argparse must be patched)
>
> Would that satisfy requirements.txt ?
short answer: Yes.

long answer: That's not the right question. There are two questions here:

- Does pkg_info recognize python's stdlib argparse?
No.
So if a package explicitly asks for "argparse" through something like pkg_info, it will fail.

- Do we care? Will it break our builds / packages?
Again, no :)
requirements.txt by themselves are irrelevant, what matters is setup.py. Explicit requirement for
argparse in setup.py would fail because of pkg_info above. (Also if setup.py reads requirements txt,
dependencies expressed in egg-info and similar, etc.)
BUT, packages recent enough to know about python 2.7 will usually have argparse guarded by
"if python < 2.7" -- which can be (and usually is) expressed even in requirements.txt

Packages *not* recent enough to know about 2.7 probably don't work anymore anyway and should be
dropped from the distro.
Packages that are alive and explicitly require argparse should be patched and patches sent upstream,
because they're doing it wrong.

- Will it break someone's scripts?
Probably not.
I mean, every change breaks someone's workflow. But here you'd need someone who is at the same time
a) advanced enough to use argparse before it was part of stdlib, b) precise enough to express this
in setup.py or ask for it in pkg_info, yet c) sloppy enough to ignore python 2.7. That's an unlikely
combination.
(The requirement could have been carried over from a legacy deployment, but again, the whole thing
would probably be long broken by now.)

regards
m.

>
>
> jan matejek-4 wrote
>> What you say is true for (some) backports modules though. The stand-alone
>> ones are imported under
>> different names.
>
> It was some time ago, perhaps you're right.
>
> Regards
>
>
>
>
> --
> View this message in context: http://opensuse.14.x6.nabble.com/call-for-volunteers-singlespec-auto-conversion-tp5094191p5094381.html
> Sent from the opensuse-packaging mailing list archive at Nabble.com.
>


signature.asc (849 bytes) Download Attachment