python single-spec, december edition

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

python single-spec, december edition

jan matejek-4
Hello packagers,

this is a progress report on the singlespec project. I also have some
things to discuss. These are way down in the e-mail, under a heading. If
you don't want to read all of it, please at least scroll down to the
"Remaining issues" section.

1. Changes from last time
-------------------------

In the last month, I've made a number of significant improvements to the
macro set.

1.1 Backwards compatibility

This one is the biggest: we can now build new-style packages in old
distributions without any prjconf changes.

The trick consists of two parts:
- placing the macro definitions in a separate python-rpm-macros package,
which is then buildrequired. You can simply link the package to your
project, or we might push it into SLE/Leap build base.
- shimming the %python_module definition in the spec file. Since the
main purpose of this macro is to avoid repetitive listing of
buildrequires, we can conditionally define it to expand to "python-$1
python3-$1" -- or, if needed, just "python-$1" -- and this will do the
necessary work.

The d:l:py:singlespec repository now builds packages for all distros
from 13.2 up.
In principle, nothing is stopping us from supporting SLE11 too, but with
all the other compatibility requirements, I consider it not worth the
effort.

This also means that you can now easily build singlespec packages in
your own repository. Just link python-rpm-macros from d:l:py:Factory.

1.2 Macro naming

Naming is now very much unified. Every macro name is prefixed with the
"flavor" name, which is the name of the binary in /usr/bin. That means
no short versions: %python2_version, %python3_version. If we ever add,
say, Jython, it's going to get %jython3_version.

Some short versions are retained as compatibility spellings, namely
"%py_ver". I have, however, dropped the old macro set "%py_sitedir,
%py_incdir" etc. I don't think they are used anywhere; if they are, we
will find out soon enough.

1.3 Fedora compatibility

I have based some macro definitions on Fedora [1]. I claim _partial_
compatibility: we don't have macros for wheels and eggs. They look
simple enough that I'll probably add them.

I also didn't copy SRPM macros, because I have no idea what they are
good for. If some Fedora people want me to include them, please explain :)

As stated above, the primary spelling of all macros is with long names:
%py2_compile in SUSE will be an alias for %python2_compile. The latter
is the preferred spelling.

[1] http://pkgs.fedoraproject.org/cgit/rpms/python-rpm-macros.git/tree/

1.4 Package naming

You can name your package anything (usual scenario is "python-foo"). In
older distros, the result will be python-foo, python3-foo (and possibly
pypy3-foo).

In newer distros (as soon as I submit the new Python 2 package that
Provides "python2-*" symbols), the result will be python2-foo,
python3-foo and an empty package python-foo.

I don't think I can get rid of the empty package, but the python2-foo
packages are created with obsoletes/provides on the name.

In order to accomplish this trick, if your package is called
"python-foo", you have to decorate all %files sections: "%files
%python_files", or "%files %{python_files subpkg}". The build will fail
if it detects an undecorated %files section.
This is not necessary if your top-level package is called "python2-foo",
"python3-foo" or anything else.

1.5 %pycache_only

In addition to %ifpython2, %ifpython3 etc, I have added a %ifpycache
section and a corresponding %pycache_only marker. This is equivalent to
"if not python2", and is applicable for flavors that use the
"__pycache__" directory to store .pyc files.

1.6 %python_expand

This macro will repeat its arguments for every flavor, replacing
"$python" with the flavor name.

For example:
%{python_expand cp command-%$python_bin_suffix
%{buildroot}%{$python_sitelib}/foomodule/command}

I'm still not sure if this is a good idea -- after all, it's introducing
a minilanguage specific to Python packages.
But it sure is useful.


2. Remaining issues
-------------------

New-style packages, "python2-foo", rely on presence of prjconf-based
definition of %python_modules. Before we can submit packages to Factory,
we need to get the definitions there. This should pose no problem, however.

In two packages now I have run into a situation where I want to create a
subpackage that is _not_ automatically replicated. In the first case,
python-pycparser has some data that can be shared between the versions;
in the second, for python-Sphinx, i want to create a subpackage for the
binaries that has a dependency only on the Python 3 version.
Both cases can be solved in a different way, however, the need for
non-replicated looks like an important feature.
How to implement it?
One idea is to not replicate any %package -n something. Subpackages of
this type are currently buggy, changing the bugginess to straight out
ignoring is easy enough :) And conversely, it is unclear how to
replicate packages of this kind anyway, so declaring them to be
non-replicating is sort of natural.
Another option is to delineate packages by a macro, say,
%python_noreplicate.
What do you think?

Requires, Suggests and Recommends are copied from the main package
declaration and changed to match the python flavor. It is somewhat
unclear what to do about Provides and Obsoletes.
They should probably be handled similarly, except the autoconverter
should wrap them in %python2_only by default.

On a related note, the %ifpython2 and %python2_only macros were
originally intended for use both in Requires and %files. However, with
the new style packages, python-foo.spec generating python2-foo.rpm and
empty python-foo.rpm, this will have some flaws.
Namely the requires/provides will be the same in the empty python-foo as
in the correct python2-foo. I foresee this causing problems: "have
choice for python-provided-symbol: python-foo python2-foo".
There is a number of solutions:

1. I can hack %python2_only to work properly, and invent an alternate
spelling for %ifpython2. Maybe %ifpython2_requires? I can then detect
whether the correct spelling is used and halt the build if it is not.
I don't like this too much, for obvious reasons :)

2. I can detect said macros and halt build without alternative, forcing
the packager to rename the whole package to python2-foo or python3-foo,
which would sidestep the problem.

3. I can simply not care and let the empty package have the same
obsoletes/provides. This would be great if we could cripple the empty
package in some other way. Introduce an unsatisfiable requirement?
"Requires: this-does-not-exist" or "Conflicts: aaa_base"? Would this
help zypper resolve the choice?
Is there a different option for saying "this package should always be
preferred" from within the package? "Prefer: python2-foo" but within the
spec?
Unsatisfiable requirement for the empty package would be a good idea in
any case, to make sure the empty packages don't sneak into the
distribution and on people's computers.

As usual, i'll be glad for all your input, answer questions, etc. etc.

regards
m.


signature.asc (817 bytes) Download Attachment