Proper way to set shebangs in singlespecs

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

Proper way to set shebangs in singlespecs

Luciano Santos
Hi openSUSE hackers,
 
I have been looking for a way to properly set the right shebang in a python
singlespec package. But without any lucky so far. The naughty package in
question is python-gobject:
 
https://build.opensuse.org/package/show/GNOME:Next/python-gobject

I did some try and error with %ifpython macro, which seems to be the usual
way to make this right, but after toasting some kittens I stopped and
realized this is not working.
 
Have someone cross this path before? If someone have succeeded in such quest
and could shed some light on this one for my curiosity's sake and the
package's health as well, of course, I would be grateful.

Looking forward for your thoughts,
Luciano.
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Proper way to set shebangs in singlespecs

Sebastian-2
For executables: Replace the /usr/bin/env with /usr/bin/python2/3 using
e.g. sed:

sed -i '1s/^#!.*/#!\/usr\/bin\/python3/'

For doumentation/example files: remove the shebangs and the executable bit.

Sebastian


On 12/21/2017 03:47 AM, Luciano Santos wrote:

> Hi openSUSE hackers,
>  
> I have been looking for a way to properly set the right shebang in a python
> singlespec package. But without any lucky so far. The naughty package in
> question is python-gobject:
>  
> https://build.opensuse.org/package/show/GNOME:Next/python-gobject
>
> I did some try and error with %ifpython macro, which seems to be the usual
> way to make this right, but after toasting some kittens I stopped and
> realized this is not working.
>  
> Have someone cross this path before? If someone have succeeded in such quest
> and could shed some light on this one for my curiosity's sake and the
> package's health as well, of course, I would be grateful.
>
> Looking forward for your thoughts,
> Luciano.
--
python programming - mail server - photo - video - https://sebix.at
cryptographic key at https://sebix.at/DC9B463B.asc and on public keyservers



signature.asc (871 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Proper way to set shebangs in singlespecs

Luciano Santos
Sorry, I wasn't clear enough, English is not my first language.

I need a conditional statement that can be satisfied to modify the shebang to use
python2 in the python2-gobject package and python3 in the python3-gobject one.
 
>For documentation/example files: remove the shebangs and the executable bit.

That's my case. It's good to know I only need to do some deletion.

>For executables: Replace the /usr/bin/env with /usr/bin/python2/3 using
>e.g. sed:
>
>sed -i '1s/^#!.*/#!\/usr\/bin\/python3/'

Although, if it wasn't, how would I make a conditional to make both packages,
python2-* and python3-*, use the right shebang as singlespecs use both pythons?
I can't think of any. If someone do, please help us clarify this particular case.
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Proper way to set shebangs in singlespecs

Sebastian-2
On 12/21/2017 10:01 PM, Luciano Santos wrote:
> Sorry, I wasn't clear enough, English is not my first language.
Neiter it is mine :) 
>> For executables: Replace the /usr/bin/env with /usr/bin/python2/3 using
>> e.g. sed:
>>
>> sed -i '1s/^#!.*/#!\/usr\/bin\/python3/'
> Although, if it wasn't, how would I make a conditional to make both packages,
> python2-* and python3-*, use the right shebang as singlespecs use both pythons?
You can only use one shebang at a time, so the files in your
python2-package need to have python2, the ones in the python3-package
the python3 shebang. Or none at all.
> I can't think of any. If someone do, please help us clarify this particular case.
In the case of the linked python-gobject with documentation files: Just
remove them. I create a submit request for that package:
https://build.opensuse.org/request/show/559181

Sebastian

--
python programming - mail server - photo - video - https://sebix.at
cryptographic key at https://sebix.at/DC9B463B.asc and on public keyservers



signature.asc (871 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Proper way to set shebangs in singlespecs

Luciano Santos
>You can only use one shebang at a time, so the files in your
>python2-package need to have python2, the ones in the python3-package
>the python3 shebang. Or none at all.

I appreciate your help Sebastian, really, but I'm afraid you are not
getting me. I know I can only use one shebang at a time and that python2
packages must use python/python2-ending shebangs, python3 ones, python3.

Now, supposing that in this case we hadn't doc/example files in
python-gobject, but script files, what kind of conditionals would we
need to use to make each package use the proper python?

  if python2; then
    #!/usr/bin/python2
  else
    #!/usr/bin/python3
  endif

Constructs like this (I'm using it only as illustration), using rpm macros
are not working, at least I didn't figure it out yet. Using bash, same thing.

Since the example files go both under python2-gobject's doc folder and
python3-gobject's doc folder as well, I have being thinking using something
like this (theoretically speaking):

  sed -i 'python2 shebang on you' /usr/share/doc/packages/python2-gobject/examples/*.py
  sed -i 'python3 shebang on you' /usr/share/doc/packages/python3-gobject/examples/*.py

Again, supposing python-gobject have scrip files rather than doc/examples.
And, of course, using proper syntax and macros. How would we achieve the
proper shebang goal?
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Proper way to set shebangs in singlespecs

Sebastian-2
On 12/22/2017 12:01 AM, Luciano Santos wrote:
> Now, supposing that in this case we hadn't doc/example files in
> python-gobject, but script files, what kind of conditionals would we
> need to use to make each package use the proper python?
It would be easier if you'd say which files we are talking about. For
executables that python's setuptools installs, there is nothing to do.
>   if python2; then
>     #!/usr/bin/python2
>   else
>     #!/usr/bin/python3
>   endif
You can use %ifpython2/%ifpython3, which is true if the package is build
for python2/3.
https://en.opensuse.org/openSUSE:Packaging_Python_Singlespec#Requirements_specific_for_one_python_flavor
> Since the example files go both under python2-gobject's doc folder and
> python3-gobject's doc folder as well, I have being thinking using something
> like this (theoretically speaking):
>
>   sed -i 'python2 shebang on you' /usr/share/doc/packages/python2-gobject/examples/*.py
>   sed -i 'python3 shebang on you' /usr/share/doc/packages/python3-gobject/examples/*.py
You can do that (using %{buildroot} for the paths).

--
python programming - mail server - photo - video - https://sebix.at
cryptographic key at https://sebix.at/DC9B463B.asc and on public keyservers



signature.asc (871 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Proper way to set shebangs in singlespecs

jan matejek-4
hello,

On 22.12.2017 10:36, Sebastian wrote:
> You can use %ifpython2/%ifpython3, which is true if the package is build
> for python2/3.

based on a couple of recent postings, I feel like i need to shout:

WRONG!
WRONG WRONG WRONG!

correct (mostly) is:
%if 0%{?have_python2}


%ifpython2 and %ifpython3 work on:
- Requires
- %files
- %pre/%post

in other words, they determine if the text *is included in section for python2/python3 subpackage*

the build scripts (%prep, %build, %install, %check) are COMMON to all the subpackages. Using
%ifpython there does the wrong thing.

I have added a warning to the wiki page. I also appreciate any suggestions on how to make this
clearer in the docs -- and how to make it fail in code, because people don't read docs :)

The problem is that there isn't actually a macro for "true if package is built for python2/3". This
needs to be fixed. Now you can use "%if 0%{?have_python2}". It's not exactly the right thing but it
works.

> https://en.opensuse.org/openSUSE:Packaging_Python_Singlespec#Requirements_specific_for_one_python_flavor
>> Since the example files go both under python2-gobject's doc folder and
>> python3-gobject's doc folder as well, I have being thinking using something
>> like this (theoretically speaking):
>>
>>   sed -i 'python2 shebang on you' /usr/share/doc/packages/python2-gobject/examples/*.py
>>   sed -i 'python3 shebang on you' /usr/share/doc/packages/python3-gobject/examples/*.py
> You can do that (using %{buildroot} for the paths).

%python_expand sed -i '$python for you' %{buildroot}%{_datadir}/doc/packages/$python-gobject .....

>



signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Proper way to set shebangs in singlespecs

Sebastian-2
On 12/22/2017 04:23 PM, jan matejek wrote:
> I also appreciate any suggestions on how to make this
> clearer in the docs
I have one: add the have_python macros to the docs.

Sebastian

--
python programming - mail server - photo - video - https://sebix.at
cryptographic key at https://sebix.at/DC9B463B.asc and on public keyservers



signature.asc (871 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Proper way to set shebangs in singlespecs

jan matejek-5
In reply to this post by jan matejek-4
hello,

On 23.12.2017 00:45,  Luciano Santos  wrote:
> I read the python singlespec on the wiki but some parts felt kind of obscure to me. If it had either
> all the macros for singlespec purposes, or at least all the most useful ones with their use cases
> description I could've made the pain go away with my own meds.

there's a full list on the github page, which is linked from the wiki:
https://github.com/openSUSE/python-rpm-macros
i'll make the link more prominent as soon as the wiki unlocks

>> The problem is that there isn't actually a macro for "true if package is built for python2/3". This
>> needs to be fixed. Now you can use "%if 0%{?have_python2}". It's not exactly the right thing but it
>> works.
>>
>> %python_expand sed -i '$python for you' %{buildroot}%{_datadir}/doc/packages/$python-gobject .....
>
> Now, that's such an interesting piece of information. The have_python macro and this use of
> %python_expand with some other information on the wiki were the missing puzzle pieces that I needed.
> Thank you Jan and Sebastian for your time.
>


signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Proper way to set shebangs in singlespecs

Christian Boltz-5
Hello,

Am Donnerstag, 28. Dezember 2017, 17:28:51 CET schrieb jan matejek:
> i'll make the link more prominent as soon as the wiki unlocks

Oops ;-)

Fixed since two hours, after Andreas told me about the issue on IRC.

For your entertainment:
I blocked a spammer earlier today, and didn't uncheck the "block the IP"
checkbox. It turned out that I still had the Provo proxy IPs whitelisted
in the wiki config instead of the Nuremberg login proxy. Therefore
blocking the "spammer's IP" blocked the login proxy, which made the
english wiki read-only for everybody. So nothing really surprising, the
only surprising thing is that this went unnoticed for so long.

Needless to say that I updated the IP whitelist in the config file ;-)


Regards,

Christian Boltz
--
One piece of advice: if you maintain a C&C server (which is both a
really bad idea and a criminal act and as such, strongly discouraged),
always use a strong password.
It's very unprofessional if your server is cracked this easily.
[http://www.sophos.com/en-us/medialibrary/PDFs/technical%20papers/
SophosInsideABlackHole.pdf]



--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Proper way to set shebangs in singlespecs

Luciano Santos
In reply to this post by jan matejek-5
> Sent: Thursday, December 28, 2017 at 2:28 PM
> From: "jan matejek" <[hidden email]
>
> there's a full list on the github page, which is linked from the wiki:
> https://github.com/openSUSE/python-rpm-macros
> i'll make the link more prominent as soon as the wiki unlocks

Holy shell! I missed that one. I did some fragmented readings and on the last
one GitHub's reference went totally unnoticed. But the python-rpm-macros surely
gave me some insights. I will add in my to-do's list helping to improve the
python singlespec when I feel comfortable enough to.

If someone is interested, I solved my 'theoretical' problem of giving the right
python version in the after-shebang header with the following measures:

  * First of all I installed manually the scripts which I wanted to edit
    the python binary path;
  * Then, under %install section I used `find` to look for the *.py scripts
    which would be edited by `sed`;
  * And, finally, under %files section I used %ifpythonX to package the
    scripts.

In my case the overall additions looked like this:

  %install
  ...
  %python_expand mkdir -p %{buildroot}%{_docdir}/$python-gobject
  %python_expand cp -R examples/* %{buildroot}%{_docdir}/$python-gobject
  %python_expand find %{buildroot}%{_docdir}/$python-gobject -type f -exec\
  sed -i 's|%{_bindir}/env python|%{_bindir}/$python|' {} ";"

  %files
  %ifpython2
  %{_docdir}/python2-gobject
  %endif
  %ifpython3
  %{_docdir}/python3-gobject
  %endif

Sounds like a lot of trouble only to give the path to the right python binary,
but it worked, again, for my 'theoretical' problem as those scripts were only
examples. And, of course, there is at least another alternative that was pointed
to me which relies on using `sed 's|#!/usr/bin/env python|#!/usr/bin/python|'`
and letting the user choose whichever python he wants to with `update-alternatives`.

My solution maybe can be polished a bit, I don't know. But it can be useful in
some specific cases I believe. Any further insights on this matter is welcomed!


Wishing the best for everyone in this ending year,
Luciano.
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Proper way to set shebangs in singlespecs

jan matejek-4
hello,

On 29.12.2017 02:57, Luciano Santos wrote:
> In my case the overall additions looked like this:
>
>   %install
>   ...
>   %python_expand mkdir -p %{buildroot}%{_docdir}/$python-gobject
>   %python_expand cp -R examples/* %{buildroot}%{_docdir}/$python-gobject
>   %python_expand find %{buildroot}%{_docdir}/$python-gobject -type f -exec\

If you care about Leap 42.3 and older, the name should be "%{$python_flavor}-gobject".
That means that you get "python-gobject" on 42.3 and "python2-gobject" on newer.

If you _don't_ care about 42.3, what you did is perfectly fine.

Also, %python_expand can do multiline blocks and this is appropriate here.
--8<--
%{python_expand # a comment here to prevent RPM barf
mkdir -p ...
cp -R ...
find -exec sed ....
}
-->8--

>   %files
>   %ifpython2
>   %{_docdir}/python2-gobject
>   %endif
>   %ifpython3
>   %{_docdir}/python3-gobject
>   %endif

Two things.
One, %python2_only and %python3_only are your friends. What you wrote is equivalent to:
--8<--
%files
%python2_only %{_docdir}/python2-gobject
%python3_only %{_docdir}/python3-gobject
-->8--

Two, even this shorter version is not great. You're listing every flavor by hand. Singlespec is all
about not doing that ;) This also fails when we add a new flavor, say, pypy3.

The right way to do this is actually very simple:
--8<--
%files
%{_docdir}/%{python_flavor}-gobject
-->8--

also you probably want to mark it as doc:
%doc %{_docdir}/...



Speaking of which, I haven't tried, but something like this might work:
--8<--
%install
(...)
%{python_expand # convert examples
mkdir doc-%{$python_flavor}
find examples -type f -exec | while read pathname; do
        sed 's/.../.../' $pathname > doc-%{$python_flavor}/$pathname
done
}

%files
%doc doc-%{python_flavor}/examples
-->8--
This relies on the %doc macro to do the installation in the properly named directory. You would
prepare a doc-$flavor directory for each flavor. Then using the %doc with a relative path, RPM will
take the argument and install it into %{_docdir}/$packagename, without the need to figure out the
$packagename.

regards
m.

>
> Sounds like a lot of trouble only to give the path to the right python binary,
> but it worked, again, for my 'theoretical' problem as those scripts were only
> examples. And, of course, there is at least another alternative that was pointed
> to me which relies on using `sed 's|#!/usr/bin/env python|#!/usr/bin/python|'`
> and letting the user choose whichever python he wants to with `update-alternatives`.
>
> My solution maybe can be polished a bit, I don't know. But it can be useful in
> some specific cases I believe. Any further insights on this matter is welcomed!



signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Proper way to set shebangs in singlespecs

Luciano Santos
> Sent: Tuesday, January 02, 2018 at 2:44 PM
> From: "jan matejek" <[hidden email]>
>
> >   %files
> >   %ifpython2
> >   %{_docdir}/python2-gobject
> >   %endif
> >   %ifpython3
> >   %{_docdir}/python3-gobject
> >   %endif
>
> Two things.
> One, %python2_only and %python3_only are your friends. What you wrote is equivalent to:
> --8<--
> %files
> %python2_only %{_docdir}/python2-gobject
> %python3_only %{_docdir}/python3-gobject
> -->8--
>
> Two, even this shorter version is not great. You're listing every flavor by hand. Singlespec is all
> about not doing that ;) This also fails when we add a new flavor, say, pypy3.
>
> The right way to do this is actually very simple:
> --8<--
> %files
> %{_docdir}/%{python_flavor}-gobject
> -->8--
>
> also you probably want to mark it as doc:
> %doc %{_docdir}/...

The cake's cherry. I knew I was missing something, the way I went didn't
seem the right one.

> Speaking of which, I haven't tried, but something like this might work:
> --8<--
> %install
> (...)
> %{python_expand # convert examples
> mkdir doc-%{$python_flavor}
> find examples -type f -exec | while read pathname; do
> sed 's/.../.../' $pathname > doc-%{$python_flavor}/$pathname
> done
> }
>
> %files
> %doc doc-%{python_flavor}/examples
> -->8--
> This relies on the %doc macro to do the installation in the properly named directory. You would
> prepare a doc-$flavor directory for each flavor. Then using the %doc with a relative path, RPM will
> take the argument and install it into %{_docdir}/$packagename, without the need to figure out the
> $packagename.

I'm actually very satisfied right now with all this feedback and even
without this last experimental chunk, but it worth a try for future
references. Thank you Jan Matejek.

Wishing a good year for everybody,
Luciano.
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]