LLVM package changes

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

LLVM package changes

Michal Srb
Hi,

Some time ago I had to make some changes to the LLVM package. It caused some
issues and took some time to stabilize, but I thought it is done and working
correctly now. Unfortunatelly, it turned out it is still broken and more
changes are needed. Before I start breaking things again, I would like to
describe here what happened and what I think still needs to happen.

The LLVM package builds several related projects: llvm, clang, libcxx, lldb,
lld, and other small ones. These projects can be either placed into a single
source tree and built all together, or they can be built individually. From
packaging point of view, it would be cleaner if we built them separately, but
since we need to bootstrap clang and then use it to build itself, we have to
build at least llvm, clang and libcxx together. So it is easier to just build
them all together.

Each of the projects is divided into components (such as LLVM/Analysis, LLVM/
AsmParser, clang/AST, clang/CodeGen, ...).

LLVM can be built in different modes:

1) Every component will compile into a shared library.
2) Every component will compile into a static library.
3) Every project will compile into a single shared library containing all its
   components.

Before I started working on the package, we were building LLVM using mode 1.
This caused us some troubles, all that I am aware of are listed in this bug:
https://bugzilla.opensuse.org/show_bug.cgi?id=1049703

When trying to fix these issues with upstream, we were told that this build
mode is only meant for the llvm developers and the issues would not happen if
we used one of the recommended modes (2 or 3).

https://github.com/klee/klee/pull/686
> Maybe openSUSE have a good reason for doing this but going against a
> project's recommended practice seems like a bad idea.

> You can file a bug on LLVM.org to investigate fixing that, however it is
> unlikely that much attention will be given to the BUILD_SHARED_LIBS build
> because it is strongly discouraged.

The same information is also stated in the LLVM build documentation. So I went
ahead and changed LLVM package to the mode 3 (single shared library for every
project). Unfortunatelly the clang project does not support this mode. So the
clang components are instead built as static libraries and distributed in the
new clang-devel-static package.

This is the current state of the package. One drawback is that the llvm
package now takes longer to build and consumes more resources in OBS. (Linking
stuff together into big libraries takes longer and the static libraries cause
increase file size. Biggest slow down is in extracting debuginfo and
compressing the RPM files.) That caused the LLVM package to be a bottleneck in
the distribution build and needed to be solved (bnc#1064786). I spent lot of
time trying to optimize the build and also had to split Mesa to a llvm and
non-llvm part (bnc#1071297) to reduce the amount of packages that transitively
depend on llvm. The situation is now better, although still not that good.

Recently a new bug got to me:
https://bugzilla.opensuse.org/show_bug.cgi?id=1065464

That bug shows that OpenCL is broken if you have multiple OpenCL
implementations which use clang installed at once. Note that it is perfectly
ok to have them installed at once. Applications are supposed to load them all,
query them and pick one to use. However, when a program loads two libraries
statically linked with the clang libraries, it aborts. The reason is that each
of them try to register a configuration option into the llvm library - but
there is only one shared llvm library used by both of them. It aborts during
the second attempt to register the same configuration option. There is a
request upstream to change the way llvm handles configuration options, but it
is a big task and not likely to be done any time soon.

So it can only work if both clang and llvm are static or both shared
libraries. I am against making them all static. In addition to all the good
reasons why we try to stay away from static libraries, it would likely make
the llvm build times even worse.

Instead, I would like to try approach similar to what Fedora does - they build
all of llvm using the mode 3 (each project as single shared library), except
for clang which they build with mode 1 ("developer" mode where every component
ends up as a shared library). Fedora has easier time doing that because they
have them in separate packages, but we can do it too using a small patch to
the llvm build system (tested).

This should:
* Fix the OpenCL bug.
* Get rid of static libraries and the clang-devel-static package.
* Reduce llvm build time.
* Maybe reintroduce some bugs since we will be partially using the unsupported
  "developer" mode again. But it would be only for clang, llvm and all the
  other components would still use the supported mode. AFAIK all the bugs I
  know of were caused by the llvm libraries. I will test for regressions
  before submitting anything.

Comments and suggestions are welcome.

Michal Srb
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: LLVM package changes

Aaron Puchert
Hi,

I like the idea, it seems like the right way to go. If it works for
Fedora, it's probably good enough.

I didn't encounter any of the mentioned bugs, but there are a few things
that I don't like about the current setup:

* The package clang-devel contains headers for both C API (in
/usr/include/clang-c) and C++ API (in /usr/include/clang), but
libclang.so from the same package only exposes symbols for the former.
* Using the C++ API, I have to install clang-devel-static, which is a
gigantic package, and linking against it is quite slow.
* Unlike libclang.so, libLLVM.so exposes both C and C++ API, which seems
inconsistent. So using both with the C++ API I link statically with one
and dynamically with another?

So technically Clang supports the single dynamic library (3), but that
library only exposes the C API. Which makes me think that the advice
against using BUILD_SHARED_LIBS only applies to LLVM and not Clang,
since otherwise there would be no shared library build exposing Clang's
C++ API. That advice is also from the LLVM documentation, and I couldn't
find anything similar for Clang.

If all else fails, and we really have to go the static library route, we
should at least have another version of clang-devel-static without debug
info. (Which should then be around 40-50 MB instead of 4 GB.)

Best regards,
Aaron

Am 19.01.2018 um 10:17 schrieb Michal Srb:

> Hi,
>
> Some time ago I had to make some changes to the LLVM package. It caused some
> issues and took some time to stabilize, but I thought it is done and working
> correctly now. Unfortunatelly, it turned out it is still broken and more
> changes are needed. Before I start breaking things again, I would like to
> describe here what happened and what I think still needs to happen.
>
> The LLVM package builds several related projects: llvm, clang, libcxx, lldb,
> lld, and other small ones. These projects can be either placed into a single
> source tree and built all together, or they can be built individually. From
> packaging point of view, it would be cleaner if we built them separately, but
> since we need to bootstrap clang and then use it to build itself, we have to
> build at least llvm, clang and libcxx together. So it is easier to just build
> them all together.
>
> Each of the projects is divided into components (such as LLVM/Analysis, LLVM/
> AsmParser, clang/AST, clang/CodeGen, ...).
>
> LLVM can be built in different modes:
>
> 1) Every component will compile into a shared library.
> 2) Every component will compile into a static library.
> 3) Every project will compile into a single shared library containing all its
>    components.
>
> Before I started working on the package, we were building LLVM using mode 1.
> This caused us some troubles, all that I am aware of are listed in this bug:
> https://bugzilla.opensuse.org/show_bug.cgi?id=1049703
>
> When trying to fix these issues with upstream, we were told that this build
> mode is only meant for the llvm developers and the issues would not happen if
> we used one of the recommended modes (2 or 3).
>
> https://github.com/klee/klee/pull/686
>> Maybe openSUSE have a good reason for doing this but going against a
>> project's recommended practice seems like a bad idea.
>
>> You can file a bug on LLVM.org to investigate fixing that, however it is
>> unlikely that much attention will be given to the BUILD_SHARED_LIBS build
>> because it is strongly discouraged.
>
> The same information is also stated in the LLVM build documentation. So I went
> ahead and changed LLVM package to the mode 3 (single shared library for every
> project). Unfortunatelly the clang project does not support this mode. So the
> clang components are instead built as static libraries and distributed in the
> new clang-devel-static package.
>
> This is the current state of the package. One drawback is that the llvm
> package now takes longer to build and consumes more resources in OBS. (Linking
> stuff together into big libraries takes longer and the static libraries cause
> increase file size. Biggest slow down is in extracting debuginfo and
> compressing the RPM files.) That caused the LLVM package to be a bottleneck in
> the distribution build and needed to be solved (bnc#1064786). I spent lot of
> time trying to optimize the build and also had to split Mesa to a llvm and
> non-llvm part (bnc#1071297) to reduce the amount of packages that transitively
> depend on llvm. The situation is now better, although still not that good.
>
> Recently a new bug got to me:
> https://bugzilla.opensuse.org/show_bug.cgi?id=1065464
>
> That bug shows that OpenCL is broken if you have multiple OpenCL
> implementations which use clang installed at once. Note that it is perfectly
> ok to have them installed at once. Applications are supposed to load them all,
> query them and pick one to use. However, when a program loads two libraries
> statically linked with the clang libraries, it aborts. The reason is that each
> of them try to register a configuration option into the llvm library - but
> there is only one shared llvm library used by both of them. It aborts during
> the second attempt to register the same configuration option. There is a
> request upstream to change the way llvm handles configuration options, but it
> is a big task and not likely to be done any time soon.
>
> So it can only work if both clang and llvm are static or both shared
> libraries. I am against making them all static. In addition to all the good
> reasons why we try to stay away from static libraries, it would likely make
> the llvm build times even worse.
>
> Instead, I would like to try approach similar to what Fedora does - they build
> all of llvm using the mode 3 (each project as single shared library), except
> for clang which they build with mode 1 ("developer" mode where every component
> ends up as a shared library). Fedora has easier time doing that because they
> have them in separate packages, but we can do it too using a small patch to
> the llvm build system (tested).
>
> This should:
> * Fix the OpenCL bug.
> * Get rid of static libraries and the clang-devel-static package.
> * Reduce llvm build time.
> * Maybe reintroduce some bugs since we will be partially using the unsupported
>   "developer" mode again. But it would be only for clang, llvm and all the
>   other components would still use the supported mode. AFAIK all the bugs I
>   know of were caused by the llvm libraries. I will test for regressions
>   before submitting anything.
>
> Comments and suggestions are welcome.
>
> Michal Srb
>

--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: LLVM package changes

Michal Srb
On středa 24. ledna 2018 1:47:47 CET Aaron Puchert wrote:
> Hi,
>
> I like the idea, it seems like the right way to go. If it works for
> Fedora, it's probably good enough.

Hi,

Thank you for the feedback!

> * The package clang-devel contains headers for both C API (in
> /usr/include/clang-c) and C++ API (in /usr/include/clang), but
> libclang.so from the same package only exposes symbols for the former.
> * Using the C++ API, I have to install clang-devel-static, which is a
> gigantic package, and linking against it is quite slow.

Luckily this problem will go away with the new changes because we'll get rid
of the clang-devel-static package and have again only clang-devel.

> * Unlike libclang.so, libLLVM.so exposes both C and C++ API, which seems
> inconsistent. So using both with the C++ API I link statically with one
> and dynamically with another?

Yeah, in the current setup that is the way to do it. And it can cause bugs
like the bnc#1065464 if you get mixed with another library that also linked
statically to clang.

> So technically Clang supports the single dynamic library (3), but that
> library only exposes the C API.

As I understand it, the libclang.so is different than libLLVM.so. As you
observed, it exposes only the stable C API and so it is linked (statically or
dynamically) only to *some* of the clang components. So the remaining
components still have to be distributed.

If you build LLVM as a single dynamic library, the libLLVM.so contains and
exposes all the available LLVM components. I do not know of a way to build
libclang.so in such way.

> Which makes me think that the advice
> against using BUILD_SHARED_LIBS only applies to LLVM and not Clang,
> since otherwise there would be no shared library build exposing Clang's
> C++ API. That advice is also from the LLVM documentation, and I couldn't
> find anything similar for Clang.

Hopefully that is true.

Michal
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]