Firefox build fails on i586 Tumbleweed

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

Firefox build fails on i586 Tumbleweed

Tom M.
Just noticed that MozillaFirefox is not in the i586
openSUSE_Tumbleweed repo anymore:
https://download.opensuse.org/tumbleweed/repo/oss/suse/i586/

Digging a little deeper it seems that the build is failing for quite
some time due to out of memory:
https://build.opensuse.org/build/openSUSE:Factory/standard/i586/MozillaFirefox/_log

Could somebody look into that?

Tom
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Firefox build fails on i586 Tumbleweed

Adrian Schröter
On Sonntag, 12. November 2017, 17:16:56 CET wrote Tom M.:
> Just noticed that MozillaFirefox is not in the i586
> openSUSE_Tumbleweed repo anymore:
> https://download.opensuse.org/tumbleweed/repo/oss/suse/i586/
>
> Digging a little deeper it seems that the build is failing for quite
> some time due to out of memory:
> https://build.opensuse.org/build/openSUSE:Factory/standard/i586/MozillaFirefox/_log
>
> Could somebody look into that?

Out of memory error ... so it seems the package needs to increase it's need
for memory in the _constraints file.

However, you need to report this to the maintainer or to the factory people
in first place. Or even create a submit request for that ;)

You can find the right development project and maintainer in the owner
search:

  https://build.opensuse.org/search/owner?utf8=%E2%9C%93&search_text=MozillaFirefox&owner=1&package=0&project=0&commit=Senden&limit=1

--

Adrian Schroeter
email: [hidden email]

SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)
 
Maxfeldstraße 5                        
90409 Nürnberg
Germany


--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Firefox build fails on i586 Tumbleweed

Wolfgang Rosenauer-4
Hi,

Am 13.11.2017 um 08:51 schrieb Adrian Schröter:

> On Sonntag, 12. November 2017, 17:16:56 CET wrote Tom M.:
>> Just noticed that MozillaFirefox is not in the i586
>> openSUSE_Tumbleweed repo anymore:
>> https://download.opensuse.org/tumbleweed/repo/oss/suse/i586/
>>
>> Digging a little deeper it seems that the build is failing for quite
>> some time due to out of memory:
>> https://build.opensuse.org/build/openSUSE:Factory/standard/i586/MozillaFirefox/_log
>>
>> Could somebody look into that?
>
> Out of memory error ... so it seems the package needs to increase it's need
> for memory in the _constraints file.

Firefox already has 6GB defined for memory. Typically in my experience
changing that higher does not even fix the issue.
We already have a patch in Firefox which tries to reduce linking
executions to use less memory which still seems not to be small enough
for 32bit systems.

In addition the upcoming Firefox 57 will not build on x86-32 for other
reasons. But I will certainly look into it. It'll just not be a quick thing.


Wolfgang
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Firefox build fails on i586 Tumbleweed

Tom M.
It seems to build with 8 jobs in parallel. How about hardcoding
MOZ_MAKE_FLAGS=-j2 or similar?


Tom


2017-11-13 9:12 GMT+01:00 Wolfgang Rosenauer <[hidden email]>:

> Hi,
>
> Am 13.11.2017 um 08:51 schrieb Adrian Schröter:
>> On Sonntag, 12. November 2017, 17:16:56 CET wrote Tom M.:
>>> Just noticed that MozillaFirefox is not in the i586
>>> openSUSE_Tumbleweed repo anymore:
>>> https://download.opensuse.org/tumbleweed/repo/oss/suse/i586/
>>>
>>> Digging a little deeper it seems that the build is failing for quite
>>> some time due to out of memory:
>>> https://build.opensuse.org/build/openSUSE:Factory/standard/i586/MozillaFirefox/_log
>>>
>>> Could somebody look into that?
>>
>> Out of memory error ... so it seems the package needs to increase it's need
>> for memory in the _constraints file.
>
> Firefox already has 6GB defined for memory. Typically in my experience
> changing that higher does not even fix the issue.
> We already have a patch in Firefox which tries to reduce linking
> executions to use less memory which still seems not to be small enough
> for 32bit systems.
>
> In addition the upcoming Firefox 57 will not build on x86-32 for other
> reasons. But I will certainly look into it. It'll just not be a quick thing.
>
>
> Wolfgang
> --
> To unsubscribe, e-mail: [hidden email]
> To contact the owner, e-mail: [hidden email]
>
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Firefox build fails on i586 Tumbleweed

Stefan Bruens
In reply to this post by Wolfgang Rosenauer-4
On Montag, 13. November 2017 09:12:17 CET Wolfgang Rosenauer wrote:

> Hi,
>
> Am 13.11.2017 um 08:51 schrieb Adrian Schröter:
> > On Sonntag, 12. November 2017, 17:16:56 CET wrote Tom M.:
> >> Just noticed that MozillaFirefox is not in the i586
> >> openSUSE_Tumbleweed repo anymore:
> >> https://download.opensuse.org/tumbleweed/repo/oss/suse/i586/
> >>
> >> Digging a little deeper it seems that the build is failing for quite
> >> some time due to out of memory:
> >> https://build.opensuse.org/build/openSUSE:Factory/standard/i586/MozillaFi
> >> refox/_log
> >>
> >> Could somebody look into that?
> >
> > Out of memory error ... so it seems the package needs to increase it's
> > need
> > for memory in the _constraints file.
>
> Firefox already has 6GB defined for memory. Typically in my experience
> changing that higher does not even fix the issue.
> We already have a patch in Firefox which tries to reduce linking
> executions to use less memory which still seems not to be small enough
> for 32bit systems.

All 32bit workers are only able to use 3GB of "physical" memory, plus the
typical 1-2GByte of swap.

Currently, kernel-obs-build packages the "default" flavour, which does not
enable pae/lpae (ix86/armv7). Non-PAE kernels are only able to use 3GByte, no
matter how much memory you pass to the VM via "-m".

To verify, dump the contents of /proc/meminfo (MemTotal) or grep for "Memory"
in the kernel log/dmesg.

Kind regards,

Stefan
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Firefox build fails on i586 Tumbleweed

Jan Engelhardt-4

On Monday 2017-11-13 16:39, Brüns, Stefan wrote:
>
>All 32bit workers are only able to use 3GB of "physical" memory, plus the
>typical 1-2GByte of swap.

That is wrong - some 32-bit workers can use 4G, notably those with a
64-bit kernel and those which have the special 4:4 split feature enabled
in their i386 kernel.
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Firefox build fails on i586 Tumbleweed

Stefan Bruens
On Montag, 13. November 2017 16:48:05 CET Jan Engelhardt wrote:
> On Monday 2017-11-13 16:39, Brüns, Stefan wrote:
> >All 32bit workers are only able to use 3GB of "physical" memory, plus the
> >typical 1-2GByte of swap.
>
> That is wrong - some 32-bit workers can use 4G, notably those with a
> 64-bit kernel and those which have the special 4:4 split feature enabled
> in their i386 kernel.

See https://build.opensuse.org/package/live_build_log/
home:StefanBruens:CheckMemory/check-memory/openSUSE_Tumbleweed/i586

_constraints set to 6G, VM is started with -m 8192, available memory (Mem +
Swap) in the build environment: 3G + 2G

-> _constraints is *not* honored

Regards,

Stefan
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Firefox build fails on i586 Tumbleweed

Adrian Schröter
On Montag, 13. November 2017, 17:57:21 CET wrote Brüns, Stefan:

> On Montag, 13. November 2017 16:48:05 CET Jan Engelhardt wrote:
> > On Monday 2017-11-13 16:39, Brüns, Stefan wrote:
> > >All 32bit workers are only able to use 3GB of "physical" memory, plus the
> > >typical 1-2GByte of swap.
> >
> > That is wrong - some 32-bit workers can use 4G, notably those with a
> > 64-bit kernel and those which have the special 4:4 split feature enabled
> > in their i386 kernel.
>
> See https://build.opensuse.org/package/live_build_log/
> home:StefanBruens:CheckMemory/check-memory/openSUSE_Tumbleweed/i586
>
> _constraints set to 6G, VM is started with -m 8192, available memory (Mem +
> Swap) in the build environment: 3G + 2G
>
> -> _constraints is *not* honored

Huh? It is, the VM is providing 8GB what is more than 6GB.

It is not up to the constraints system what the content (here the kernel)
is doing with that...

--

Adrian Schroeter
email: [hidden email]

SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)
 
Maxfeldstraße 5                        
90409 Nürnberg
Germany


--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Firefox build fails on i586 Tumbleweed

Stefan Bruens
On Dienstag, 14. November 2017 07:33:22 CET you wrote:

> On Montag, 13. November 2017, 17:57:21 CET wrote Brüns, Stefan:
> > On Montag, 13. November 2017 16:48:05 CET Jan Engelhardt wrote:
> > > On Monday 2017-11-13 16:39, Brüns, Stefan wrote:
> > > >All 32bit workers are only able to use 3GB of "physical" memory, plus
> > > >the
> > > >typical 1-2GByte of swap.
> > >
> > > That is wrong - some 32-bit workers can use 4G, notably those with a
> > > 64-bit kernel and those which have the special 4:4 split feature enabled
> > > in their i386 kernel.
> >
> > See https://build.opensuse.org/package/live_build_log/
> > home:StefanBruens:CheckMemory/check-memory/openSUSE_Tumbleweed/i586
> >
> > _constraints set to 6G, VM is started with -m 8192, available memory (Mem
> > +
> > Swap) in the build environment: 3G + 2G
> >
> > -> _constraints is *not* honored
>
> Huh? It is, the VM is providing 8GB what is more than 6GB.
>
> It is not up to the constraints system what the content (here the kernel)
> is doing with that...

From the user perspective this is clearly unexpected behaviour - the user has
requested a worker with 8 GB of memory, but only gets 4...5 GByte (RAM + Swap,
as documented).

A worker which is started with more than 3 GByte (-m >= 3000) should be using
a PAE/HIGHMEM64G kernel.

Currently, the constraints have the opposite effect to the users expectations
- a job is more likely to fail on a 8GB/-smp 8 system than on a 4GB/-smp 4
system.

Kind regards,

Stefan
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Firefox build fails on i586 Tumbleweed

Jan Engelhardt-4

On Tuesday 2017-11-14 12:04, Brüns, Stefan wrote:

>> >
>> > _constraints set to 6G, VM is started with -m 8192, available memory (Mem
>> > + Swap) in the build environment: 3G + 2G
>> >
>> > -> _constraints is *not* honored
>>
>> Huh? It is, the VM is providing 8GB what is more than 6GB.
>>
>> It is not up to the constraints system what the content (here the kernel)
>> is doing with that...
>
>From the user perspective this is clearly unexpected behaviour - the user has
>requested a worker with 8 GB of memory, but only gets 4...5 GByte (RAM + Swap,
>as documented).
>
>A worker which is started with more than 3 GByte (-m >= 3000) should be using
>a PAE/HIGHMEM64G kernel.

I can confirm; kvm is started with 64-bit "hardware", but uses the non-PAE
32-bit kernel (kernel-default).

It either has to use kernel-pae, or has to use the 64-bit kernel-default.
init=/.build/build will switch to the ILP32 personality anyway.


I have a feeling though, that such won't be enough to get those unspeakably fat
browsers to link - it may become necessary to use a 64-bit toolchain with -m32
to complete it, and then the baselibs mechanism to export the build results.
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Firefox build fails on i586 Tumbleweed

Andreas Schwab-2
On Nov 14 2017, Jan Engelhardt <[hidden email]> wrote:

> I have a feeling though, that such won't be enough to get those unspeakably fat
> browsers to link - it may become necessary to use a 64-bit toolchain with -m32
> to complete it, and then the baselibs mechanism to export the build results.

Getting all build dependencies ready for -m32 would be a challenge,
though.

Andreas.

--
Andreas Schwab, SUSE Labs, [hidden email]
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Firefox build fails on i586 Tumbleweed

Stefan Bruens
In reply to this post by Jan Engelhardt-4
On Dienstag, 14. November 2017 16:36:12 CET you wrote:

> On Tuesday 2017-11-14 12:04, Brüns, Stefan wrote:
> >> > _constraints set to 6G, VM is started with -m 8192, available memory
> >> > (Mem
> >> > + Swap) in the build environment: 3G + 2G
> >> >
> >> > -> _constraints is *not* honored
> >>
> >> Huh? It is, the VM is providing 8GB what is more than 6GB.
> >>
> >> It is not up to the constraints system what the content (here the kernel)
> >> is doing with that...
> >
> >From the user perspective this is clearly unexpected behaviour - the user
> >has requested a worker with 8 GB of memory, but only gets 4...5 GByte (RAM
> >+ Swap, as documented).
> >
> >A worker which is started with more than 3 GByte (-m >= 3000) should be
> >using a PAE/HIGHMEM64G kernel.
>
> I can confirm; kvm is started with 64-bit "hardware", but uses the non-PAE
> 32-bit kernel (kernel-default).
>
> It either has to use kernel-pae, or has to use the 64-bit kernel-default.
> init=/.build/build will switch to the ILP32 personality anyway.
>
>
> I have a feeling though, that such won't be enough to get those unspeakably
> fat browsers to link - it may become necessary to use a 64-bit toolchain
> with -m32 to complete it, and then the baselibs mechanism to export the
> build results.

You may be right and it will fail later anyway, but currently, the failing
processes are at about 1.5 GByte each (from the log in the first message):

[ 2881s] virtual memory exhausted: Cannot allocate memory
[ 2881s] virtual memory exhausted: Cannot allocate memory
[ 2881s]
[ 2881s] cc1plus: out of memory allocating 335128 bytes after a total of
13721600 bytes
[ 2881s]
[ 2881s] /usr/lib/gcc/i586-suse-linux/7/../../../../i586-suse-linux/bin/as:
out of memory allocating 4064 bytes after a total of 16891904 bytes
[ 2881s] {standard input}: Assembler messages:
[ 2881s] {standard input}:291371: Fatal error: can't close
Unified_cpp_layout_generic3.o: Memory exhausted
[ 2881s]
[ 2881s] /usr/lib/gcc/i586-suse-linux/7/../../../../i586-suse-linux/bin/as:
out of memory allocating 4064 bytes after a total of 13647872 bytes

Kind regards,

Stefan
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Firefox build fails on i586 Tumbleweed

Jan Engelhardt-4
In reply to this post by Andreas Schwab-2

On Tuesday 2017-11-14 16:47, Andreas Schwab wrote:
>On Nov 14 2017, Jan Engelhardt <[hidden email]> wrote:
>
>> I have a feeling though, that such won't be enough to get those unspeakably fat
>> browsers to link - it may become necessary to use a 64-bit toolchain with -m32
>> to complete it, and then the baselibs mechanism to export the build results.
>
>Getting all build dependencies ready for -m32 would be a challenge,
>though.

It could yes. The alternative would be to run a 32-bit userspace with a
64-bit linker program, requiring only a few -64bit baselibs.
If we ever do x32, modifying mkbaselibs will be a necessity anyway.
Might as well do it now :-)
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Firefox build fails on i586 Tumbleweed

Stefan Bruens
On Dienstag, 14. November 2017 17:08:18 CET Jan Engelhardt wrote:

> On Tuesday 2017-11-14 16:47, Andreas Schwab wrote:
> >On Nov 14 2017, Jan Engelhardt <[hidden email]> wrote:
> >> I have a feeling though, that such won't be enough to get those
> >> unspeakably fat browsers to link - it may become necessary to use a
> >> 64-bit toolchain with -m32 to complete it, and then the baselibs
> >> mechanism to export the build results.>
> >Getting all build dependencies ready for -m32 would be a challenge,
> >though.
>
> It could yes. The alternative would be to run a 32-bit userspace with a
> 64-bit linker program, requiring only a few -64bit baselibs.
> If we ever do x32, modifying mkbaselibs will be a necessity anyway.
> Might as well do it now :-)

For the time being, can we just package kernel-(l)pae as kernel-obs-build on
32 bit archs, i.e. i586/armv7l?

A PAE-enabled kernel has a slight overhead, but this is likely more than
compensated by hitting the guests swap later.

This does not change the 3GByte per process limit, which may cause failing
linker runs. Using a 64bit linker is much more effort, so I would vote for a
PAE kernel first and *if* we still see OOM errors, reconsider if we should go
the 64bit userspace route (i586 is quite dead, but armv7l may stay for a bit
longer).

CC'ed opensuse-kernel, as I already sent a patch addressing kernel-obs-build a
week ago.

Kind regards,

Stefan
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]