RFC: Proposed relocation of /var/lib/rpm

classic Classic list List threaded Threaded
54 messages Options
123
Reply | Threaded
Open this post in threaded view
|

RFC: Proposed relocation of /var/lib/rpm

Richard Brown
Hi all,

Currently, rpm stores its rpmdb (the record of all a systems packages)
in /var/lib/rpm

Various openSUSE & SUSE distributions have a number of problems with this

This has a major impact on our default btrfs snapshot & rollback feature.
We need the contents of the rpmdb contained within the snapshots.

This means /var/lib/rpm must be contained in the root snapshot in
order to preserve the "system-state";
Without it a rolled-back system still thinks it has packages
installed/removed which were changed as a result of the rollback.

But /var and /var/lib both contain a great deal of data which we DO
NOT want to include in the root snapshot;
For example, we most certainly don't want snapper rolling back a
database in /var/lib/mysql for example, and destroying user data in
the process.

Therefore we currently carry a rather complicated default list of subvolumes:
https://en.opensuse.org/SDB:BTRFS#Default_Subvolumes

This list contains a number of specific directories in /var/* we wish
to preserve, which means it's almost continually out of date, leading
to user data being at risk if we haven't made a specific subvolume for
that data.

The rpmdb is the only "system-state" information we have in /var which
we need to preserve, making it quite a disruptive presence in the /var
filesystem hierarchy.

The problem is further complicated in *SUSE distributions that use a
Read-Only root filesystem (eg. openSUSE Kubic & SUSE CaaS Platform)
because it results in a situation where we really need to micromanage
countless subvolumes under /var - any location we miss otherwise ends
up read-only and packages can't write to the files they expect to be
able to in /var

Red Hat had a similar problem with their rpm-ostree tooling, which
they solved by relocating the RPM database to /usr/share/rpm

https://rpm-ostree.readthedocs.io/en/latest/#why-not-implement-these-changes-in-an-existing-package-manager

I would like to propose we adopt the same location for rpmdb in all
*SUSE distributions

This would mean we could make a single /var subvolume and dramatically
simplify our btrfs configuration

For systems with a read-only rootfs, /usr/share/rpm would be immutable
and pristine, just as we require it to be. Our tooling should already
be able to handle the new location without any major issues.

For systems with a read-write rootfs, we will be slightly bending the
rules of the FHS, in the sense that the FHS claims /usr should be
'read-only data'

http://www.pathname.com/fhs/pub/fhs-2.3.html#THEUSRHIERARCHY

However, installing packages routinely modifies the contents of /usr
and it's various subdirectories, that is the nature of installing
packages.

Therefore it seems logical to me that the record of what is installed
is also stored under the same conditions.

I'm working on a patchset in an OBS home project to test this proposal
and shake loose any issues, and will also be proposing this location
change to rpm upstream.

Therefore any comments are not only welcome, but the quicker the
better, so I can factor them into what I'm doing.

I would also like suggestions as how best to handle the migration of
an existing rpmdb in /var/lib/rpm to it's new location in
/usr/share/rpm, as I'm certain this is something we want to get
absolutely perfect, and I'm understandably nervous about changing the
rpmdb's location during an rpm transaction which will be altering the
rpmdb.

Thanks in advance,
--
Richard Brown
Linux Distribution Engineer - Future Technology Team

Phone +4991174053-361
SUSE Linux GmbH,  Maxfeldstr. 5,  D-90409 Nuernberg
GF: Felix Imendörffer, Jane Smithard, Graham Norton,
HRB 21284 (AG Nürnberg)
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: RFC: Proposed relocation of /var/lib/rpm

Carlos E. R.-2
On 2017-10-04 13:16, Richard Brown wrote:
> For systems with a read-write rootfs, we will be slightly bending the
> rules of the FHS, in the sense that the FHS claims /usr should be
> 'read-only data'

What about machines that share /usr from a central machine over the network?

--
Cheers / Saludos,

                Carlos E. R.
                (from 42.2 x86_64 "Malachite" at Telcontar)


signature.asc (188 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: RFC: Proposed relocation of /var/lib/rpm

Richard Brown
On 4 October 2017 at 13:38, Carlos E. R. <[hidden email]> wrote:
> On 2017-10-04 13:16, Richard Brown wrote:
>> For systems with a read-write rootfs, we will be slightly bending the
>> rules of the FHS, in the sense that the FHS claims /usr should be
>> 'read-only data'
>
> What about machines that share /usr from a central machine over the network?

Putting the rpmdb in /usr/share is no worse than the current situation

Right now machines sharing /usr from a central machine over the
network are going to have an rpmdb that will contain an invalid
picture of the contents of /usr

Any person with root access to any machine accessing that shared /usr
can attempt to install/remove packages. If the share is read-write
that will currently lead to changes for all systems but only the
system doing the installation will be aware of what it did and able to
uninstall the rpms that changed /usr

This will lead to untrackable inconsistencies in other areas of the
filesystem, as only /usr is shared.

If the share is read-only the rpm install will likely fail.

With this proposed change any user with root access to a machine
accessing a shared /usr will change the contents of /usr and update
the (now shared) rpmdb

While this can still lead to inconsistencies on other systems as only
/usr will be shared, this will at least be 'trackable' as all systems
with a shared /usr will now be able to verify what files they should
have, where as currently they cannot.

If the share is read-only, the rpm install will still fail, just as it
would today.
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: RFC: Proposed relocation of /var/lib/rpm

Marcus Rueckert-3
In reply to this post by Richard Brown

tbh ... /etc/rpm/db would be a path i could live with ... /usr/share/rpm
feels wrong to me. this is a path where i would expect static data.

   darix

--
           openSUSE - SUSE Linux is my linux
               openSUSE is good for you
                   www.opensuse.org
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: RFC: Proposed relocation of /var/lib/rpm

Andreas Schwab-2
In reply to this post by Richard Brown
On Okt 04 2017, Richard Brown <[hidden email]> wrote:

> Red Hat had a similar problem with their rpm-ostree tooling, which
> they solved by relocating the RPM database to /usr/share/rpm

Why did they choose /usr/share/rpm instead of somewhere below /usr/lib?

Andreas.

--
Andreas Schwab, SUSE Labs, [hidden email]
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: RFC: Proposed relocation of /var/lib/rpm

Josef Reidinger-3
In reply to this post by Richard Brown
Hi,

sorry for top posting, but it is general feedback not specific to any part of of email and pasting it in bottom is easy to overlook.

1. /usr/share/ sounds bad for me, as it can be often read-only FS ( e.g. on my system it is ).

2. this snapshot and rollback make assumption that data in /var/lib is version agnostic. Can you please specify on which data you make such assumption? I think this is simply not true and just because we do not run it long enough we do not face problems. But e.g. consider that mysql as part of update migrate /var/lib/mysql to more efficient binary format and then you do rollback. Bum! And e.g. with pgsql it is even more true, as it contain also part of configuration. On my system is 62 entries in /var/lib and I am not sure if every survive bigger downgrade of software that is using it.

According to this I suggest to instead move version agnostic data that should not be snapshotted to own directory for which it is clear its purpose and have such guarantie.

Josef


On Wed, 4 Oct 2017 13:16:58 +0200
Richard Brown <[hidden email]> wrote:

> Hi all,
>
> Currently, rpm stores its rpmdb (the record of all a systems packages)
> in /var/lib/rpm
>
> Various openSUSE & SUSE distributions have a number of problems with this
>
> This has a major impact on our default btrfs snapshot & rollback feature.
> We need the contents of the rpmdb contained within the snapshots.
>
> This means /var/lib/rpm must be contained in the root snapshot in
> order to preserve the "system-state";
> Without it a rolled-back system still thinks it has packages
> installed/removed which were changed as a result of the rollback.
>
> But /var and /var/lib both contain a great deal of data which we DO
> NOT want to include in the root snapshot;
> For example, we most certainly don't want snapper rolling back a
> database in /var/lib/mysql for example, and destroying user data in
> the process.
>
> Therefore we currently carry a rather complicated default list of subvolumes:
> https://en.opensuse.org/SDB:BTRFS#Default_Subvolumes
>
> This list contains a number of specific directories in /var/* we wish
> to preserve, which means it's almost continually out of date, leading
> to user data being at risk if we haven't made a specific subvolume for
> that data.
>
> The rpmdb is the only "system-state" information we have in /var which
> we need to preserve, making it quite a disruptive presence in the /var
> filesystem hierarchy.
>
> The problem is further complicated in *SUSE distributions that use a
> Read-Only root filesystem (eg. openSUSE Kubic & SUSE CaaS Platform)
> because it results in a situation where we really need to micromanage
> countless subvolumes under /var - any location we miss otherwise ends
> up read-only and packages can't write to the files they expect to be
> able to in /var
>
> Red Hat had a similar problem with their rpm-ostree tooling, which
> they solved by relocating the RPM database to /usr/share/rpm
>
> https://rpm-ostree.readthedocs.io/en/latest/#why-not-implement-these-changes-in-an-existing-package-manager
>
> I would like to propose we adopt the same location for rpmdb in all
> *SUSE distributions
>
> This would mean we could make a single /var subvolume and dramatically
> simplify our btrfs configuration
>
> For systems with a read-only rootfs, /usr/share/rpm would be immutable
> and pristine, just as we require it to be. Our tooling should already
> be able to handle the new location without any major issues.
>
> For systems with a read-write rootfs, we will be slightly bending the
> rules of the FHS, in the sense that the FHS claims /usr should be
> 'read-only data'
>
> http://www.pathname.com/fhs/pub/fhs-2.3.html#THEUSRHIERARCHY
>
> However, installing packages routinely modifies the contents of /usr
> and it's various subdirectories, that is the nature of installing
> packages.
>
> Therefore it seems logical to me that the record of what is installed
> is also stored under the same conditions.
>
> I'm working on a patchset in an OBS home project to test this proposal
> and shake loose any issues, and will also be proposing this location
> change to rpm upstream.
>
> Therefore any comments are not only welcome, but the quicker the
> better, so I can factor them into what I'm doing.
>
> I would also like suggestions as how best to handle the migration of
> an existing rpmdb in /var/lib/rpm to it's new location in
> /usr/share/rpm, as I'm certain this is something we want to get
> absolutely perfect, and I'm understandably nervous about changing the
> rpmdb's location during an rpm transaction which will be altering the
> rpmdb.
>
> Thanks in advance,

--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: RFC: Proposed relocation of /var/lib/rpm

Bruno Friedmann-2
In reply to this post by Marcus Rueckert-3
On mercredi, 4 octobre 2017 14.37:01 h CEST Marcus Rueckert wrote:
> tbh ... /etc/rpm/db would be a path i could live with ... /usr/share/rpm
> feels wrong to me. this is a path where i would expect static data.
>
>    darix

I feel the same, /etc being part of the rootfs and rpmdb being the
"expression" of the perticular state of installed software of this computer, I
found it more logical to find /etc/rpm/db

But as I'm not that familiar with btrfs trick and so, I can have a biased
view.

btw in the wiki of subvolumes I guess /var/lib/dhcp should also be subvol
due to the dhcpd db lease (which should be consitent with named one if dynamic
dns is in used) otherwise TXT records will not be in sync.


--

Bruno Friedmann
 Ioda-Net Sàrl www.ioda-net.ch
 Bareos Partner, openSUSE Member, fsfe fellowship
 GPG KEY : D5C9B751C4653227
 irc: tigerfoot


--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: RFC: Proposed relocation of /var/lib/rpm

Dominique Leuenberger / DimStar
In reply to this post by Marcus Rueckert-3
On Wed, 2017-10-04 at 14:37 +0200, Marcus Rueckert wrote:
> tbh ... /etc/rpm/db would be a path i could live with ...
> /usr/share/rpm
> feels wrong to me. this is a path where i would expect static data.

/etc feels wrong in different ways: the rpm db is not any file we
expect an admin to go fiddle and change content of it

Most correct would likely be /usr/lib/something - but what is the
benefit over /usr/share (except improved feeling) compared to being
able to share patchwork with another distro?

Cheers
Dominique

signature.asc (201 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: RFC: Proposed relocation of /var/lib/rpm

Frederic Crozat-4
In reply to this post by Marcus Rueckert-3
Le mercredi 04 octobre 2017 à 14:37 +0200, Marcus Rueckert a écrit :
> tbh ... /etc/rpm/db would be a path i could live with ...
> /usr/share/rpm
> feels wrong to me. this is a path where i would expect static data.

OTOH, most of the files installed in /usr/share/ are handled by rpm,
which aren't that static, as soon as you update a package.

Moreover, we already have some generated db in /usr (desktop
applications and mime database ).

--
Frederic Crozat
Enterprise Desktop Release Manager
SUSE

--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: RFC: Proposed relocation of /var/lib/rpm

Marcus Rueckert-3
In reply to this post by Dominique Leuenberger / DimStar
On 2017-10-04 12:59:34 +0000, Dominique Leuenberger / DimStar wrote:

> On Wed, 2017-10-04 at 14:37 +0200, Marcus Rueckert wrote:
> > tbh ... /etc/rpm/db would be a path i could live with ...
> > /usr/share/rpm
> > feels wrong to me. this is a path where i would expect static data.
>
> /etc feels wrong in different ways: the rpm db is not any file we
> expect an admin to go fiddle and change content of it
>
> Most correct would likely be /usr/lib/something - but what is the
> benefit over /usr/share (except improved feeling) compared to being
> able to share patchwork with another distro?

/usr is more or less static for me. no modified data in it. /etc/ is
configuration at least. and the admin *does* fiddle with the files in
/etc/rpm/db/ ... via the rpm utility. not all files in /etc/ are being
edited with $EDITOR. Just to name one example: tzselect.

which also bring the question ... does someone have a reference to the
fedora/redhat discussion about this subject? it might short cut the
whole discussion.

    darix

--
           openSUSE - SUSE Linux is my linux
               openSUSE is good for you
                   www.opensuse.org
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: RFC: Proposed relocation of /var/lib/rpm

Thorsten Kukuk
On Wed, Oct 04, Marcus Rueckert wrote:

> On 2017-10-04 12:59:34 +0000, Dominique Leuenberger / DimStar wrote:
> > On Wed, 2017-10-04 at 14:37 +0200, Marcus Rueckert wrote:
> > > tbh ... /etc/rpm/db would be a path i could live with ...
> > > /usr/share/rpm
> > > feels wrong to me. this is a path where i would expect static data.
> >
> > /etc feels wrong in different ways: the rpm db is not any file we
> > expect an admin to go fiddle and change content of it
> >
> > Most correct would likely be /usr/lib/something - but what is the
> > benefit over /usr/share (except improved feeling) compared to being
> > able to share patchwork with another distro?
>
> /usr is more or less static for me. no modified data in it. /etc/ is
> configuration at least.

The RPM database is as static as any other file in /usr. It is only
modified, if you modify files in /usr. It is not modified, if you
modify files in /etc.
So according to your own argumentation, the rpmdb should be in /usr.

Beside that moving the rpm database to /etc does not solve any
problem, but the result would be even worse than with /var/lib.

And if somebody finally finishes the /bin -> /usr move, having
the RPMDB outside of /usr doesn't make any sense anymore.

> which also bring the question ... does someone have a reference to the
> fedora/redhat discussion about this subject? it might short cut the
> whole discussion.

I don't think there was any dicussion, Red Hat decided to do so for
RPM-Ostree, since they had no choice.

  Thorsten
--
Thorsten Kukuk, Distinguished Engineer, Senior Architect SLES & CaaSP
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany
GF: Felix Imendoerffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nuernberg)
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: RFC: Proposed relocation of /var/lib/rpm

Thorsten Kukuk
In reply to this post by Josef Reidinger-3
On Wed, Oct 04, Josef Reidinger wrote:

> Hi,
>
> sorry for top posting, but it is general feedback not specific to any part of of email and pasting it in bottom is easy to overlook.
>
> 1. /usr/share/ sounds bad for me, as it can be often read-only FS ( e.g. on my system it is ).

Which means you have an problem if you update man-pages anyways.

> 2. this snapshot and rollback make assumption that data in /var/lib is version agnostic. Can you please specify on which data you make such assumption? I think this is simply not true and just because we do not run it long enough we do not face problems. But e.g. consider that mysql as part of update migrate /var/lib/mysql to more efficient binary format and then you do rollback. Bum! And e.g. with pgsql it is even more true, as it contain also part of configuration. On my system is 62 entries in /var/lib and I am not sure if every survive bigger downgrade of software that is using it.

Correct, most software is not able to handle such a version downgrade
itself.
But only think about a customer, who runs a mariadb based web shop and is
doing a rollback. Yes, the old mariadb cannot access the data, but if you
would rollback the database, the last orders would be lost.
So the whole idea behind the design is, to never loose important customer
data during a rollback.

  Thorsten

> According to this I suggest to instead move version agnostic data that should not be snapshotted to own directory for which it is clear its purpose and have such guarantie.
>
> Josef
>
>
> On Wed, 4 Oct 2017 13:16:58 +0200
> Richard Brown <[hidden email]> wrote:
>
> > Hi all,
> >
> > Currently, rpm stores its rpmdb (the record of all a systems packages)
> > in /var/lib/rpm
> >
> > Various openSUSE & SUSE distributions have a number of problems with this
> >
> > This has a major impact on our default btrfs snapshot & rollback feature.
> > We need the contents of the rpmdb contained within the snapshots.
> >
> > This means /var/lib/rpm must be contained in the root snapshot in
> > order to preserve the "system-state";
> > Without it a rolled-back system still thinks it has packages
> > installed/removed which were changed as a result of the rollback.
> >
> > But /var and /var/lib both contain a great deal of data which we DO
> > NOT want to include in the root snapshot;
> > For example, we most certainly don't want snapper rolling back a
> > database in /var/lib/mysql for example, and destroying user data in
> > the process.
> >
> > Therefore we currently carry a rather complicated default list of subvolumes:
> > https://en.opensuse.org/SDB:BTRFS#Default_Subvolumes
> >
> > This list contains a number of specific directories in /var/* we wish
> > to preserve, which means it's almost continually out of date, leading
> > to user data being at risk if we haven't made a specific subvolume for
> > that data.
> >
> > The rpmdb is the only "system-state" information we have in /var which
> > we need to preserve, making it quite a disruptive presence in the /var
> > filesystem hierarchy.
> >
> > The problem is further complicated in *SUSE distributions that use a
> > Read-Only root filesystem (eg. openSUSE Kubic & SUSE CaaS Platform)
> > because it results in a situation where we really need to micromanage
> > countless subvolumes under /var - any location we miss otherwise ends
> > up read-only and packages can't write to the files they expect to be
> > able to in /var
> >
> > Red Hat had a similar problem with their rpm-ostree tooling, which
> > they solved by relocating the RPM database to /usr/share/rpm
> >
> > https://rpm-ostree.readthedocs.io/en/latest/#why-not-implement-these-changes-in-an-existing-package-manager
> >
> > I would like to propose we adopt the same location for rpmdb in all
> > *SUSE distributions
> >
> > This would mean we could make a single /var subvolume and dramatically
> > simplify our btrfs configuration
> >
> > For systems with a read-only rootfs, /usr/share/rpm would be immutable
> > and pristine, just as we require it to be. Our tooling should already
> > be able to handle the new location without any major issues.
> >
> > For systems with a read-write rootfs, we will be slightly bending the
> > rules of the FHS, in the sense that the FHS claims /usr should be
> > 'read-only data'
> >
> > http://www.pathname.com/fhs/pub/fhs-2.3.html#THEUSRHIERARCHY
> >
> > However, installing packages routinely modifies the contents of /usr
> > and it's various subdirectories, that is the nature of installing
> > packages.
> >
> > Therefore it seems logical to me that the record of what is installed
> > is also stored under the same conditions.
> >
> > I'm working on a patchset in an OBS home project to test this proposal
> > and shake loose any issues, and will also be proposing this location
> > change to rpm upstream.
> >
> > Therefore any comments are not only welcome, but the quicker the
> > better, so I can factor them into what I'm doing.
> >
> > I would also like suggestions as how best to handle the migration of
> > an existing rpmdb in /var/lib/rpm to it's new location in
> > /usr/share/rpm, as I'm certain this is something we want to get
> > absolutely perfect, and I'm understandably nervous about changing the
> > rpmdb's location during an rpm transaction which will be altering the
> > rpmdb.
> >
> > Thanks in advance,
>
> --
> To unsubscribe, e-mail: [hidden email]
> To contact the owner, e-mail: [hidden email]

--
Thorsten Kukuk, Distinguished Engineer, Senior Architect SLES & CaaSP
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany
GF: Felix Imendoerffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nuernberg)
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: RFC: Proposed relocation of /var/lib/rpm

Daniel Morris
In reply to this post by Andreas Schwab-2
On Wed, Oct 04, 2017 at 02:51:01PM +0200, Andreas Schwab wrote:
> On Okt 04 2017, Richard Brown <[hidden email]> wrote:
>
> > Red Hat had a similar problem with their rpm-ostree tooling, which
> > they solved by relocating the RPM database to /usr/share/rpm
>
> Why did they choose /usr/share/rpm instead of somewhere below /usr/lib?
>

And will it make it into FHS 3.1, or has everyone given up on the notion
of the LSB?

http://refspecs.linuxfoundation.org/FHS_3.0/fhs-3.0.pdf

 Daniel
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: RFC: Proposed relocation of /var/lib/rpm

Thorsten Kukuk
On Wed, Oct 04, Daniel Morris wrote:

> On Wed, Oct 04, 2017 at 02:51:01PM +0200, Andreas Schwab wrote:
> > On Okt 04 2017, Richard Brown <[hidden email]> wrote:
> >
> > > Red Hat had a similar problem with their rpm-ostree tooling, which
> > > they solved by relocating the RPM database to /usr/share/rpm
> >
> > Why did they choose /usr/share/rpm instead of somewhere below /usr/lib?
> >
>
> And will it make it into FHS 3.1, or has everyone given up on the notion
> of the LSB?

LSB is dead. There will be no new LSB version anymore. For a very long
time, LSB was even completly removed from the linuxfoundation web sites.
I doubt that there will ever be a new FHS version, too.

Beside that /usr/share/rpm is ok according to FHS 3.0, no change of FHS
is needed.

  Thorsten

--
Thorsten Kukuk, Distinguished Engineer, Senior Architect SLES & CaaSP
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany
GF: Felix Imendoerffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nuernberg)
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: RFC: Proposed relocation of /var/lib/rpm

Jan Engelhardt-4
In reply to this post by Thorsten Kukuk
On Wednesday 2017-10-04 15:17, Thorsten Kukuk wrote:

>> > > tbh ... /etc/rpm/db would be a path i could live with ...
>> > > /usr/share/rpm
>> > > feels wrong to me. this is a path where i would expect static data.
>> >
>> > /etc feels wrong in different ways: the rpm db is not any file we
>> > expect an admin to go fiddle and change content of it
>> >
>> > Most correct would likely be /usr/lib/something - but what is the
>> > benefit over /usr/share (except improved feeling) compared to being
>> > able to share patchwork with another distro?
>>
>> /usr is more or less static for me. no modified data in it. /etc/ is
>> configuration at least.
>
>The RPM database is as static as any other file in /usr. It is only
>modified, if you modify files in /usr.

So that must have been the reason we had --sharedstatedir=/usr/com! ;-)


-------------------------------------------------------------------
openSUSE:Factory/rpm/rpm.changes
Wed Jul  5 16:28:46 CEST 2017 - [hidden email]

- Define %_sharedstatedir as /var/lib, which is the path for
  shared state content in Red Hat/Fedora; Mageia; and Debian/Ubuntu.
  The old path (/usr/com) isn't recognized by FHS, whereas /var/lib
  is recognized as suitable for this purpose.
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: RFC: Proposed relocation of /var/lib/rpm

Jan Engelhardt-4
In reply to this post by Thorsten Kukuk

On Wednesday 2017-10-04 16:27, Thorsten Kukuk wrote:
>>
>> And will it make it into FHS 3.1, or has everyone given up on the notion
>> of the LSB?
>
>LSB is dead. There will be no new LSB version anymore.
>I doubt that there will ever be a new FHS version, too.

It may not be called FHS. It just takes someone to post something like a
standard. And it looks like systemd has been active in that area, having given
the world new paths (with justification) and continuing to do so.
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: RFC: Proposed relocation of /var/lib/rpm

Anton Aylward-2
In reply to this post by Carlos E. R.-2
On 04/10/17 07:38 AM, Carlos E. R. wrote:
> On 2017-10-04 13:16, Richard Brown wrote:
>> For systems with a read-write rootfs, we will be slightly bending the
>> rules of the FHS, in the sense that the FHS claims /usr should be
>> 'read-only data'
>
> What about machines that share /usr from a central machine over the network?

Indeed!
This was the original purpose of /usr/share and many today still use it that
way, consolidating centrally thing like the manual pages that apply EQUALLY to
all machines.

If you want a BtrFS rollback location /etc is better.



--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: RFC: Proposed relocation of /var/lib/rpm

Richard Brown
On 4 October 2017 at 16:38, Anton Aylward <[hidden email]> wrote:

> On 04/10/17 07:38 AM, Carlos E. R. wrote:
>> On 2017-10-04 13:16, Richard Brown wrote:
>>> For systems with a read-write rootfs, we will be slightly bending the
>>> rules of the FHS, in the sense that the FHS claims /usr should be
>>> 'read-only data'
>>
>> What about machines that share /usr from a central machine over the network?
>
> Indeed!
> This was the original purpose of /usr/share and many today still use it that
> way, consolidating centrally thing like the manual pages that apply EQUALLY to
> all machines.

And in that situation, how do you ensure that the manual pages you're
sharing are accurate for the version of binaries on the machines?
(especially on openSUSE where we have lots of binaries that aren't in
/usr atm).
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: RFC: Proposed relocation of /var/lib/rpm

Anton Aylward-2
In reply to this post by Richard Brown
On 04/10/17 08:05 AM, Richard Brown wrote:

> On 4 October 2017 at 13:38, Carlos E. R. <[hidden email]> wrote:
>> On 2017-10-04 13:16, Richard Brown wrote:
>>> For systems with a read-write rootfs, we will be slightly bending the
>>> rules of the FHS, in the sense that the FHS claims /usr should be
>>> 'read-only data'
>>
>> What about machines that share /usr from a central machine over the network?
>
> Putting the rpmdb in /usr/share is no worse than the current situation
>
> Right now machines sharing /usr from a central machine over the
> network are going to have an rpmdb that will contain an invalid
> picture of the contents of /usr


That's a ridiculous straw-man argument, Richard.

/usr/share was INTENDED quitre clearly and is there in the file ssytem
definition/description to that end.

No such for /usr

>
> If the share is read-only the rpm install will likely fail.

100% true.
And /usr/share being shared across machines, NFS or CIFS, is going to be read
only for most of them.

Heck, even when we had the old SU way of  'hoteling' and pretty dumb
workstations where almost everyting was NFS, particualr the /home/$USER, it was
the /etc/ that was ALWAYS machine-local and machine-specific.


> With this proposed change any user with root access to a machine
> accessing a shared /usr will change the contents of /usr and update
> the (now shared) rpmdb

Which is why it should NOT be under /usr/share specifically.
And why the database should be in a non-shared, machine specific location.

--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: RFC: Proposed relocation of /var/lib/rpm

Mathias Homann-2
In reply to this post by Richard Brown
Hi,


Call me stupid, but why do we absolutely have to have btrfs? I mean I
really do not know enough about what BTRFS can do and what it can't do
to have any well funded opinion on that matter... but from following the
mailing lists I am getting the feeling that one of the biggest
"features" of btrfs is that it runs out of space quickly (snapshots)
when people get lots of updates often (TW, Plasma, Gnome from OBS).


I do believe that "we are not Red Hat" wouldn't really be the best reason.

Cheers,

Mathias


Am 04.10.2017 um 13:16 schrieb Richard Brown:

> Hi all,
>
> Currently, rpm stores its rpmdb (the record of all a systems packages)
> in /var/lib/rpm
>
> Various openSUSE & SUSE distributions have a number of problems with this
>
> This has a major impact on our default btrfs snapshot & rollback feature.
> We need the contents of the rpmdb contained within the snapshots.
>
> This means /var/lib/rpm must be contained in the root snapshot in
> order to preserve the "system-state";
> Without it a rolled-back system still thinks it has packages
> installed/removed which were changed as a result of the rollback.
>
> But /var and /var/lib both contain a great deal of data which we DO
> NOT want to include in the root snapshot;
> For example, we most certainly don't want snapper rolling back a
> database in /var/lib/mysql for example, and destroying user data in
> the process.
>
> Therefore we currently carry a rather complicated default list of subvolumes:
> https://en.opensuse.org/SDB:BTRFS#Default_Subvolumes
>
> This list contains a number of specific directories in /var/* we wish
> to preserve, which means it's almost continually out of date, leading
> to user data being at risk if we haven't made a specific subvolume for
> that data.
>
> The rpmdb is the only "system-state" information we have in /var which
> we need to preserve, making it quite a disruptive presence in the /var
> filesystem hierarchy.
>
> The problem is further complicated in *SUSE distributions that use a
> Read-Only root filesystem (eg. openSUSE Kubic & SUSE CaaS Platform)
> because it results in a situation where we really need to micromanage
> countless subvolumes under /var - any location we miss otherwise ends
> up read-only and packages can't write to the files they expect to be
> able to in /var
>
> Red Hat had a similar problem with their rpm-ostree tooling, which
> they solved by relocating the RPM database to /usr/share/rpm
>
> https://rpm-ostree.readthedocs.io/en/latest/#why-not-implement-these-changes-in-an-existing-package-manager
>
> I would like to propose we adopt the same location for rpmdb in all
> *SUSE distributions
>
> This would mean we could make a single /var subvolume and dramatically
> simplify our btrfs configuration
>
> For systems with a read-only rootfs, /usr/share/rpm would be immutable
> and pristine, just as we require it to be. Our tooling should already
> be able to handle the new location without any major issues.
>
> For systems with a read-write rootfs, we will be slightly bending the
> rules of the FHS, in the sense that the FHS claims /usr should be
> 'read-only data'
>
> http://www.pathname.com/fhs/pub/fhs-2.3.html#THEUSRHIERARCHY
>
> However, installing packages routinely modifies the contents of /usr
> and it's various subdirectories, that is the nature of installing
> packages.
>
> Therefore it seems logical to me that the record of what is installed
> is also stored under the same conditions.
>
> I'm working on a patchset in an OBS home project to test this proposal
> and shake loose any issues, and will also be proposing this location
> change to rpm upstream.
>
> Therefore any comments are not only welcome, but the quicker the
> better, so I can factor them into what I'm doing.
>
> I would also like suggestions as how best to handle the migration of
> an existing rpmdb in /var/lib/rpm to it's new location in
> /usr/share/rpm, as I'm certain this is something we want to get
> absolutely perfect, and I'm understandably nervous about changing the
> rpmdb's location during an rpm transaction which will be altering the
> rpmdb.
>
> Thanks in advance,

--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

123