Wiki status update

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Wiki status update

Christian Boltz-5
Hello,

good news: after a month of hunting a) spammers and b) server admins,
the needed wiki extensions to prevent and delete wiki spam finally got
installed and the database updated so that those extensions can work.
(Done in the english wiki, the others will follow.)

This means the wiki is editable again :-)

It also means we now have an easy way to block spam, and can mass-delete
existing spam pages.

As every good news, there's also a bit of bad news - the database update
cleared all cache, which means the wiki is terribly slow at the moment.

BTW: If I accidently delete something that was not spam while doing the
mass deletion, please tell me ;-)

For those who are interested in details:
- the Nuke and AbuseFilter extensions were installed
- I just added the first spam filter - let's see how it works out (and
  tell me if it accidently produces false positives [1])
- I just started to delete all the spam. If someone has admin
  permissions on the english wiki and wants to help, head over to
  https://en.opensuse.org/Special:Nuke and enter something like   %1_8%
  as page title search term. It uses MySQL LIKE syntax, as a regex the
  example means   .*1.8.*   (the "all pages" list will give you more
  spam page titles if you sort by creation date)
- If you are interested in helping with AbuseFilter rules, see
  https://www.mediawiki.org/wiki/Manual:Combating_spam/AbuseFilter_examples and
  https://www.mediawiki.org/wiki/Extension:AbuseFilter#Documentation_and_management
  for documentation.


If you reply to this mail, please answer only to one list. Cross-posting
the announcement is enough ;-)


Regards,

Christian Boltz

[1] Funnily I was the first victim of the spam filter ;-)

    I didn't exclude the 'delete' action on the first attemp. When I
    tried to delete pages with spammy title, this was denied because
    the page title matched the spam filter ;-)

    Lesson learned: I need to exclude the 'delete' action from the filter ;-)


--
For Geralds problem "rpm -e digikam-doc" would be the solution: nobody
is going to read 19.3MB of documentation anyway, given that people do
not even read short README's :-P [Stefan Seyfried in opensuse-factory]

--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Wiki status update

Christian Boltz-5
Hello,

Am Dienstag, 29. März 2016, 22:24:16 CEST schrieb Christian Boltz:
> good news: after a month of hunting a) spammers and b) server admins,
> the needed wiki extensions to prevent and delete wiki spam finally got
> installed and the database updated so that those extensions can work.

Two weeks later, I have another update - hopefully the final one on this
topic.

We deleted all spam from the english wiki, and have set up filter rules
(using the AbuseFilter extension) to prevent new spam.

"All spam" means: we had to delete 6612 spam files, 513 pages in the main
namespace and and about 50 pages spread over the other namespaces.

I had the most fun with deleting about 5000 pages and files, and Micah
(one of the Admins in Provo who is also a good Mail/IRC to SSH relay ;-)
deleted the remaining 2200 spam files after cleaning up the recentchanges
table (see the "Nuke" section below).


While working on this, some funny things happened - as usual when I use
or touch some code the first time, it breaks into parts ;-) and someone
needs to fix it.

AbuseFilter was my first victim - it worked fine for blocking spam pages,
but it didn't block uploading of spam files with similar titles.
It turned out that AbuseFilter used a wrong variable name for file
uploads, making it impossible to filter for upload filenames. This was
fixed (after I poked some people on #mediawiki) in
    https://gerrit.wikimedia.org/r/#/c/281234/2
I applied this fix to the openSUSE wiki.

This bug might also explain why the spammers prefer file uploads - lots
of wikis around the world don't have this fix yet and therefore make it
easy to upload spam files. (Dear spammers, sorry for finding this bug and
making your life harder *g,d&r*)


My second victim was the Nuke extension - after deleting some of the
spam, I noticed that it offered the same files for deletion over and over
again, even if they were already deleted. (This happened only for files,
not for "normal" pages.)

It turned out that Nuke works based on the "recentchanges" table. This
works fine for normal pages because Mediawiki deletes their "create"
event from recentchanges when you delete them. However, it doesn't do
this with "upload" events when deleting a file. (I'll re-check this
behaviour with a newer version of MediaWiki, and open a bugreport if
it's still the same.)

As a workaround, we used a manual delete query on the recentchanges
table. Not too nice, but it worked.


> (Done in the english wiki, the others will follow.)

Micah deployed the Nuke and AbuseFilter extensions on the non-english
staging wikis (for example destage.o.o), and will push them to all
production wikis next week.

This means the admins of the non-english wikis will be able to use
Special:Nuke and Special:Abusefilter soon. (See my previous mail on this
topic for some links and instructions.)


My request from the lat mail still stands:

> If you reply to this mail, please answer only to one list.
> Cross-posting the announcement is enough ;-)


Regards,

Christian Boltz
--
Wenn derjenige hinterher herumjammert, "Zwar hängt jetzt das Bild, aber
ich habe ein Loch in der Wand und ein Nagel steht hervor...", dann habe
ich große Zweifel daran ob es so gut war, dass derjenige einen Hammer
und Nagel in die Hand bekommen hat. [Igor Sverkos in postfixbuch-users]

--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [opensuse-project] Re: Wiki status update

PatrickD Garvey
On Sun, Apr 17, 2016 at 3:28 AM, Christian Boltz <[hidden email]> wrote:
> Hello,
>
> Micah deployed the Nuke and AbuseFilter extensions on the non-english
> staging wikis (for example destage.o.o), and will push them to all
> production wikis next week.
>

Nuke returns "Permission error" because "The action you have requested
is limited to users in the group: Administrators."
but
AbuseFilter returns "Welcome to the Abuse Filter management interface."

That seems a bit inconsistent.
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [opensuse-project] Re: Wiki status update

Christian Boltz-5
Hello,

[only replying to -wiki because this is a completely "technical" mail]

Am Dienstag, 19. April 2016, 16:08:13 CEST schrieb PatrickD Garvey:

> On Sun, Apr 17, 2016 at 3:28 AM, Christian Boltz wrote:
> > Micah deployed the Nuke and AbuseFilter extensions on the
> > non-english
> > staging wikis (for example destage.o.o), and will push them to all
> > production wikis next week.
>
> Nuke returns "Permission error" because "The action you have requested
> is limited to users in the group: Administrators."
> but
> AbuseFilter returns "Welcome to the Abuse Filter management
> interface."
>
> That seems a bit inconsistent.

Not really ;-)

If Nuke would be public, you could use it to search for pages matching a
specific pattern, namespace and/or username. Maybe it could even display
a list of matching pages to you, but a) that's not too useful because
you can get similar list via special:allpages, special:contributions and
special:recentchanges and b) the main action is to _delete_ those pages,
which only admins can do.

AbuseFilter allows non-admins to view the filters and the filter log,
which is useful for admins of other wikis who might want to "steal" a
filter, for users who want to find out why one of their edits got
accidently blocked - and for statistics fans because you can also see
the matches each filter produced (unless a filter gets marked as non-
public). So it's much more useful for non-admins - the only thing they
can't do is editing the filters.

As a sidenote - the "Notes: (private)" field of each filter isn't as
private as the label might indicate - it's visible without being logged
in...


Regards,

Christian Boltz
--
> Naja das ist hier ziehmlich OT aber ich werde
> trotzdem mal meinen Senf hinzufügen.
Senf? Beleidige nicht diese tolle Gewürzpaste, ja. ;-)
[> "mrgates" und Matthias Houdek in suse-linux]

--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [opensuse-project] Re: Wiki status update

PatrickD Garvey
On Wed, Apr 20, 2016 at 3:25 PM, Christian Boltz <[hidden email]> wrote:

> Hello,
>
> [only replying to -wiki because this is a completely "technical" mail]
>
> Am Dienstag, 19. April 2016, 16:08:13 CEST schrieb PatrickD Garvey:
>> On Sun, Apr 17, 2016 at 3:28 AM, Christian Boltz wrote:
>> > Micah deployed the Nuke and AbuseFilter extensions on the
>> > non-english
>> > staging wikis (for example destage.o.o), and will push them to all
>> > production wikis next week.
>>
>> Nuke returns "Permission error" because "The action you have requested
>> is limited to users in the group: Administrators."
>> but
>> AbuseFilter returns "Welcome to the Abuse Filter management
>> interface."
>>
>> That seems a bit inconsistent.
>
> Not really ;-)
>
> If Nuke would be public, you could use it to search for pages matching a
> specific pattern, namespace and/or username. Maybe it could even display
> a list of matching pages to you, but a) that's not too useful because
> you can get similar list via special:allpages, special:contributions and
> special:recentchanges and b) the main action is to _delete_ those pages,
> which only admins can do.
>
> AbuseFilter allows non-admins to view the filters and the filter log,
> which is useful for admins of other wikis who might want to "steal" a
> filter, for users who want to find out why one of their edits got
> accidently blocked - and for statistics fans because you can also see
> the matches each filter produced (unless a filter gets marked as non-
> public). So it's much more useful for non-admins - the only thing they
> can't do is editing the filters.
>
> As a sidenote - the "Notes: (private)" field of each filter isn't as
> private as the label might indicate - it's visible without being logged
> in...
>
>

Thank you.
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]