shazbot

joined 1 year ago
7
submitted 3 months ago* (last edited 3 months ago) by shazbot@kbin.social to c/kbinMeta@kbin.social
 

KES is the Kbin Enhancement Suite, a userscript/extension for (k/m)bin that provides a variety of customizable tools for users.

The last minor version included an initial attempt at a spam filter. This was chiefly done to address the low-hanging fruit of spam and scrub the most persistent pharma ads, etc. The approach was similar to that used by Ublock Origin or Steven Black's hosts file, in that it was a monolithic list of filter rules.

This was alright as a stopgap measure, but to the surprise of no one, the types of spam that continued to appear were innumerable in variety.

In an effort to have some filtering rather than none, the feature also went against the KES dictum about giving users choice to tweak their own settings: it was an all-or-nothing filter.

This update introduces v2 of spam filtering. The old logic has been retired, but may make a comeback at some point. A new sidebar page titled "Spam" has been added, and this will be a central place for anti-spam features.

The first of these efforts is a new filter that exposes the following options:

  • Hide posts from very new accounts

Users commented that there is no "gating" mechanism with registration, causing bogus accounts to be constantly recreated as a form of ban evasion. If you find that very new accounts have a high tendency to be spam, you can tick this setting to remove posts from very new users from the feed.

This does not block the user outright, merely hiding the post. This effectively gives some "break-in" time for the user to prove themselves. After a certain threshold, the user will fall outside of the "new account" window; if they continue to engage in spam-like behavior at that time, they should be caught by the other ban filters below.

  • Hide posts with abnormally low relative rank

Users with aggregate posting activity that does not necessarily resemble spam at first glance, but which periodically post spam/off-topic/controversial threads in the wrong communities, have a high likelihood of being astroturfers. This option hides such posts from the feed if the relative vote weight is egregiously outside the norm. Like the above, it does not block them outright.

  • Block users with spam-like activity

This feature blocks users outright if they engage in spam, mass posting, and other robotic behavior. It is less permissive than v1 of the spam filters, which failed to catch a lot of spam that was not explicitly blacklisted in the filter list. This new approach is more generic and does not need constant updating of the spam filters.


By default, all three options are enabled. I find this combination gives the best results so far. Note that infinite scrolling should be enabled for best results, and it may take a little while when navigating to a magazine for the results to be filtered.

Give it a try and see if this negates the spam issue or improves your experience. What other filter options would you like to see added?

The remainder of the changelog follows:

Add-ons


Filter advertisements (@shazbot)

Location: Spam > Filter advertisements

Updated filtering logic to v2, as described in the preamble of this post.

Collapse pinned posts (@shazbot)

Location: Threads > Collapse pinned posts

This simple feature groups together pinned posts on the magazine index and collapses them by default. Click the toggle area at the top of the magazine index to expand pinned posts. This is especially useful on magazines with many pinned posts that are not regularly archived, such as /m/kbinmeta.

API


Added the helper function isThread() to the API. This returns true if the current window location is a thread inside of a magazine. This can be used to abort if the feature is intended to only apply on the thread index. In the future, a more expanded function will be provided that returns the type of page from a list of enums.

[–] shazbot@kbin.social 3 points 4 months ago

If your main goal is exporting your magazine subscriptions between accounts across instances, may I suggest trying EXIT tool. If you are looking for more complex export settings (friends/favorites?), unfortunately, only subscriptions are supported at this time.

[–] shazbot@kbin.social 2 points 4 months ago

A few suggestions that may or may not be satisfactory for you:

Using KES, enable General > Hide sidebar elements > Random threads, Random mags, Random posts. The randomly populated sidebar is fundamentally flawed; I suggest disabling its content altogether.

Next, enable General > Filter advertisements. This second feature is by no means foolproof, but will reduce a lot of noise, and is periodically updated on a rolling basis.

[–] shazbot@kbin.social 5 points 4 months ago

Using this approach, I am seeing none of those posts on /science. I updated the filters a bit today. The top post is a legitimate article from 2024-04-13 and is by HeartyBeast.

Now, I understand that this is seen as an unnecessary step (too fancy) for some. People want zero ads out of the box without anything extra. So I'm thinking about the next approach here.

Framing the problem:

  • Filtering should be automatic
  • End-user wants zero additional setup
  • There is no active upstream development
  • It's not possible to inherit moderation of a magazine due to some queue of moderator application requests that is not being approved

The third point and fourth points are important here, since that's currently intractable. You can't reconcile zero additional setup with that.

But let's suppose becoming moderator of a defunct magazine (point 4) were possible while point 3 remained unresolved. In other words, at least moderators can try to pick up the pieces. Something being underestimated here is how annoying it would be for the moderator to manually cull posts every single day. I think you would have instant turnover after a couple of weeks once the tedium sets in. Manual solution is not good. Clearly, automation is needed on the moderation side.

So assuming you could actually inherit a magazine, but with no guarantee of upstream development, what about restructuring the tool above so that it's for moderators, instead of end-users? That's pretty easy, and I could make it something the moderator clicks once and it's done, auto-banning the posts. This is a pretty good method.

But you can't inherit moderation right now, so that's back to square one.

Realistically, that leaves these options at the moment:

  • Wait (a long time) and see
  • Use the tool above and make magazines readable, albeit at some sacrifice of convenience (?)
  • Migrate to another instance

Third approach is the path of least resistance and is best for most casual users. Second is for diehards who cannot move instances due to some personal or technical reason. First approach is the most annoying and eventually leads to the third approach after frustration sets in.

Pick your poison, I guess. I can't think of any other prophylactic approach at the moment, maybe this comment triggers some idea.

[–] shazbot@kbin.social 1 points 4 months ago* (last edited 4 months ago)

If the script can automatically block any user whose post it suppresses, it would be awesome.

It does! I've reworded the OP to hopefully make that clearer. After using this approach for a few days, my blocklist (generated entirely programmatically) is ten pages long, and there is nary a bad post in sight. I'm expanding the filters on a daily basis.

I think the auto report function is severely needed; it's happening everywhere.

The idea is that it takes the burden off of myriad (N) users having to manually do this themselves, and lets a single user (the KES custodian) prepare the filters, which then propagate out to any user of KES. Instead of 1,000 people manually blocking, one person builds the heuristics, and everyone benefits.

Preventing this issue doesn't seem like a userscript issue...but I think the issue is that we need to get support top-down on this.

I understand, but the stated goal of KES is addressing issues that can't, or won't (due to some design conflict), be addressed, or which fall through the cracks. At the moment I'm seeing a lot of people voicing frustration, but due to the skeleton crew situation with administration of the site, it seems like screaming into the void. Not that there's anything wrong with that, and hopefully it gets some traction. But my job with KES is just to provide fixes for the end-user, albeit of a third-party nature.

[–] shazbot@kbin.social 1 points 4 months ago (1 children)

I understand if you don't want to add yet another third-party tool just to browse the site, but if you feel at all inclined, give it a try. For me 99% of magazines went from unusable to essentially clean.

[–] shazbot@kbin.social 2 points 4 months ago

I don't read magazines in-depth much either, so I wasn't aware of the extent of the issue at first, but I was appalled at what I saw. I agree with you that it creates a negative impression for anyone wanting to venture into or use a magazine. I know that needing KES upfront may be a non-starter for some people, but for me the difference with filtering is night and day now.

[–] shazbot@kbin.social 1 points 4 months ago

@bayaz @insomniac_lemon

Yep, I meant to say /books when I was talking about Amazon links. Sorry.

As for the posts on /food, I have totally vanilla settings (no language filtering or anything) and I saw all of the posts you both mentioned.

Well, with the exception of the garbage posts, because in answer to lemon's earlier question, those have long since been scrubbed on the filter side. But they did exist before I started filtering.

All of that low hanging fruit kind of stuff has been banished, and I am mainly working on edge cases at this point.

[–] shazbot@kbin.social 2 points 4 months ago (15 children)

Of course, I'm not trying to suggest that a third-party prophylactic tool is a definitive solution to what is ultimately a separate problem, just trying to be pragmatic here and restore basic readability for end-users, whether the filtering is done at the source or after the fact.

Let's be real here, we are talking about unmoderated magazines on an instance where the developer is AWOL and using a framework that is lacking many basic features. Even with moderators, manual moderation can be a big ask and is time-consuming for free volunteers, depending on the volume of posts or how rudimentary the moderation tools are.

I actually don't read kbin magazines much, so I wasn't aware of the extent of the problem until I started opening those magazines more closely, and felt that something is better than nothing.

On the magazines you mentioned, I do see a few anomalous patterns that I'll start filtering. For the most part, with filtering enabled, they were almost entirely free of garbage, save for a few patterns I may have missed on the first few passes. /programming and /food I need to take a deeper look at. The /food thing is good intel, because the use of Amazon referral links in the threads is something that can be generalized to other situations beyond books. Posting referral links is definitively block-worthy.

I also noticed some stuff that by any other name would be considered a thinly-veiled ad, such as specific users only posting articles to web sites they own and operate. I'm not talking about bots as such, but actively promoting one's own content--even when such content is on-topic for the magazine. I declined to filter this stuff yet, because it received a lot of upvotes and seemed to be received favorably, maybe because the readers felt it was at least germane to the topic at hand? I think this is probably true for /food as well, because the line between "content" and "promotion" is unclear here, since what is a food blog if not a product generating click revenue? It seems like the tolerance threshold for that sort of thing is higher in a magazine like /food versus some other magazine. Anyway, I digress. I'm not treating such stuff as in scope, just filtering what is blatantly noise.

[–] shazbot@kbin.social 2 points 4 months ago (17 children)

You don't like bufo toad venom? I like to start my mornings by sipping a little bufo toad venom while reading kbin. Buy bufo toad venom today.

In all seriousness...I don't know if you saw my last thread about KES in this magazine, but I suggest giving it a try. I've extended the filter coverage based on your feedback, and those magazines should essentially be expunged of garbage for the time being.

As for the sidebar, I believe the implementation is fundamentally flawed because it loads content that, AFAIK, doesn't respect your actual block settings. I suggest disabling the random threads element altogether in KES by navigating to General > Hide sidebar elements.

[–] shazbot@kbin.social 2 points 4 months ago (20 children)

Could you point me to some of the magazines where you feel this is particularly rampant right now?

[–] shazbot@kbin.social 1 points 4 months ago (3 children)

How do you typically update your KES?

 

The blurb below is excerpted verbatim from the release notes. For the full release notes, see here.

Repository: KES

Many of you are aware of the "canned meat" problem on kbin.social, with some magazines being inundanted with garbage posts.

The latest version of KES ships with an experimental new feature you can enable that attemps to filter these posts and block the users who posted them based on certain heuristics.

This feature is experimental, but I see a lot of users voicing frustration at the problem, so now seems like a good time to start collecting feedback. You can start using this feature immediately and it should not have any adverse effects, but its coverage is still being expanded.

You can find it under General > Filter advertisements. For best results, it should be used in conjunction with infinite scrolling enabled in the kbin sidebar, so that new content is loaded in as posts are removed.

As you navigate through a magazine, KES will remove offending posts from the index and then permanently block the user of the post. This feature is also preventive, as variations of posts made under different usernames will continue to be flagged. The goal is to avoid the tedious process of "whack-a-mole" and cull these posts without manual intervention.

Initially, KES will be removing posts from the index, but as it builds your blocklist up for you, such posts will stop appearing in the thread index altogether, and you should see the overall signal to noise ratio improving. Outside of your blocklist, subsequent posts that meet certain criteria will continue to be culled regardless or when or where they appear.

I am currently using /m/science and /m/opensource as a control. If you navigate to those magazines and compare the results before and after enabling this feature, the difference should be clear. After enabling the feature and scrolling all the way back to 2023, there should be few if any unsolicited ads on the page.

Hopefully this improves readability and encourages participation in communities that otherwise seemed impenetrable at first glance. In fact, once you scrub the garbage posts, you'll be surprised to find that there are legitimate posts being made fairly frequently in these seemingly "dead" communities--the posts were just buried in the heap.

However, canned meat comes in a lot of different flavors, and each magazine has slightly different permutations. The coverage in this initial version is not exhaustive, but it attempts to be thorough. This should greatly cut down on the most annoying ads. If there are specific (most likely unmoderated) magazines you are still having a problem with, please leave a comment listing the magazine. You don't need to point to specific posts or users; the magazine name is enough here for me to analyze what kinds of posts are appearing.

Some additional notes:

  • For the time being, this feature does not report the post to the magazine's moderator (usually nonexistent). By kbin's design, a post can only be reported at most by a single user, so this seemed like a reduplication of efforts to me. But auto-report can be added if necessary.
  • This feature works on any instance, but is chiefly designed for kbin.social and is probably unnecessary elsewhere.
  • I have not taken a look at microblogs yet, so I don't know if this problem is happening there, too (please let me know). For now, this works on the thread index of magazines.
  • For best results (if you want to quickly bootstrap your blocklist), I suggest enabling the feature and scrolling through an affected magazine for awhile with infinite scroll on to build up the blocklist as new posts load in, then refreshing the page if necessary.
  • The "random threads" sidebar is fundamentally flawed because it shows content even if you've already blocked it. So I recommend enabling General > Hide sidebar elements > Random threads in conjunction with this feature.
[–] shazbot@kbin.social 1 points 5 months ago

Addressing your issue, I have bumped the version number to 0.1.3 and made a change to the async method handling so that instances not available at the remote get added to the fail log correctly.

This doesn't explicitly address the fact that some instances are unfederated, but it will make the log results clean.

As for the federation issue, what I've initially found is that a user on an instance has to visit the remote instance for the home instance to be aware of this remote instance, and a user (could be a different user) has to subscribe to that instance for the posts to start federating. What is unclear is how a user on an instance visits a remote instance from the home instance, as this is implementation-specific and could vary from instanc to instance.

 

KES is the Kbin Enhancement Suite.

This is just an update to note that the tool has undergone a total quality audit to ensure that all of its add-ons support both kbin and mbin instances and behave in an expected fashion regardless of which instance type you are using.

If you had previously tried KES on an mbin-type instance and encountered anomalous behavior, you should now find that it has full compatibility.

The full release notes can be found here

 

"EXIT" -- Export Across Instances Tool

This is a simple and self-contained tool that helps automate the process of exporting your magazine subscriptions from one instance to another, provided you have accounts on both.

Could also be used to copy subscriptions from one named account to another named account on the same instance, or to back them up for later.

Instructions and tool available here

Code runs locally in your browser only.

view more: next ›